For some reason this also requires a fix in pmb.install._install.py,
didn't bother investigating why but the necessary change seems fine to
me.
Requires mypy 1.14 (maybe 1.14.1, not sure).
There was an oversight when this API was originally created and it
implicitly assumed that the pmaports repository was always named
"pmaports". This unfortunately broke some peoples workflows.
Introduce a new "pkgrepo_name()" function and adjust the codebase to use
it, as well as adjusting pkgrepo internally to special case the
"pmaports" repo so that it's always named pmaports no matter what the
directory itself is named.
This is probably more complexity than we should be dealing with here, we
should probably create a new type to encode this behaviour.
Fixes: #2412
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
Since master_staging_systemd doesn't have its own channel, handle the
special case where the user ran pmbootstrap init while on the systemd
branch, with systemd enabled. In this case it's probably unintuitive to
switch pmaports to the master branch.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Fix for:
$ pmbootstrap pull
[21:09:43] aports_upstream (branch: 3.20-stable): not on one of the official branches (master), skipping pull!
Replace the get_branches_official() function that according to git log
was never able to tell if an aports branch was official. The new
function branch_looks_official() can do that by just checking if the
branch follows the typical naming pattern.
The comment I had put into get_branches_official earlier was not true
anymore, by now this did not only get used by "pmbootstrap status", but
also by "pmbootstrap pull".
Generalise pmb.helpers.other.cache with a more python decorator.
The Cache decorator takes a list of function arguments to use as cache
keys, keyword args can be used to restrict caching so that it is skipped
entirely unless the attribute has a specific value.
For example, pmb.helpers.pmaports.get() has the decorator:
@Cache("pkgname", subpackages=True)
This means the return value will be cached only when subpackages is
True, otherwise it will always miss.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Use a different binary repo depending on the source repository for the
package. This makes it possible to split out systemd packages into their
own repository.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.
This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
It's been 6(?) years since pmaports moved out of the pmbootstrap repo.
Let's finally drop these checks.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
With the new chroot type, we can now write fancy paths in the pythonic
way. Convert most of the codebase over, as well as adding various other
type hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We use a custom verbose log level in pmbootstrap, unfortunately it isn't
possible to correctly type this due to some limitations in the logging
library [1], [2].
Given that our usecase is fairly simple, we can just wrap the module
with our own so we only have to tell mypy to ignore the error once
instead of at every callsite.
[1]: https://github.com/cryptax/droidlysis/issues/15
[2]: https://github.com/python/typing/discussions/980
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The init_cache() function just assigned some default constants, simplify
this by just declaring it that way to begin with, as well as adding type
hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a new module: pmb.core to contain explicitly typed pmbootstrap
API. The first component being Suffix and SuffixType. This explicitly
defines what suffixes are possible, future changes should aim to further
constrain this API (e.g. by validating against available device
codenames or architectures for buildroot suffixes).
Additionally, migrate the entire codebase over to using pathlib.Path.
This is a relatively new part of the Python standard library that uses a
more object oriented model for path handling. It also uses strong type
hinting and has other features that make it much cleaner and easier to
work with than pure f-strings. The Chroot class overloads the "/"
operator the same way the Path object does, allowing one to write paths
relative to a given chroot as:
builddir = chroot / "home/pmos/build"
The Chroot class also has a string representation ("native", or
"rootfs_valve-jupiter"), and a .path property for directly accessing the
absolute path (as a Path object).
The general idea here is to encapsulate common patterns into type hinted
code, and gradually reduce the amount of assumptions made around the
codebase so that future changes are easier to implement.
As the chroot suffixes are now part of the Chroot class, we also
implement validation for them, this encodes the rules on suffix naming
and will cause a runtime exception if a suffix doesn't follow the rules.
Add the following question to "pmbootstrap init":
[22:12:57] Based on your UI selection, 'default' will result in installing systemd.
[22:12:57] Install systemd? (default/always/never) [default]:
Determine whether the UI prefers to have systemd or not, based on
"pmb:systemd" in the UI package's APKBUILD.
Determine whether the currently selected branch supports systemd, by
checking for a "[repo:systemd]" section in pmaports.cfg. This section
will also contain bootstrap information, to be used in future patches.
When running a pmbootstrap command and the pmaports dir does not exist,
it would print the following error:
WARNING: Can't create log file in '/home/user/.local/var/pmbootstrap', path does not exist!
[02:14:37] ERROR: We have split the aports repository from the pmbootstrap repository (#383). Please run 'pmbootstrap init' again to clone it.
[02:14:37] See also: <https://postmarketos.org/troubleshooting>
Run 'pmbootstrap log' for details. Alternatively you can use '--details-to-stdout' to get more output, e.g. 'pmbootstrap --details-to-stdout init'.
This is long obsolete. By now the reason for the missing pmaports dir is
not that we had moved it from pmbootstrap.git to a separate git
repository, it's most likely that the user did not run
'pmbootstrap init'. Adjust the message, and quit without link to
troubleshooting etc:
[02:15:28] ERROR: pmaports dir not found: /home/user/.local/var/pmbootstrap/cache_git/pmaports
[02:15:28] Did you run 'pmbootstrap init'?
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230408002810.2006-1-ollieparanoid@postmarketos.org%3E
So that githooks can be used in pmaports without the risk that symlinks
pose when checking out random branches from random people. The hooks
are copied, only when doing "pmbootstrap init", selecting the edge
channel and having the master branch checked out (not any other branch
on the edge channel or on any other channel).
Closes: pmaports#2055
Related: pmaports!4008
Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Co-developed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3CDB9P192MB129107CAA025F4EF59D3FB53C7999@DB9P192MB1291.EURP192.PROD.OUTLOOK.COM%3E
Replace "args.cache" with a global variable in order to
avoid passing "args" to all functions. This is a step to get rid of this
args-passed-to-all-functions pattern in pmbootstrap.
Translate the pmaports channels "stable" to "v20.05" and "stable-next"
to "v21.03", so these have the same channel name as the pmaports.git
branch name.
The original plan was to switch the "stable" channel from the "v20.05"
branch to the "v21.03" branch when the release is done. However, now
that we are close to that, I'm realizing that this would not be useful.
It would lead to conflicts in the dir with locally built packages
(default: ~/.local/var/pmbootstrap/packages/$CHANNEL). And it would make
it awkward to go back to a previous branch (we may name it old-stable
for the time being, but what after that, old-old-stable?).
Many of pmbootstrap's actions require root rights. When after requesting
sudo access pmbootstrap takes longer than the sudo timeout interval to finish
execution, the password will have to be entered again on the next sudo
action.
This change adds an opt-in feature to run sudo -v in a background loop
in order to prevent having to enter the password more than once for a single
pmbootstrap run. The loop runs as a daemon timer which automatically gets
canceled when pmbootstrap exits.
Closes: #1677
This made sense for Alpine's aports.git repository as we were only using
the master branch. But now that we are using more branches, we need the
entire git repository with all its branches cloned.
Migrate to workdir version 5 and move already built packages into the edge
channel subdir, for example:
$WORK/packages/x86_64/hello-world-1-r5.apk
to:
$WORK/packages/edge/x86_64/hello-world-1-r5.apk
The build.postmarketos.org code has already been adjusted to find built
packages in either directory structure.
Use mirrordir_pmos and mirrordir_alpine from channels.cfg to generate
the mirror URLs for postmarketOS and Alpine, which get written to
/etc/apk/repositories and which postmarketOS uses to download the
APKINDEX files.
Remove hardcoded "master" at the end of the postmarketOS mirror and use
mirrordir_pmos instead (which is "master" for the edge channel). Let the
postmarketOS mirror end in a '/' for consistency with the Alpine mirror
in pmb/config/__init__.py.
Remove obsolete --alpine-version. To experiment with a different Alpine
version, one should pass a custom --config-channels from now on.
Ask for release channel and switch pmaports branch to the related branch
defined in channels.cfg.
Store in pmbootstrap.cfg whether the user chose a channel (boolean). If
the user did not choose a channel yet, suggest the recommended channel
from channels.cfg (currently "edge").
Do not make the parsed pmaports.cfg from pmaports.git available as
args.pmaports anymore. This de-bloats the args variable a bit.
First I thought that we didn't even need to cache it, but it was pointed
out that later patches do access it frequently to read the current channel
from pmaports.cfg. Therefore it is using a cache now.
Related: #1879, #1855
While at it, also remove unnecessary "#!/usr/bin/env python3" in files
that only get imported, and adjust other empty/comment lines in the
beginnings of the files for consistency.
This makes files easier to read, and makes the pmbootstrap codebase more
consistent with the build.postmarketos.org codebase.
Add a shortcut for "git pull --ff-only" in all repositories cloned by
pmbootstrap (currently pmaports and aports_upstream, new pmdevices
repository coming soon).
'pmbootstrap pull' will only update the repositories, if:
* they are on an officially supported branch (e.g. master)
* the history is not conflicting (fast-forward is possible)
* the git workdirs are clean
Otherwise it shows the user a descriptive message about what to do. The
list of supported branches is only "master" right now, and will be
extended in later commits, so we can have a stable branch for pmaports
based on Alpine's releases. More about that in the project direction
2020 issue.
Closes: #1858
Do not install git in the native chroot and use it from there. Remove the
chown_to_user argument from pmb.helpers.git.clone(), the resulting dir
is now always owned by the user. While at it, refactor the function and
display the clone URL.
Previously we had cloned aports_upstream (from Alpine) with
chown_to_user=False (legacy) and pmaports with chown_to_user=True.
pmb.helpers.git.rev_parse() would only work after chown_to_user=True.
Check if git is installed in "pmbootstrap init", and remove the same
check from rev_parse(). Add a new work dir version, that checks for git
and changes ownership of already checked out aports_upstream to the
host system's user.
When creating a new work dir, create cache_git instead of cache_http.
cache_http is created on demand already, with proper permissions. But
cache_git must be created, otherwise pmb.helpers.mount.bind will create it
as root.
This is in preparation for the "pmbootstrap pull" feature, as it allows
using the host system's git in all new code paths. We will be able to
handle repositories even if they were cloned outside of the work dir
(which we do in a few CI scripts for example).
Related: #1858
