Now that we have moved the kconfigcheck configuration into pmaports
branches via kconfigcheck.toml, it is time to get rid of the hardcoded
list of valid "pmb:kconfigcheck-…" options for APKBUILDs. Generate it
from the kconfigcheck.toml of the current branch, too.
These options are passed from "pmbootstrap lint" to "apkbuild-lint",
which we run in pmaports CI.
The output of apkbuild-lint is hard to spot among other log messages
from pmbootstrap, because it is not colorized.
Add "*** apkbuild-lint output ***"" before and after the output of
apkbuild-lint, so we get a standing out message in green around the
apkbuild-lint output.
It is not needed to run the linter as root, so don't do it. This is
probably a leftover from early days pmbootstrap, where pmaports.git
wasn't chowned by the user running pmbootstrap.
When running some commands like "pmbootstrap chroot" or even
"pmbootstrap work_migrate", pmbootstrap will attempt to read the
pmaports.cfg from origin/master *before* doing the work dir migration.
In order to do this, the "get_upstream_remote" function tries to find
the upstream remote by URL. Let it search with the outdated URLs too, so
this doesn't fail right before migrating to the new URLs.
This could be re-purposed in the future in case we migrate URLs again.
Co-authored-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
Now that we don't need weird apk-tools hacks for systemd, we can
re-implement this optimisation and always run apk static rather than
running apk through the chroot.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
For packages selected by the user (see last commit), set the
provider_priority to 999999 instead of 999. The previous number seems
not high enough, we might set this in the APKBUILDs.
This function is used by "pmbootstrap repo_missing", which is only used
by bpo. In order to support building the split repository in bpo, this
patch is required.
Return "systemd" for the packages that are in extra-repos/systemd, and
None for all other packages (indicating that they don't get split into a
separate repository).
Checking the parent-parent dir of the APKBUILD is a bit fragile
(assuming we don't have subdirs in the systemd repository), but that
assumption was made with the previous implementation as well (would only
return "systemd" in that case). I don't see a better solution with
reasonable effort right now, given that we also support multiple
pmaports dirs. We can improve this in the future, meanwhile let's just
not use subdirs in the systemd repository.
Don't attempt to find packages of other arches in repo_missing code.
This leads to unexpected results in general (we want to check if a
binary package exists for only one specific architecture!) - and in case
of master_staging_systemd, it current causes errors because pmbootstrap
tries to fetch armhf, riscv etc. APKINDEXes for which we don't build the
staging repositories.
As chroot.glob() now returns a generator, convert it to a list first to
check if it is empty. This may not be the most efficient method, but
since we expect it to be empty here it is fine.
Without this patch, it always shows the error even if there are no
locally built packages:
ERROR: --no-local-pkgs specified, but locally built packages found. Consider 'pmbootstrap zap -p' to delete them.
When a dependency cannot be found in source or binary packages of the
selected architecture, then don't look for binary packages in other
architectures package indexes.
This leads to a confusing error down the line when building packages for
extra_repos_systemd (and e.g. stable branches), because we don't have
binary packages for all arches that pmbootstrap supports, and so it
tries to fetch a non-existing APKINDEX.
Instead, print a nice error about a non-existing dependency.
Get rid of 3 years and older migration code, so we don't need to
maintain it anymore. If the user should have run "pmbootstrap init" 3
years ago and suddenly upgrades to pmbootstrap v3, then they will just
need to delete their work dir (as pmbootstrap instructs to do) and start
over. This is fine, the work dir is just a bunch of caches and locally
built packages.
repo is a Path obj now, and its name is used as a key in the dest_paths dict.
Fixes this failure:
working_dir=dest_paths[repo],
~~~~~~~~~~^^^^^^
KeyError: PosixPath('/home/clayton/src/pmaports')
Remove arguments to set postmarketOS and Alpine mirrors.
The current implementation is broken with the way the mirrors are now
configured: The current implementation fills a list, but the mirror
configuration code expects a string. It is not longer possible to set
an arbitrary amount of mirrors for the postmarketOS mirror. The benefit
is that we don't have comma separated values in the config file anymore
but instead just simple strings. Now it is possible to have one proper
mirror for aports, pmaports and pmaports_systemd, and one _custom one
to override each of these.
We already have too many options in "pmbootstrap -h", and changing a
mirror is more of a base configuration that you do once and maybe change
a few times, but don't need to be able to set it with each pmbootstrap
run. Users are not going to write this out manually on their
command-lines, it only makes sense in scripts and wrappers for
pmbootstrap, and there you could as well use another method (as now
described in docs/mirrors.md) to set the mirrors.
Less important, but another reason is that using "-mp" (two letters as
short argument) isn't really elegant.
Fix writing config options that are stored as CSV-list. Without this
patch:
$ pmbootstrap config aports /tmp/pmaports
[23:43:58] Config changed: aports='/tmp/pmaports'
$ pmbootstrap config aports
[PosixPath('/'), PosixPath('t'), PosixPath('m'), PosixPath('p'), PosixPath('/'), PosixPath('p'), PosixPath('m'), PosixPath('a'), PosixPath('p'), PosixPath('o'), PosixPath('r'), PosixPath('t'), PosixPath('s')]
Storing strings as CSV list isn't great and we probably want to
implement this more elegantly, see the related issue. But let's fix this
first.
Related: pmbootstrap issue 2412
Allow setting _custom mirrors in the config:
* alpine_custom
* pmaports_custom
* systemd_custom
When these are set, they are added to /etc/apk/repositories before real
repositories. This is used by bpo to build packages with a WIP
repository enabled, in addition to the final repository.
All mirrors can also be set to "none" to be disabled. This is important
for bootstrapping from pure Alpine without any binary repository, and
the bpo testsuite also uses this.
I've discussed with Caleb whether to name it _wip instead of _custom,
but the latter is more generic and people may also use this for other
use cases than the bpo wip repository thing.
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
Using pmbootstrap v3 with the old config can cause problems, for example
when having $WORK in the pmaports dir instead of the actual work path.
This is not supported anymore by v3 to reduce complexity. The format of
how mirrors are stored in the config also has changed.
Use a separate config file, so users can go back from v3 to 2.3.x if
they need to (for figuring out a regression) and so users won't run into
bugs when moving from 2.3.x to v3.
chroot's aren't automatically initialized when running a command in them
anymore. Add the explicit chroot.init() call before checksum.
Fixes#2395
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This revers b82c4eb167 ("chroot: always run apk static (MR 2252)")
since the current iteration of the apk-tools /usr merge patches
don't handle --root properly and will behave differently based on your
host systems directory structure (specifially for /usr merge).
This commit can be reverted once we fix apk-tools.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Remove a bunch of commented out code:
* debug leftovers
* update_work(): a function that used to replace $WORK in all kinds of
arguments to pmbootstrap, with the actual work dir. This seemed like a
good idea when I made this initially, but in hindsight it's clear that
this made args just much more complex. I'm glad that Caleb removed
this. Now $WORK only gets removed in one place, for mount points,
which makes much more sense.
* args.from_argparse and related comment: pmbootstrap used to make a
full copy of its args, so in some cases it could go back before $WORK
was replaced IIRC. This isn't great, but even worse is that this
caused each stack trace to say something like infinite recursion on
args. I'm also glad that Caleb got rid of this :)
Add a warn_once() function using Cache. Since we use f-strings
everywhere we don't really need the varargs.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Don't crash on master_staging_systemd (and in the future the merged
systemd branch) if "none" is selected as UI.
Fix for:
ERROR: Could not find aport for package: postmarketos-ui-none
