Previously, it would only fall back to parsing all APKBUILDs if a main
package was guessed for this potential subpackage. We do however want to
scan all APKBUILDs in case the package is provided by one of the
packages in pmaports.
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
g++ is a subpackage of gcc, but cannot be detected as such easily by
pmbootstrap. This is because g++ is not gcc-g++ or any other variant of
such package names. Similarly, the detection for musl-dev-ppc64le and
other architectures is broken, since the -dev suffix detection does not
work if there is an architecture suffix.
To fix it, add special case handling for cross toolchain and packages
and have that fixup the -dev cases and hardcode g++ as a subpackage of
gcc.
To reproduce:
- Generate ppc64le cross packages:
pmbootstrap aportgen gcc-ppc64le musl-ppc64le
- Build a package (to trigger building cross compilers):
pmbootstrap build hello-world --arch=ppc64le
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
self.arch is a property and calls Arch.from_str, which errors upon
encountering an unknown architecture. Therefore, the error message is
changed and needs to be adjusted in the tests. Since Arch.supported is a
set, error messages were not deterministic before, so we need to sort
the list of architectures now.
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
Otherwise abuild will try and fail to create /dev/null
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
[ci:skip-build]: already built successfully in CI
recent changes to the kernels build infra in -next seem to have exposed
some issues with "pmbootstrap build --envkernel", specifically in the
case where the APKBUILD doesn't build out-of-tree.
We used to rely on the fact that the kernel output directory contains a
Makefile which points to the source directory, however this Makefile
now(?) contains an absolute path on the host, which won't match what's
in the chroot.
As a result, it's now necessary to build with the same output directory
as the APKBUILD.
We probably need some smarter code here... and/or more consistent
APKBUILDs.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This can otherwise fail if the package was only built for some
architectures so far, but not for the architecture that is being generated.
E.g. right now musl-1.2.5-r6 was built already for x86_64 but not aarch64,
which causes errors like the following when running "pmbootstrap aportgen
musl-x86_64" on aarch64:
>>> musl-x86_64: Fetching musl-1.2.5-r5-x86_64-edge.apk::http://dl-4.alpinelinux.org/alpine//edge/main/x86_64/musl-1.2.5-r5.apk
Connecting to dl-4.alpinelinux.org (147.75.40.42:80)
Connecting to dl-4.alpinelinux.org (147.75.40.42:443)
wget: server returned error: HTTP/1.1 404 Not Found
>>> ERROR: musl-x86_64: checksum failed
"y" only works here by accident as it's considered truthy by Python and
as such gets handled equivalently to True in the current implementation
of pmb.helpers.cli.confirm(), but it's still relying on unintended
behaviour. Change this to use a bool as intended.
After extending CI to always run "pmbootstrap zap -a" it revealed
another bug: we did not ensure at this point that apk.static is
available. Rung apk_static.init() to ensure it gets downloaded and
extracted at this point if it is missing.
The tmp dir is owned by root, when it gets created earlier by apk
running as root to store the apk progress fifo. Use another directory to
work around this for now (getting close to the 3.0.0 release), we can
rework the apk progress fifo later on if we want.
Fixes: issue 2491
Replace aports -> pmaports in these messages.
$ pmbootstrap build systemd
[17:37:02] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:37:02] ERROR: Could not find package 'systemd' in pmaports
If the user tries to build a package, but it can't be found: check if
systemd is disabled and the package is in extra-repos/systemd, display a
hint about it.
$ pmbootstrap build systemd
[17:33:53] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:33:53] ERROR: Could not find aport for package: systemd
Fixes: issue 2398
Replace the boolean skip_extra_repos with a new with_extra_repos
argument that can be default, enabled or disabled.
This will be used to explicitly enable extra repos in a package search
even if systemd is currently disabled, so we can display a hint when a
package was not found because systemd is disabled in a follow-up patch.
Add some tests to ensure that we spit out correct deviceinfo options for
boot images of header v0, 2, and 3.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add some unit tests to ensure that the parser correctly detects boot
images with header v0, 2, and 3.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
make sure that calls to get_upstream_remote() will succeed by setting
pmb.config.git_repos() for the entire run. This might pollute some other
tests...
TODO: make less things depend on pmaports being cloned...
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Commit 826bb4f2dd (pmb: Properly type Bootimg (MR 2464)) broke the
bootimg_analyze command since some assumptions in whether the bootimg
dictionary had empty keys or non-existent keys were changed without
their usage being updated.
Fix this and apply a trivial cleanup to aportgen.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When generating an ssh key it's simple to change the name of the key to
something that's not e.g. id_rsa.pub or id_ed25519.pub, so let's try to
capture ssh keys for more users.
Since a user can modify the glob to be somewhat arbitrary, use that to
check if any ssh keys exist and ask in pmbootstrap init.
Otherwise ssh_key_glob would only work if it was a subset of
~/.ssh/id_*.pub, so e.g. ~/.ssh/id_foo.pub but ~/.ssh/foo.pub wouldn't
work.
We don't care about the output of "losetup --json" here, we just want to
make sure that the argument exists. If it does not exist, we already
give an error to the user.
Set the output to null so it does not clutter the log since it runs with
every pmbootstrap command now.
Remove this chroot-specific helper and merge it in with the existing
require_programs() logic. Now we have one unified way to discover and
use programs from the host and where we can put any additional special
handling we might need.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
On debian systems in particular special handling is needed for
/usr/sbin. Let's do this once and then provide the host deps we use in a
nice accessible dictionary.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
In Debian, /usr/sbin is not in PATH of regular users unless using sudo.
Fix that pmbootstrap won't find kpartx and losetup in /usr/sbin, and
complains about these even though they are installed.
Fixes: issue 2489
On some kernels (notably, WSL2) parted fails to inform the kernel
about the new partition layout, leading to errors when trying to
create the new filesystems.
This change runs an additional `partprobe` after updating the
partition table, informing the kernel about the new layout.
Fixes: #2422
Signed-off-by: Dominik Bayerl <dominik.bayerl@carissma.eu>
[caleb: move call so it runs after partition_cgpt() as well]
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
There is a lot of context and state management needed when using
pmbootstrap, sometimes it can be a lot to keep in your head. Let's print
the output of "pmbootstrap status" when stuff goes wrong, this might
help remind people if e.g. their pmaports checkout is on the wrong
branch, or they're building for the wrong device.
Additionally, don't print the "run pmbootstrap log for details" message
if pmbootstrap was called with --details-to-stdout
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
You can't check enum membership with __contains__() on Python 3.10, poke
inside the enum instead.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We shouldn't overwrite config.aports if the user specified a custom
--aports directory. We only want to do this to update the default if
they're using a custom workdir.
Make the code here less aggressive, and correctly use the last entry in
config.aports as "pmaports" instead of the first.
Fixes: #2386
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
There was an oversight when this API was originally created and it
implicitly assumed that the pmaports repository was always named
"pmaports". This unfortunately broke some peoples workflows.
Introduce a new "pkgrepo_name()" function and adjust the codebase to use
it, as well as adjusting pkgrepo internally to special case the
"pmaports" repo so that it's always named pmaports no matter what the
directory itself is named.
This is probably more complexity than we should be dealing with here, we
should probably create a new type to encode this behaviour.
Fixes: #2412
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When pmbootstrap starts up, it loads the config file (or uses the
defaults) and then merged some arguments (e.g. --work) into it. However
in config.init.frontend() we were creating yet another new Config, but
it's too late to merge args in at this point...
Rework this slightly to fix that major oversight and remove the totally
unnecessary hack of calling pmb.config.load() again.
This fixes calling pmbootstrap init with --work
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This was always a hack, and it looks like there (hopefully) aren't any
placs where we still need to handle this.
Possibly expecting regressions... But then we have something to write a
test for.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When running "apk add" we must always have --cache-dir set! We also
expect --no-interactive to be set.
Add some asserts so we don't regress here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
