On some kernels (notably, WSL2) parted fails to inform the kernel
about the new partition layout, leading to errors when trying to
create the new filesystems.
This change runs an additional `partprobe` after updating the
partition table, informing the kernel about the new layout.
Fixes: #2422
Signed-off-by: Dominik Bayerl <dominik.bayerl@carissma.eu>
[caleb: move call so it runs after partition_cgpt() as well]
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
There is a lot of context and state management needed when using
pmbootstrap, sometimes it can be a lot to keep in your head. Let's print
the output of "pmbootstrap status" when stuff goes wrong, this might
help remind people if e.g. their pmaports checkout is on the wrong
branch, or they're building for the wrong device.
Additionally, don't print the "run pmbootstrap log for details" message
if pmbootstrap was called with --details-to-stdout
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
You can't check enum membership with __contains__() on Python 3.10, poke
inside the enum instead.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Use Alpine 3.17 to test pmbootstrap with older versions of Python
Fixes: #2477
Suggested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We shouldn't overwrite config.aports if the user specified a custom
--aports directory. We only want to do this to update the default if
they're using a custom workdir.
Make the code here less aggressive, and correctly use the last entry in
config.aports as "pmaports" instead of the first.
Fixes: #2386
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
There was an oversight when this API was originally created and it
implicitly assumed that the pmaports repository was always named
"pmaports". This unfortunately broke some peoples workflows.
Introduce a new "pkgrepo_name()" function and adjust the codebase to use
it, as well as adjusting pkgrepo internally to special case the
"pmaports" repo so that it's always named pmaports no matter what the
directory itself is named.
This is probably more complexity than we should be dealing with here, we
should probably create a new type to encode this behaviour.
Fixes: #2412
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When pmbootstrap starts up, it loads the config file (or uses the
defaults) and then merged some arguments (e.g. --work) into it. However
in config.init.frontend() we were creating yet another new Config, but
it's too late to merge args in at this point...
Rework this slightly to fix that major oversight and remove the totally
unnecessary hack of calling pmb.config.load() again.
This fixes calling pmbootstrap init with --work
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This was always a hack, and it looks like there (hopefully) aren't any
placs where we still need to handle this.
Possibly expecting regressions... But then we have something to write a
test for.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When running "apk add" we must always have --cache-dir set! We also
expect --no-interactive to be set.
Add some asserts so we don't regress here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Since parse.deviceinfo() returns a custom class now, it can't be
trivially serialized as JSON.
This has been broken for months now so it's clear nobody is using this.
Let's take the chance to get rid of it.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Since all apk invocations withing pmbootstrap now use apk.static by
default, apk_static.run() isn't called anymore, so let's remove it.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
In 0b4fb9119f (chroot: always run apk static v2 (MR 2423)) we adjusted
install_run_apk() to run apk static on the host and pass in the local
binary repo with "--repository". This function can call apk in two ways,
either with the progress bar handling or without, the second case was
never updated and still ran apk inside the chroot incorrectly and with
an incorrect --repository flag.
Let's finish the job by refactoring helpers/apk.py to support all our
usecases and pointing everything to it, removing the last few situations
where we call "pmb.chroot.root(["apk", ...]).
The apk_with_progress() function is replaced by a generic "run()"
function which takes a boolean to indicate if we should render apk
progress.
Additionally, a new cache_clean() function is added so that "pmbootstrap
zap --pkgs-online-mismatch" can FINALLY be refactored to not rely on a
chroot existing. This requires some hacks but nothing serious, see the
comments in the function for details.
The chroot.init() code is now simplified since handling the --root,
--arch, --cache-dir, and --repository flags is now all done by
apk._prepare_cmd() as and when appropriate.
Lastly, this fixes a (previously unnoticed) bug where apk.static was
actually using /var/cache/apk on your host machine for its cache... This
is definitely not good behaviour....
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This function better belongs here, especially as it will be used outside
of the context of a chroot() soon.
Additionally, it's adjusted to take the rootfs path as its first
argument rather than a chroot, since it could operate on any rootfs.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Lazy load pmb.config.styles and move the pmb.__version__ print elsewhere
so the logging module is (closer to) a standalone entity. This is
necessary to be able to import it in pmb/helpers/apk.py otherwise we get
a cyclical dependency.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This should match the previous behaviour and fix crashes when these
properties don't exist.
Tested with a htc-primou bootimg.
Fixes 826bb4f2dd
[ci:skip-build]: already built successfully in CI
Refactor the integration test script to support running a variety of
tests, and add CI jobs for them.
Additionally, move the integration tests to their own stage so we don't
potentially waste a bunch of CI time running them on broken branches.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Write some basic parser tests for deviceinfo. The parser is quite basic
so there isn't much to test yet. Expecting this to gain features as we
look at e.g. making the parser the "source of truth" for the deviceinfo
format.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Otherwise, if pmb.parse.deviceinfo().cgpt_kpart is an empty string, for
example, we end up passing an empty string to get_partition_layout()
instead of False.
I think the "these types make no sense" comment were about
output_return_buffer, which defaulted to False but was to be set to
list[bytes] if used. I changed it so that it defaults to None, which
probably is more conventional. Other than that I didn't notice anything
weird about the types here.
Also check for None as necessary to appease mypy.
img_path_2nd is a Path, not a str, so we can't just rely on string
concatenation here.
img_path_2nd is a Path as that's what create_second_storage() returns
since 198f302a36.
Reusing this variable might lead to confusing bugs later if this code is
changed in the future, and also leads to mypy being confused about the
type of size_mb as it then gets assigned both str and int values.
At some point, this function was updated to use the Arch type instead of
plain strings, but the return type was left omitted and the docstring
outdated.
Flush the progress bar displayed by the function before logging a
warning message for 404 not found responses from the server. Without
flushing, this message gets lost.
In addition to the warning, display a NOTE and ERROR at the end to
explain what went wrong:
[20:16:50] Update package index for x86_64 (4 file(s))
[20:16:51] WARNING: file not found: http://mirror.postmarketos.org/postmarketos_get_404_test/edge/main/x86_64/APKINDEX.tar.gz
[20:16:51] NOTE: check the [mirrors] section in 'pmbootstrap config'
[20:16:51] ERROR: getting APKINDEX from binary package mirror failed!
Without handling this properly, pmbootstrap would fail slightly later
with a much less useful error:
[15:14:28] ERROR: expected str, bytes or os.PathLike object, not NoneType
Fixes: pmbootstrap issue 2424
Don't update the git URL again when coming from 7-2.x where this step
was already done. This didn't fail, but was potentially confusing for
the user since the git URLs were already upgraded from 7 -> 7-2x.
Some packages in aports have their own subpackages in their
dependencies, for some reason. This causes our resolver to assume there
is a cyclical dependency.
Adjust the resolver to safely ignore these dependencies, since they'll
be built as a part of the package that depends on them.
For example, modemmanager depends on libmm-glib which is a subpackage of
modemmanager. Currently to build modemmanager in pmaports it would be
necessary to manually modify the APKBUILD after copying it from aports.
This change avoids that.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
