I tried my best to get the right author based on git commit history, but
in the case of code being moved from one file to make another I might
have gotten something wrong.
This might have saved me a couple of minutes of debugging, at least.
While working with the aportgen code I had two distinct issues:
1. APKINDEX was outdated, so I got an error about not being able to
download the supposedly latest version of BusyBox as it no longer
existed on Alpine's mirror.
2. APKINDEX had not been created by any other command, so aportgen
would tell me that it couldn't find the "busybox" package.
The changes in this commit resolves both of these issues from my
testing.
These were almost identical across the Busybox, GRUB EFI, and musl
generators. Unify them by creating functions they all (and any future
aportgen generators) can call instead of having to maintain three
separate copies of this code.
Add some tests to ensure that we spit out correct deviceinfo options for
boot images of header v0, 2, and 3.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This can otherwise fail if the package was only built for some
architectures so far, but not for the architecture that is being generated.
E.g. right now musl-1.2.5-r6 was built already for x86_64 but not aarch64,
which causes errors like the following when running "pmbootstrap aportgen
musl-x86_64" on aarch64:
>>> musl-x86_64: Fetching musl-1.2.5-r5-x86_64-edge.apk::http://dl-4.alpinelinux.org/alpine//edge/main/x86_64/musl-1.2.5-r5.apk
Connecting to dl-4.alpinelinux.org (147.75.40.42:80)
Connecting to dl-4.alpinelinux.org (147.75.40.42:443)
wget: server returned error: HTTP/1.1 404 Not Found
>>> ERROR: musl-x86_64: checksum failed
Add some tests to ensure that we spit out correct deviceinfo options for
boot images of header v0, 2, and 3.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Commit 826bb4f2dd (pmb: Properly type Bootimg (MR 2464)) broke the
bootimg_analyze command since some assumptions in whether the bootimg
dictionary had empty keys or non-existent keys were changed without
their usage being updated.
Fix this and apply a trivial cleanup to aportgen.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
* Make sure all Python modules are documented by adding a bit of shell
to .ci/docs.sh
* Remove non-existing module references from .rst
* Fix various warnings from sphinx by adjusting Python docstrings
* Add class member docs to ApkindexBlock
Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.
The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a Deviceinfo class and use it rather than the dictionary. This
gives us sweet sweet autocomplete, and lays the foundation for having a
proper deviceinfo validator in the future.
Additionally, continue refactoring out args...
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.
This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
With the new chroot type, we can now write fancy paths in the pythonic
way. Convert most of the codebase over, as well as adding various other
type hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We use a custom verbose log level in pmbootstrap, unfortunately it isn't
possible to correctly type this due to some limitations in the logging
library [1], [2].
Given that our usecase is fairly simple, we can just wrap the module
with our own so we only have to tell mypy to ignore the error once
instead of at every callsite.
[1]: https://github.com/cryptax/droidlysis/issues/15
[2]: https://github.com/python/typing/discussions/980
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a new module: pmb.core to contain explicitly typed pmbootstrap
API. The first component being Suffix and SuffixType. This explicitly
defines what suffixes are possible, future changes should aim to further
constrain this API (e.g. by validating against available device
codenames or architectures for buildroot suffixes).
Additionally, migrate the entire codebase over to using pathlib.Path.
This is a relatively new part of the Python standard library that uses a
more object oriented model for path handling. It also uses strong type
hinting and has other features that make it much cleaner and easier to
work with than pure f-strings. The Chroot class overloads the "/"
operator the same way the Path object does, allowing one to write paths
relative to a given chroot as:
builddir = chroot / "home/pmos/build"
The Chroot class also has a string representation ("native", or
"rootfs_valve-jupiter"), and a .path property for directly accessing the
absolute path (as a Path object).
The general idea here is to encapsulate common patterns into type hinted
code, and gradually reduce the amount of assumptions made around the
codebase so that future changes are easier to implement.
As the chroot suffixes are now part of the Chroot class, we also
implement validation for them, this encodes the rules on suffix naming
and will cause a runtime exception if a suffix doesn't follow the rules.
Often I find myself confused and subsequently frustrated by the
behaviour of --fork-alpine as usually I apply whatever changes I want to
the aports package before forking. This is due to that in the proper
aports repo I can leverage the full power of git and apply patches or
revert commits, whereas after the package has been forked to pmaports
the context git needs to do this is lost.
To fix that, this commit introduces --fork-alpine-retain-branch which
works the same way as --fork-alpine except it doesn't change the aports
branch to match the current channel.
We have been discussing making device package versioning more
consistent and ended up settling on using integer versioning for device
packages[1]. As such, let's start by tackling this at the source and
start generating new device packages with 1 as the starting pkgver
instead of 0.1.
[1]: https://gitlab.com/postmarketOS/pmaports/-/issues/2610#note_1796178414
