Add some unit tests to ensure that the parser correctly detects boot
images with header v0, 2, and 3.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Commit 826bb4f2dd (pmb: Properly type Bootimg (MR 2464)) broke the
bootimg_analyze command since some assumptions in whether the bootimg
dictionary had empty keys or non-existent keys were changed without
their usage being updated.
Fix this and apply a trivial cleanup to aportgen.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We shouldn't overwrite config.aports if the user specified a custom
--aports directory. We only want to do this to update the default if
they're using a custom workdir.
Make the code here less aggressive, and correctly use the last entry in
config.aports as "pmaports" instead of the first.
Fixes: #2386
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This should match the previous behaviour and fix crashes when these
properties don't exist.
Tested with a htc-primou bootimg.
Fixes 826bb4f2dd
[ci:skip-build]: already built successfully in CI
Write some basic parser tests for deviceinfo. The parser is quite basic
so there isn't much to test yet. Expecting this to gain features as we
look at e.g. making the parser the "source of truth" for the deviceinfo
format.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add a basic test to ensure that the cache does get hit as we expect.
This doesn't test for cache conflicts.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Sometimes these fields are not set, e.g.:
pdb> p ret
{'depends': ['build-base', 'bash', 'bison', 'findutils', 'flex', 'musl-dev', 'openssl-dev', 'perl', 'python3'], 'arch': 'noarch', 'version': '20241022.022230', 'pkgname': '.makedepends-linux-lenovo-21bx', 'provides': []}
I hit this crash when I installed the build deps for the
linux-lenovo-21bx package in the native chroot, and then ran
`pmbootstrap stats`
[ci:skip-build]: already built successfully in CI
* Make sure all Python modules are documented by adding a bit of shell
to .ci/docs.sh
* Remove non-existing module references from .rst
* Fix various warnings from sphinx by adjusting Python docstrings
* Add class member docs to ApkindexBlock
pmb.parse.apkindex.providers now returns a dict of string:ApkindexBlock,
probably from 71772b9b6
This adds type hinting to package_provider to reflect this, and fixes
code that calls it to properly deal with the possible return types.
Without this, building FDE images is broken.
Instead of printing an URL with each warning, print one once at the
end. This gives less cluttered output, and we don't have information
about each and every option on that wiki page anyway (and given that we
now have so many options, I don't think it's good use of time to add it
there.)
This can be used when building images for generic device targets that
support devices with different sector size requirements.
For example, trailblazer prebuilts are currently expected to be flashed
to a USB drive where a 4096 sector size would be unsuitable since the
bootloader wouldn't detect it. But when building for a Qualcomm phone,
one would use --split and --sector-size to build the root and boot
partitions with a 4k sector size which is appropriate to the UFS
storage.
This flag could also be used by BPO to build both variants.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Abort the "pmbootstrap kconfig check" on the first error, unless the
--keep-going argument was passed. This makes it easier to go through the
failed kernels one by one until they are all fixed.
Remove arguments to set postmarketOS and Alpine mirrors.
The current implementation is broken with the way the mirrors are now
configured: The current implementation fills a list, but the mirror
configuration code expects a string. It is not longer possible to set
an arbitrary amount of mirrors for the postmarketOS mirror. The benefit
is that we don't have comma separated values in the config file anymore
but instead just simple strings. Now it is possible to have one proper
mirror for aports, pmaports and pmaports_systemd, and one _custom one
to override each of these.
We already have too many options in "pmbootstrap -h", and changing a
mirror is more of a base configuration that you do once and maybe change
a few times, but don't need to be able to set it with each pmbootstrap
run. Users are not going to write this out manually on their
command-lines, it only makes sense in scripts and wrappers for
pmbootstrap, and there you could as well use another method (as now
described in docs/mirrors.md) to set the mirrors.
Less important, but another reason is that using "-mp" (two letters as
short argument) isn't really elegant.
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
- Change some arguments' types to bool to avoid type mismatch errors
after refactoring in the future
- Add more arguments from parse/arguments with corresponding types
- Change type of pmbootstrap log --lines argument to int
Using pmbootstrap v3 with the old config can cause problems, for example
when having $WORK in the pmaports dir instead of the actual work path.
This is not supported anymore by v3 to reduce complexity. The format of
how mirrors are stored in the config also has changed.
Use a separate config file, so users can go back from v3 to 2.3.x if
they need to (for figuring out a regression) and so users won't run into
bugs when moving from 2.3.x to v3.
Add the missing runtime conversion when --config is given on the
cmdline.
This expects a pathlib.Path object as the argument, but args.config is
only a string.
Remove a bunch of commented out code:
* debug leftovers
* update_work(): a function that used to replace $WORK in all kinds of
arguments to pmbootstrap, with the actual work dir. This seemed like a
good idea when I made this initially, but in hindsight it's clear that
this made args just much more complex. I'm glad that Caleb removed
this. Now $WORK only gets removed in one place, for mount points,
which makes much more sense.
* args.from_argparse and related comment: pmbootstrap used to make a
full copy of its args, so in some cases it could go back before $WORK
was replaced IIRC. This isn't great, but even worse is that this
caused each stack trace to say something like infinite recursion on
args. I'm also glad that Caleb got rid of this :)
Move all kconfig rules from pmb.config to a separate toml file
pmb/data/kconfigcheck.toml. This is a fallback, pmbootstrap now prefers
loading kconfigcheck.toml from the currently checked out pmaports branch
if it exists.
This finally allows having separate kconfig check rules per pmaports
branch and makes the workflow of adjusting these rules much more
pleasant as the rules and kernel configs can just be adjusted at the
same time in pmaports!
This patch also moves the definition of what rules should be checked for
community and main devices, those that have pmb:kconfigcheck-community
in their linux APKBUILD, to the new kconfigcheck.toml. This should make
it much more intuitive, previously one needed to find the place in the
pmbootstrap source and edit it there.
Furthermore the "enforce_check" logic is removed. Previously pmbootstrap
would print warnings for failed config checks in some cases, but not
exit with error which was very confusing. Now exit 0 means all checks
passed and exit 1 means, that there is at least one error.
Use toml for the file, as discussed in pmbootstrap issue 2165. Python
3.11 has a native toml reader, use tomli for previous Python versions
for compatibility.
Prepare to move the kconfigcheck config into pmaports, so it can be
different per branch and so we can adjust it at the same time as
adjusting the kernel configs.
Once it is moved, pmbootstrap will not know the cateogry names in
advance anymore, so remove the logic for having category-specific
argument names.
In a future patch, we could add an option to manually override the
categories to be tested with one specific category (taking a string
argument), if there is demand for that.
Python 3.10.12 fails without this patch:
File "/media/gompa/73d88639-a730-456c-a428-6d500b4020b7/pmbootstrap/pmb/parse/arguments.py", line 844 help="Alpine Linux mirror, default: " f"{default_config.mirrors["alpine"]}",
SyntaxError: f-string: unmatched '['
Related: https://peps.python.org/pep-0701/
(Commit message written by Oliver)
Sometimes the local binary repo will have no packages but still an
APKINDEX. It will be empty in this case. Handle this gracefully.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".
For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.
The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
