Commit graph

3101 commits

Author SHA1 Message Date
Oliver Smith
f9bca7cf09
Prepare 3.1.0 release
Prepare a new release, so pmbotostrap can be used with Alpine 3.21 /
postmarketOS v24.12. Also the big rewrite is done, so we can do more
frequent releases again.

Related: https://wiki.postmarketos.org/wiki/Pmbootstrap_release
2024-12-10 17:04:18 +01:00
Anri Dellal
d4985c9699
.gitignore: add note for missing text editors (MR 2501)
The note suggests to make local changes to git configuration
instead of expanding .gitingore with more editor files
2024-12-05 23:25:13 +01:00
Anri Dellal
dbf3e21446
pmb.aportgen.device: fix skipping when analyzing boot image (MR 2507)
Fix #2516
2024-12-05 23:11:26 +01:00
Anri Dellal
c831302a57
pmb.install._install: add setup_locale() (MR 2497)
Moves locale setup code to separate function
Adds code to install keyboard configuration based on chosen locale
2024-12-03 13:17:15 +01:00
Anri Dellal
bdc4a4872e
pmb.helpers.locale: add unit tests (MR 2497) 2024-12-03 13:17:14 +01:00
Anri Dellal
556160b200
pmb.helpers.locale: new module (MR 2497)
Add module that provides utilities and information related to localization.

Initial implementation includes code to generate keyboard configs
for locale set by user.
2024-12-03 13:17:14 +01:00
Oliver Smith
3e18183aed
docs: document PMB_APK_FORCE_MISSING_REPOSITORIES (MR 2506) 2024-12-03 10:08:56 +01:00
Oliver Smith
1dcb68bdca
docs: add env vars page with PMB_SUDO (MR 2506)
Move the description of PMB_SUDO from the README to docs.
2024-12-03 10:08:43 +01:00
Oliver Smith
559710c69a
CI: docs: fail on warning (MR 2506) 2024-12-03 10:08:43 +01:00
Oliver Smith
77125ff9fb
docs: fix 'WARNING: Title underline too short' (MR 2506) 2024-12-03 10:08:43 +01:00
Oliver Smith
268478a202
docs/api/pmb.commands: fix "failed to import" (MR 2506)
Fix the following:

  WARNING: Failed to import pmb.commands.kconfig_check.
  WARNING: Failed to import pmb.commands.kconfig_edit.
  WARNING: Failed to import pmb.core.crosstool.

Crosstool has been dropped and kconfig_check/_edit code has been merged
into a kconfig module.
2024-12-03 10:08:43 +01:00
Oliver Smith
07f04ae12a
pmb.helpers.repo.update: support PMB_APK_FORCE_MISSING_REPOSITORIES (MR 2505)
Do not abort if one or more APKINDEX files cannot be downloaded, if
PMB_APK_FORCE_MISSING_REPOSITORIES is set. This is needed when
bootstrapping new stable branches. The same environment variable is
already used in pmbootstrap code to pass --force-missing-repositories to
apk in pmb.chroot.apk.install_run_apk().
2024-12-03 06:24:03 +01:00
Caleb Connolly
aa847501c8
build: envkernel: overmount outdir/Makefile to use local include (MR 2504)
the kernel has pending patches in -next which adjust the generated
Makefile in the output directory to include the source tree makefile
with an absolute path, this breaks envkernel which relies on it being a
relative path.

Fix this by mounting our own Makefile instead using the relative path.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-12-01 19:40:28 +01:00
Caleb Connolly
932ac18791
build: envkernel: default to .output again (MR 2504)
We used to use .output as the default kbuild_out but this was removed
recently, add it back!

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-12-01 19:40:23 +01:00
Anri Dellal
3204577530
Fix apkbuild parsing in edge cases (MR 2503)
This commit removes separator in split function arguments, so arrays
where items are separates by tabulation don't fail to parse properly.
2024-11-30 11:53:17 +03:00
Caleb Connolly
bb12609753
test: conftest: require_programs() automatically (MR 2483)
Pmbootstrap finds all its host dependencies during init, ensure we do
that for pytest too.

Adapt require_programs() to use subprocess directly rather than have to
solve the weird dependency chain of pmb.helpers.run which depends on the
global context being initialised before it's usable.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-29 16:50:04 +01:00
Caleb Connolly
cefe98fb69
test: aportgen: test deviceinfo fastboot content (MR 2477) (MR 2483)
Add some tests to ensure that we spit out correct deviceinfo options for
boot images of header v0, 2, and 3.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-29 16:50:04 +01:00
Caleb Connolly
f090e86126
test: parse: add tests for bootimg parsing (MR 2477) (MR 2483)
Add some unit tests to ensure that the parser correctly detects boot
images with header v0, 2, and 3.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-29 16:50:04 +01:00
Caleb Connolly
39a696ceed
test: conftest: adjust pmaports fixture (MR 2477) (MR 2483)
make sure that calls to get_upstream_remote() will succeed by setting
pmb.config.git_repos() for the entire run. This might pollute some other
tests...

TODO: make less things depend on pmaports being cloned...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-29 16:50:04 +01:00
Caleb Connolly
1131db214a
Revert "CI: Add bootimg analysis integration test (MR 2467)" (MR 2483)
This test includes a binary blob which we want to avoid in our codebase.

It is also better suited as a unit test where it can contribute to code
coverage. Revert this and use a unit test version instead.

This reverts commit d19de946b3.
2024-11-29 16:49:54 +01:00
Newbyte
7b0db9f864
pmb.core.arch: Use annotations from future instead of quoted Arch type (MR 2498) 2024-11-29 10:43:09 +01:00
Newbyte
0752a202aa
pmb.core.arch: Handle unsupported arches in from_machine_type() better (MR 2498)
Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2501
2024-11-29 10:43:09 +01:00
Newbyte
8012b3cefe
pmb.core.arch: Use match statement instead of mappings (MR 2498)
I think this is easier to read and has more obvious error handling.
2024-11-29 10:43:05 +01:00
Jens Reidel
828f80056e
qemu: run: Add support for ppc64le (MR 2476)
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 14:27:10 +01:00
Jens Reidel
1e3e41907f
qemu: run: Invoke the correct qemu command (MR 2476)
pmbootstrap qemu did not respect architecture-specfic QEMU command name
mappings.

Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 14:27:10 +01:00
Jens Reidel
13d8154373
core: arch: Add ppc64le qemu mapping (MR 2476)
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 14:26:59 +01:00
Jens Reidel
822e89ed13
helpers: pmaports: Always fall back to parsing all APKBUILDS (MR 2476)
Previously, it would only fall back to parsing all APKBUILDs if a main
package was guessed for this potential subpackage. We do however want to
scan all APKBUILDs in case the package is provided by one of the
packages in pmaports.

Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 13:09:53 +01:00
Jens Reidel
4b4ba1e6ec
core: arch: Add ppc64le as a supported architecture (MR 2476)
Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 13:08:50 +01:00
Jens Reidel
e2354ec26f
helpers: pmaports: Fix package detection for cross g++ and musl-dev (MR 2476)
g++ is a subpackage of gcc, but cannot be detected as such easily by
pmbootstrap. This is because g++ is not gcc-g++ or any other variant of
such package names. Similarly, the detection for musl-dev-ppc64le and
other architectures is broken, since the -dev suffix detection does not
work if there is an architecture suffix.

To fix it, add special case handling for cross toolchain and packages
and have that fixup the -dev cases and hardcode g++ as a subpackage of
gcc.

To reproduce:
- Generate ppc64le cross packages:
  pmbootstrap aportgen gcc-ppc64le musl-ppc64le
- Build a package (to trigger building cross compilers):
  pmbootstrap build hello-world --arch=ppc64le

Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 13:08:50 +01:00
Jens Reidel
9ca0ada582
core: chroot: Deduplicate supported arch list (MR 2476)
self.arch is a property and calls Arch.from_str, which errors upon
encountering an unknown architecture. Therefore, the error message is
changed and needs to be adjusted in the tests. Since Arch.supported is a
set, error messages were not deterministic before, so we need to sort
the list of architectures now.

Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 13:08:50 +01:00
Jens Reidel
6d72043ac2
tests: Assert that riscv64 is supported (MR 2476)
riscv64 is supported but currently not covered by the arch test suite.

Signed-off-by: Jens Reidel <adrian@travitia.xyz>
2024-11-25 13:08:50 +01:00
Robert Eckelmann
07592b2ab6
.ci/docs.sh: add py3-sphinxcontrib-jquery to fix the failing search (MR 2500)
Signed-off-by: Robert Eckelmann <longnoserob@postmarketos.org>
2024-11-24 18:30:06 +09:00
Caleb Connolly
91f0a4feeb
build: newapkbuild: initialize chroot before use (MR 2495)
Otherwise abuild will try and fail to create /dev/null

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
[ci:skip-build]: already built successfully in CI
2024-11-22 12:18:52 +01:00
Caleb Connolly
04635dcb3e
build: envkernel: cleanup behaviour (MR 2491)
recent changes to the kernels build infra in -next seem to have exposed
some issues with "pmbootstrap build --envkernel", specifically in the
case where the APKBUILD doesn't build out-of-tree.

We used to rely on the fact that the kernel output directory contains a
Makefile which points to the source directory, however this Makefile
now(?) contains an absolute path on the host, which won't match what's
in the chroot.

As a result, it's now necessary to build with the same output directory
as the APKBUILD.

We probably need some smarter code here... and/or more consistent
APKBUILDs.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-20 22:49:32 +01:00
Stephan Gerhold
60da40232d
pmb/aportgen: lookup package in APKINDEX with correct arch (MR 2493)
This can otherwise fail if the package was only built for some
architectures so far, but not for the architecture that is being generated.

E.g. right now musl-1.2.5-r6 was built already for x86_64 but not aarch64,
which causes errors like the following when running "pmbootstrap aportgen
musl-x86_64" on aarch64:

>>> musl-x86_64: Fetching musl-1.2.5-r5-x86_64-edge.apk::http://dl-4.alpinelinux.org/alpine//edge/main/x86_64/musl-1.2.5-r5.apk
Connecting to dl-4.alpinelinux.org (147.75.40.42:80)
Connecting to dl-4.alpinelinux.org (147.75.40.42:443)
wget: server returned error: HTTP/1.1 404 Not Found
>>> ERROR: musl-x86_64: checksum failed
2024-11-20 22:05:02 +01:00
Oliver Smith
6836a55d60
pmb.config.apk_tools_min_version: add v3.21 2024-11-18 20:25:41 +01:00
Iuri Jikidze
215dfcc663
pmb.helpers.git: fix get_upstream_remote() (MR 2496)
Fixes bec2a4b154

[ci:skip-build]: already built successfully in CI
2024-11-18 12:16:42 +01:00
J. Pablo Navarro
bec2a4b154
helpers: git: ignore case in get_upstream_remote() 2024-11-18 04:12:38 +01:00
Newbyte
472726a9dc
pmb: Add more type hints (MR 2489) 2024-11-14 23:16:29 +01:00
Newbyte
0e9a2e596f
pmb.config.init: Call pmb.helpers.cli.confirm() correctly (MR 2489)
"y" only works here by accident as it's considered truthy by Python and
as such gets handled equivalently to True in the current implementation
of pmb.helpers.cli.confirm(), but it's still relying on unintended
behaviour. Change this to use a bool as intended.
2024-11-14 22:34:52 +01:00
Oliver Smith
79c6e4c36d
Prepare 3.0.0 release 2024-11-10 19:13:39 +01:00
Oliver Smith
cfa455cc5e
zap: cache_clean: get apk.static if missing (MR 2481)
After extending CI to always run "pmbootstrap zap -a" it revealed
another bug: we did not ensure at this point that apk.static is
available. Rung apk_static.init() to ensure it gets downloaded and
extracted at this point if it is missing.
2024-11-10 18:22:43 +01:00
Oliver Smith
7a5c04f660
ci/integration: run "zap -a" afterwards (MR 2481)
Make sure we don't regress in "pmbootstrap zap -a" by running it after
each integration test.
2024-11-10 18:22:43 +01:00
Oliver Smith
506b7685c2
Fix pmb zap -a permissions error (MR 2481)
The tmp dir is owned by root, when it gets created earlier by apk
running as root to store the apk progress fifo. Use another directory to
work around this for now (getting close to the 3.0.0 release), we can
rework the apk progress fifo later on if we want.

Fixes: issue 2491
2024-11-10 18:22:38 +01:00
Oliver Smith
2a7864b080
Tweak messages for 'Could not find aport' (MR 2480)
Replace aports -> pmaports in these messages.

$ pmbootstrap build systemd
[17:37:02] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:37:02] ERROR: Could not find package 'systemd' in pmaports
2024-11-10 17:43:39 +01:00
Oliver Smith
67318823b5
pmaports.find: show extra-repos/systemd hint (MR 2480)
If the user tries to build a package, but it can't be found: check if
systemd is disabled and the package is in extra-repos/systemd, display a
hint about it.

$ pmbootstrap build systemd
[17:33:53] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:33:53] ERROR: Could not find aport for package: systemd

Fixes: issue 2398
2024-11-10 17:43:39 +01:00
Oliver Smith
0a0f6ead33
Replace skip_extra_repos -> with_extra_repos (MR 2480)
Replace the boolean skip_extra_repos with a new with_extra_repos
argument that can be default, enabled or disabled.

This will be used to explicitly enable extra repos in a package search
even if systemd is currently disabled, so we can display a hint when a
package was not found because systemd is disabled in a follow-up patch.
2024-11-10 17:43:39 +01:00
Oliver Smith
444d9e256f
pmaports._find_apkbuilds: use new cache logic (MR 2480)
Use the nice @Cache decorator that Caleb introduced earlier.
2024-11-10 17:43:39 +01:00
Oliver Smith
214abed0bb
gitignore: add junit.xml (MR 2480) 2024-11-10 17:43:39 +01:00
Oliver Smith
7b5487dad8
Revert "test: aportgen: test deviceinfo fastboot content (MR 2477)" (MR 2482)
This caused a regression in CI tests.
This reverts commit 6ee82740a4.

Related: issue 2497
2024-11-10 17:23:00 +01:00