Commit graph

3309 commits

Author SHA1 Message Date
Clayton Craft
f9add1b6d2
pmb.helpers.apk: fix crash computing progress from apk3
Fixes:

tot = float(cur_tot[1])
      ^^^^^^^^^^^^^^^^^
ValueError: could not convert string to float: '5383561 download'

Because this line now includes " download" at the end:

  ipdb> p line
  '10727/5383561 download\n'

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2635
2025-07-02 17:25:33 +02:00
Clayton Craft
a8988872b6
pmb.helpers.apk_static: fix version check for apk3
ipdb> p version_bin
'3.0.0_rc5_git20250613-r0'
ipdb> p version
'3.0.0_rc5_git20250613-r0'

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2635
2025-07-02 17:25:32 +02:00
Pablo Correa Gómez
7d36c604f1
test: add small test for argument parsing
This is just a small detail, but in some way CLI options are similar
to public API. I did this while trying some changes, even if in the
end the issue to solve was not in pmbootstrap, I thought it'd make
sense to contribute this.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2626
2025-06-15 10:25:30 +02:00
Pablo Correa Gómez
ed396475f5
pmb: drop unused import
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2621

[ci:skip-build]: already built successfully in CI
2025-06-09 22:43:57 +02:00
Pablo Correa Gómez
527fc9359f
export: drop support for flavor in export
Official mkinitfs flavors were dropped long ago. v23.12 already
released with "supported_mkinitfs_without_flavors=True". So there is
no need to keep this around anymore. When we have initramfs and
kernels that have suffixes (e.g: from alpine), we already deal with
them.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2621
2025-06-09 22:43:57 +02:00
Pablo Correa Gómez
d9560962ac
export: allow to export vmlinuz files which have any suffix
This is useful, for example, to export alpine linux kernels.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2621
2025-06-09 22:43:57 +02:00
Oliver Smith
562e6519d5
Prepare 3.5.0 release 2025-06-09 19:32:22 +02:00
Luca Weiss
606eb669fc
pmb.aportgen: Add default value to fork_alpine parameter
No need to specify this when fork_alpine has nothing to do with the
usage.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2610
2025-06-04 10:12:55 +02:00
Luca Weiss
ed57b3dac3
pmb.aportgen: Match aportgen prefixes explicitly
Instead of using getattr with an arbitrary prefix, just match the couple
of cases explicitly.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2610
2025-06-04 10:12:54 +02:00
Luca Weiss
f1a6f0034b
pmb.aportgen.core: Initialize chroot properly
Make sure to initialize the native chroot before trying to run a command
in it.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2610
2025-06-04 10:12:53 +02:00
Oliver Smith
47be020c7a
Remove "pmbootstrap repo_bootstrap"
The repo bootstrap logic has worked well enough in the initial systemd
bringup. But we have decided to replace the approach of building
packages multiple times during the repo_bootstrap with building each
package only once and adding -stage0 packages where necessary (currently
only a systemd-stage0 package).

Advantages:
* Replace the often breaking repo_bootstrap logic with something less
  fragile (repo_bootstrap is currently broken again).
* Can get rid of the whole bootstrap logic in pmaports.cfg, pmbootstrap,
  build.postmarketos.org. This will make pmbootstrap and bpo easier to
  maintain.
* Fix problems we have seen when upgrading two or more of (systemd,
  dbus, linux-pam-pmos) at once: it doesn't pass in CI and it doesn't
  pass in BPO
* You don't need to do a whole bootstrap at once, it is again broken
  down by package. This means if building one package fails (locally or
  in bpo), we don't need to do the whole thing again, just start at the
  package that failed. This also means it is much easier to optimize
  e.g. the stage0 packages to build faster.
* Fixes some specific bugs we currently have (abuild twice in pmaports,
  bpo tests disabled because of that, ...)

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2588
2025-06-03 17:25:41 +02:00
Oliver Smith
f6b0ebd78c
pmb/helpers/other: fix typo found by codespell
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2588
2025-06-03 17:25:41 +02:00
Oliver Smith
26c310f374
init: systemd: fix always switching pma to edge
Fix that pmbootstrap doesn't recognize v25.06 with systemd enabled as
valid channel and suggests switching to edge. This happens because
pmb.config.pmaports.read_config() unconditionally prepends "systemd-"
before the channel name if systemd is enabled, even though the init code
doesn't expect this.

The result is when running 'yes "" | pmbootstrap init' (as we do in BPO
and CI), pmbootstrap attempts to replace the correctly checked out
v25.06 branch with edge:

Fix for:
  [20:36:37] Available (12):
  [20:36:37] * edge: Rolling release / Most devices / Occasional breakage: https://postmarketos.org/edge
  [20:36:37] * v25.06: Upcoming stable release (DO NOT USE)
  [20:36:37] * v24.12: Latest release / Recommended for best stability
  [20:36:37] Channel [edge]:
  [20:36:37] Currently checked out branch 'v25.06' of pmaports.git is on channel 'v25.06'.
  [20:36:37] Switching to branch 'master' on channel 'edge'...
  …
  ERROR: pmbootstrap switched to the wrong branch: master

Related: https://builds.sr.ht/~postmarketos/job/1497179#task-bpo_setup-23

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2613
2025-05-30 11:08:34 +02:00
Clayton Craft
c39ac5fe99
pmb.qemu.run: fix x86_64 CPU check for EFI support (MR 2612)
Apparently python reads uname for this value, the correct value for
64-bit intel/amd stuff is `x86_64` even though python's docs list
`amd64` as an example of a value `platform.machine()` can return 🤷

fixes #2612
[ci:skip-build]: already built successfully in CI
2025-05-27 11:33:16 +02:00
Oliver Smith
63dea3993d
pmb.config.apk_tools_min_version: add v3.22
Prepare for the upcoming postmarketOS v25.06 release.
2025-05-26 22:02:00 +02:00
Newbyte
548a1dfe2e
pmb: Warn users about downstream ports
And make the archived port detection more robust while at it.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2549
2025-05-25 19:26:57 +02:00
Newbyte
cd672222c4
pmb: Split devices by category during codename selection
This also reworks list_codenames() somewhat. The option to show archived
devices is removed as it never actually was used. It should be easy to
restore if someone is interested.

Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2558

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2549
2025-05-25 19:26:57 +02:00
Newbyte
a9c3628297
pmb.helpers.logging: Fix type for args
This is supposed to be `object` rather than `str` according to typeshed:
57cb510fd2/stdlib/logging/__init__.pyi (L129-L212)

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2549
2025-05-25 19:26:57 +02:00
Newbyte
a1057c9770
pmb.config.__init__: Add escape sequence for magenta text
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2549
2025-05-25 19:26:57 +02:00
Newbyte
c426c14f47
pmb.helpers.pmaports: Clean up some strings
Use f-strings to make them easier to read and combine some not-so-long
ones into one line for easier grepping.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2549
2025-05-25 19:26:52 +02:00
Newbyte
5783d1c507
Prepare 3.4.1 release 2025-05-25 15:13:41 +02:00
Clayton Craft
f4001310f2
pmb.qemu.run: command_qemu: refactor applying arch-specific options
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2560
2025-05-25 12:05:56 +02:00
Clayton Craft
b214fa6c4a
pmb.qemu.run: add support for EFI on aarch64
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2560
2025-05-25 12:05:56 +02:00
Clayton Craft
4a7ec11f4c
pmb.qemu.run: use virgl on aarch64 hosts
This enables faster/more featureful 3D accel support in VMs, which is necessary
for running some future UIs.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2560
2025-05-25 12:05:56 +02:00
Oliver Smith
cdb51686c3
CI: fix coverage regex
Use exactly the same regex as in BPO where this works.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2609
2025-05-25 11:55:27 +02:00
Pablo Correa Gómez
7ca76314cd
ci: add test coverage to the GitLab UI
This is a small nit, but allows to see the coverage percentage change
across MRs. It gives a small dopamine boost to see an MR you sent has
tests and made a small impact in the overall coverage :)

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2596
2025-05-25 11:48:33 +02:00
Alexey Andreyev
8f564edd78
docs/mirrors: mention env variables example
Mention Environment Variables section too

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2598
2025-05-25 11:44:14 +02:00
Alexey Andreyev
383266a7a8
docs/index: update local deploy example with pmb.ci
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2599
2025-05-25 11:42:01 +02:00
Alexey Andreyev
a61fd50d6f
docs/index: introduce local deploy example
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2599
2025-05-25 11:42:01 +02:00
Stefan Hansson
abb8d92437
Revert "CI: ruff: use 0.11.0 (Cannot use star annotation)"
The bug has been fixed upstream for a while now.

This reverts commit aced51132c

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2600
2025-05-25 11:39:14 +02:00
Stefan Hansson
da3fa09ca2
pmb.helpers.apk: Avoid concatenation
Appeases Ruff.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2600
2025-05-25 11:39:14 +02:00
Stefan Hansson
017522382f
pmb.config.init: Use heapq for sorting timings
This works because heapq sorts based on the first value in tuples
and heap[0] always is the smallest element.

See:
 - https://docs.python.org/2/library/heapq.html#basic-examples
 - https://docs.python.org/3/library/heapq.html

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2582
2025-05-25 11:37:12 +02:00
J. Neuschäfer
5c03cdd37f
pmb.arch: Add support for s390x
This commit adds s390x to the list of supported architectures, to allow
users/developers to experiment with it.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2536
2025-05-25 11:32:01 +02:00
Newbyte
0d672a7dc7
pmb.qemu.run: Use NonBugError for invalid device selection error
This eliminates all noise and results in an output that only contains
the actual relevant and helpful error message.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2603
2025-05-25 11:30:21 +02:00
Clayton Craft
226eeed6ff
ci: only run integration tests when there are code changes
This prevents running the integration tests, which cause load on the CI
and take time, when the change is unrelated to pmb functionality (e.g.
documentation updates).

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2608
2025-05-25 11:28:51 +02:00
Newbyte
15007e17d2
pmb.helpers.pmaports: Add must_exist to cache signature for find()
As must_exist affects the return value of this function by changing it
from returning None to raising an exception, we need to include it in
the cache key. Otherwise, the must_exist value effectively gets "stuck"
on whatever value it was set to when the given cache key was created.

This causes weird issues, see [1] and [2]. As such, include it in the
cache signature to fix this.

 [1]: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2601
 [2]: https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/6507#note_481093

Supersedes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2601
Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2610

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2604
2025-05-25 11:18:01 +02:00
Clayton Craft
740a5717d9
ci: only run markdown lint if markdown files are changed
This prevents running md linting in MRs where unrelated files are
changing, since the markdownlint tool could be upgraded to add new tests
that start failing and make these unrelated MRs unmergeable.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2606
2025-05-23 09:35:43 -07:00
Newbyte
75bc105605
CONTRIBUTING.md, README.md: Use descriptive links
Markdownlint complained about the links not being descriptive enough for
accesibility purposes, so make them actually describe what we're linking
to.

Alternative to https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2602

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2605
[ci:skip-build]: already built successfully in CI
2025-05-21 23:13:59 +02:00
Stefan Hansson
aebf99e988
Prepare 3.4.0 release 2025-05-13 19:42:56 +02:00
Pablo Correa Gómez
ba6bb4272d
pmb.parse.apkindex: account for provider_priority when locating packages
This fixes an issue where the local apk index has two packages in it
that provide the same thing, but this function was returning only the
one with the shortest name... which could be very bad :)

https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/6475#note_480167

In the process, add tests for pmb.parse.apkindex.package function. Some of
the tests would fail without this patch, surfacing a bug where priorities
are not considered, but should, and that we are now fixing.

Co-authored-by: Clayton Craft <craftyguy@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2594
2025-05-13 19:19:00 +02:00
slonkazoid
71f01ddbeb
pmb.core.config: remove +1 from nproc defaults
Since Linux has switched to EEVDF[1], the $(($(nproc)+2)) hack is no
longer faster. The new scheduler is way more efficient and performs
better with only `nproc` threads.

 [1]: https://docs.kernel.org/scheduler/sched-eevdf.html

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2595
2025-05-11 17:28:35 +02:00
Luca Weiss
0bca7555d7
pmb.netboot: Initialize chroot before checking path in chroot
Otherwise with zapped chroots, the check will fail and pmbootstrap won't
be able to locate the netboot image.

Fixes: d01f6b90 ("pmb.netboot: Only attempt to install nbd after rootfs existance check (MR 2540)")

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2583
2025-05-08 23:14:05 +02:00
Oliver Smith
2177271fc8
build: fix duplicate install messages
Since cross-native2 was introduced, pmbootstrap now tells the user which
host and which build dependencies are getting installed. This makes
sense if they get installed into separate chroots, but not if they get
installed into the same chroot:

	$ pmbootstrap build hello-world-wrapper --force
	[16:47:19] Building 1 package
	[16:47:19]    * edge/hello-world-wrapper
	[16:47:19] => (1/1) edge/hello-world-wrapper: Installing dependencies
	[16:47:19] *** Install host dependencies
	[16:47:19] (native) install hello-world
	[16:47:20] *** Install build dependencies
	[16:47:20] (native) install hello-world
	[16:47:22] => edge/hello-world-wrapper: Building package

Don't list the depends separately if we are installing them into the
same chroot:

	$ pmbootstrap build hello-world-wrapper --force
	[16:51:35] Building 1 package
	[16:51:35]    * edge/hello-world-wrapper
	[16:51:35] => (1/1) edge/hello-world-wrapper: Installing dependencies
	[16:51:35] (native) install hello-world
	[16:51:36] => edge/hello-world-wrapper: Building package

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2589
2025-05-08 22:45:18 +02:00
Anri Dellal
d7b12d98e6
pmb: Rename pmb:gpu-accel and deviceinfo_gpu_accelerated
pmb:gpu-accel -> pmb:drm
deviceinfo_gpu_accelerated -> deviceinfo_drm

Allow deviceinfo_gpu_accelerated as deprecated property

Keep pmb:gpu-accel as valid option to avoid failures for older branches
of pmaports.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2591
2025-05-08 17:33:11 +02:00
Pablo Correa Gómez
496d1b657d
Revert "install: openrc: fix installing alpine-base early"
This reverts commit 6eb8cab84d.

This is a problematic way to solve the issue. We want to eventually
allow people to migrate from openrc, and having alpine-base in the
world file is an issue. Moreover, a package that runs rc-update
and doesn't depend on openrc simply has a packaging bug. This is
something that is documented, and should be generally fixed
(https://gitlab.postmarketos.org/postmarketOS/pmaports/-/issues/2097).
If we manage to detect more issues, we should fix them in packaging
instead, like with
https://gitlab.postmarketos.org/postmarketOS/pmaports/-/merge_requests/6474

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2593
2025-05-05 21:58:17 +02:00
Oliver Smith
b982e45c8d
pmb.build.package: fix strict mode
In strict mode, pmbootstrap uninstalls packages between builds. This was
broken since we switched from installing dependencies via abuild to
pmbootstrap since our code doesn't use "--virtual .makedepends-…" when
installing the makedepends:

  (088776) [15:32:34] (native) % cd /home/pmos/build; busybox su pmos -c SUDO_APK='abuild-apk --no-progress' HOME=/home/pmos abuild undeps ;
  ERROR: No such package: .makedepends-systemd-stage0

Instead of restoring the logic and worrying about uninstalling deps in
the right chroots, just zap all chroots in strict mode.

I've also considered implementing "--virtual .makedepends-…" again, but
running zap on the chroots makes the logic much simpler, is easier to
maintain and makes so little speed difference that it is worth the
trade-off. We don't need to pass additional parameters fore the 2nd
chroot to zap in cross-native2, no need to figure out the virtual
package name in two places, etc.

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2587
2025-04-22 16:15:51 +02:00
Oliver Smith
fda8963ed7
init: fix currently checked out channel detection
In "pmbootstrap init", always use the pmaports root path when figuring
out the channel of the checked out commit (-> edge, v24.12, ...).

Otherwise the systemd path may get used, which returns channels prefixed
with "systemd-", such as "systemd-edge".

The default UI is "console", and it was recently switched to use systemd
by default. Since then the channel in pmaports CI gets detected as
"systemd-edge", which is not the same as "edge" and so pmbootstrap
switched to the master branch during "pmbootstrap init", resulting in CI
running on the wrong branch:

[10:54:44] Currently checked out branch 'HEAD' of pmaports.git is on channel 'systemd-edge'.
[10:54:44] Switching to branch 'master' on channel 'edge'...

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2586
2025-04-21 13:30:30 +02:00
Oliver Smith
6eb8cab84d
install: openrc: fix installing alpine-base early
With recent changes we don't install alpine-base in the chroots anymore.
It still gets installed as postmarketos-base depends on it (for now,
this will change soon in master). But it is not guaranteed to be
installed before other packages which may run "rc-update" in their
postinst scripts.

Fix this by explicitly installing alpine-base early on during
"pmbootstrap install".

Fix for e.g. v24.12:pine64-pinephone:phosh images:

  (342/976) Installing eg25-manager-openrc (0.4.6-r5)
  (343/976) Installing device-pine64-pinephone-openrc (7-r0)
  Executing device-pine64-pinephone-openrc-7-r0.post-install
  lib/apk/exec/device-pine64-pinephone-openrc-7-r0.post-install: line 3: rc-update: not found
  ERROR: device-pine64-pinephone-openrc-7-r0.post-install: script exited with error 127

Related: https://builds.sr.ht/~postmarketos/job/1473669#task-img-743
Fixes: 12d125ab ("chroot: don't depende on alpine-base")

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2585
2025-04-18 10:38:34 +02:00
Oliver Smith
9e8c4b2810
lint: adjust comment for empty apkbuild_paths
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2584
2025-04-15 15:57:55 +02:00
Clayton Craft
7ab40a4336
pmb.lint: Don't try to lint apkbuild paths that are empty
This fixes an issue where pmb lint tries to lint an apkbuild using a repo path that the apkbuild doesn't exist in.
For example, it fixes this:

```
[22:13:26] (native) linting postmarketos-ui-cosmic with apkbuild-lint
[22:13:26] *** apkbuild-lint output ***
Error: File APKBUILD does not exist
[22:13:26] *** apkbuild-lint output ***
[22:13:26] ERROR: Linter failed!
Traceback (most recent call last):
  File "/builds/postmarketOS/pmaports/.ci/lib/apkbuild_linting.py", line 26, in <module>
    common.run_pmbootstrap(["-q", "lint"] + packages)
  File "/builds/postmarketOS/pmaports/.ci/lib/common.py", line 52, in run_pmbootstrap
    subprocess.run(cmd, universal_newlines=True, check=True)
  File "/usr/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pmbootstrap', '--aports', '/builds/postmarketOS/pmaports', '-q', 'lint', 'postmarketos-ui-cosmic']' returned non-zero exit status 2.
```

Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2584
2025-04-15 11:51:29 +02:00