This patch made the assumption that if the Alpine APKINDEX for the
"main" repository exist, all other Alpine indexes ("community",
"testing") as well as the postmarketOS indexes (normal one, systemd)
exist as well.
This is not always the case, e.g. when the internet connection dropped
while downloading indexes and so only "main" was downloaded. While this
is unlikely to happen, it is currently also breaking BPO where the
postmarketOS repository does not get set during "pmbootstrap init" on
purpose as it causes problems when bringing up new releases.
This reverts commit 752a3a98f5.
I'll make a different fix in the next patch for the problem that this
patch was made for, having a broken "pmbootstrap zap -a" with apkv3.
Related: BPO issue 160
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2636
While this hypothetically could be triggered by a malicious server,
more likely is that this would be caused by a bug in pmbootstrap. As
such, soften the message a bit and also remove references to the removed
-m flag.
Fixes ebfda16d6d
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2635
[ci:skip-build]: already built successfully in CI
When using `pmb zap --pkgs-online-mismatch`, the local index is
downloaded if missing and `update()` is cached, however pmb zap removes
the index and subsequent calls to `update()` don't actually do anything
(i.e. redownload the index) so later things that expect the indext to
exist (like finding providers) blow up.
Wrap update() with a new update() method that checks if the APKINDEX
exists, and always do a "real" update if it doesn't. Also rename
update() to _update().
Co-Developed-by: Clayton Craft <clayton@craftyguy.net>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2635
The repo bootstrap logic has worked well enough in the initial systemd
bringup. But we have decided to replace the approach of building
packages multiple times during the repo_bootstrap with building each
package only once and adding -stage0 packages where necessary (currently
only a systemd-stage0 package).
Advantages:
* Replace the often breaking repo_bootstrap logic with something less
fragile (repo_bootstrap is currently broken again).
* Can get rid of the whole bootstrap logic in pmaports.cfg, pmbootstrap,
build.postmarketos.org. This will make pmbootstrap and bpo easier to
maintain.
* Fix problems we have seen when upgrading two or more of (systemd,
dbus, linux-pam-pmos) at once: it doesn't pass in CI and it doesn't
pass in BPO
* You don't need to do a whole bootstrap at once, it is again broken
down by package. This means if building one package fails (locally or
in bpo), we don't need to do the whole thing again, just start at the
package that failed. This also means it is much easier to optimize
e.g. the stage0 packages to build faster.
* Fixes some specific bugs we currently have (abuild twice in pmaports,
bpo tests disabled because of that, ...)
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2588
This fixes an issue where pmb lint tries to lint an apkbuild using a repo path that the apkbuild doesn't exist in.
For example, it fixes this:
```
[22:13:26] (native) linting postmarketos-ui-cosmic with apkbuild-lint
[22:13:26] *** apkbuild-lint output ***
Error: File APKBUILD does not exist
[22:13:26] *** apkbuild-lint output ***
[22:13:26] ERROR: Linter failed!
Traceback (most recent call last):
File "/builds/postmarketOS/pmaports/.ci/lib/apkbuild_linting.py", line 26, in <module>
common.run_pmbootstrap(["-q", "lint"] + packages)
File "/builds/postmarketOS/pmaports/.ci/lib/common.py", line 52, in run_pmbootstrap
subprocess.run(cmd, universal_newlines=True, check=True)
File "/usr/lib/python3.12/subprocess.py", line 571, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['pmbootstrap', '--aports', '/builds/postmarketOS/pmaports', '-q', 'lint', 'postmarketos-ui-cosmic']' returned non-zero exit status 2.
```
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2584
Add a new environment variable that disables apk's caching feature for
space constrained environments such as the bpo image build jobs.
Currently we have a workaround in place in bpo: the apk cache is moved
to a tmpfs. But this is fragile and just disabling the apk cache is a
more elegant solution.
I've decided to make this an env var instead of a full pmbootstrap
option since this option is not relevant for normal users, only for CI
jobs in space constrained environments. Also we already have another
`PMB_APK_` env var.
Related: bpo issue 136
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2553
The .supported() method errs on the side of exposing too many
architectures to make porting for new or less-common platforms easier.
It therefore isn't suitable for using as a list of supported
architectures we can probe the binary repository for.
Introduce a .supported_binary() method to export only the architectures
where we have a binary repository.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
According to Python documentation, p.match checks
if characters at the beginning of string match the regex
This commit changes validation to full length of string
Add a flag to bind-mount in the necessary sysfs directories for USB
devices to be accessed inside the chroot. This is the same as how we do
it for "pmbootstrap flasher" but allows running arbitrary commands in
the chroot.
This is useful to allow using pmbootstrap chroot's as a sandbox for
custom flashing procedures.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Fix that repo_missing would pick the wrong package information if a
package is in both the normal repository and the split repository.
This would lead to having the wrong version and depends:
"pkgname": "gnome-shell-mobile",
"repo": null, # not in the systemd repo
"version": "99946.1-r1",
"depends": [
Instead of the correct version:
"pkgname": "gnome-shell-mobile",
"repo": null, # not in the systemd repo
"version": "46.1-r1",
"depends": [
Fix this by calling pmb.parse.apkbuild() directly with the APKBUILD path
when iterating over the APKBUILDs, instead of pmb.helpers.package.get().
Fix that abuild gets added twice in packages that exist in both the main
and split systemd repository:
"pkgname": "gnome-settings-daemon-mobile",
"repo": null,
"version": "99946.0-r0",
"depends": [
"abuild",
"abuild",
"alsa-lib-dev",
"colord-dev",
This fixes the following error in bpo when it tries to use the output of
"pmbootstrap repo_missing":
UNIQUE constraint failed: package_dependency.package_id, package_dependency.dependency_id
Related: https://postmarketos.org/edge/2025/01/09/systemd-soon/
In the systemd repository, we currently have a forked version of
abuild, which needs to be used to build all other packages. Check if we
have a forked abuild, and if it is the case, add it to the dependencies
of all other packages.
Closes: issue 2401
The "pmbootstrap repo_missing" action is used exclusively by bpo. It
calls it only with these arguments:
pmbootstrap repo_missing --built --arch "$ARCH"
A blocker for merging systemd into pmaports master is, that the
current repo_missing code cannot display packages that are both in
extra-repos/systemd and in another path. I have considered just not
supporting this and discussed doing that with Caleb and Clayton, but we
figured it would be a major obstacle in the future to not be able to
easily override packages with systemd specific versions (currently we
need this for 3 packages).
Integrating this into the existing repo_missing code would be hacks upon
hacks. Also the scope of the current repo_missing code has many extra
features that are not used and would be extra effort to carry along:
* Allow specifying a pkgname
* Running without --built
* --overview
So I decided to replace the repo_missing code with a much simpler, more
modern implementation, that does exactly what is needed:
* Duplicate packages in systemd and non-systemd dirs are displayed
* The output always include all packages, no matter if they are already
built or not (same behavior as with --built)
* Removed --overview and selecting specific packages too
* The code for filling "repo" (either "systemd" or None) is more
resilient now, as it can use proper relative paths to the root of
pmaports. Unlike the previous implementation, it will not fail if
subdirs are added to the systemd dir.
I have made sure that the output is exactly the same as before on
current pmaports master.
Related: bpo issue 144
Related: bpo issue 140
I tried my best to get the right author based on git commit history, but
in the case of code being moved from one file to make another I might
have gotten something wrong.
The parameters of this function were needlessly complicated. While I
understand the intent of trying to replicate the API of retrieve()
without duplicating the parameter list, in practice this feels like
reducing duplication to a fault to me. This is due to that the third
parameter, <allow_404>, doesn't make sense in the context of
retrieve_json() as it would raise an exception[1] if retrieve() returns
None (which is the effect of <allow_404> when a 404 occurs), thus merely
changing the exception that gets raised to a less descriptive one
instead of actually behaving like retrieve() would with that parameter
set to True.
With <allow_404> eliminated, there's just two parameters left, and I
don't think duplicating those is a big deal. So make the function easier
to read and annotate by getting rid of the args/kwargs logic.
[1]: json.loads() raises an exception if its first argument is None
