Commit graph

2733 commits

Author SHA1 Message Date
Clayton Craft
a7f288634f
pmb.qemu.run: only set show-cursor when display!=none (MR 2309) 2024-06-23 13:30:29 +02:00
Andrew Robbins
1c3572dee2
Correct btrfs subvolume creation in install step (MR 2319)
btrfs can only create a single subvolume at a time.
2024-06-23 13:10:47 +02:00
Oliver Smith
6ebd4cccb9
README: mention 3.0.0_alpha 2024-06-23 12:49:25 +02:00
Caleb Connolly
0fe6244f3c
sideload: fix double semicolon (MR 2252)
flat_cmd and everything really needs cleaned up.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
b89e300c26
tests: fix tests (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
e6c737f342
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
cb6cd3bc4c
chroot: apk: safety net when chroot unitialized (MR 2252)
We recently changed how we manage chroots, requiring the user (of the
chroot) to initialize it before doing stuff like installing packages.

There are however still certain edgecases in pmbootstrap where this
doesn't get done (for example qemu/run.py would attempt to install the
kernel package for the device, but we don't initialize the rootfs chroot
in QEMU since that doesn't make sense to do). Check for and catch this
situation explicitly so we don't ruin someones day, but still be loud
about it so we can hopefully catch the remaining instances of this.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Oliver Smith
794048e2d5
pull: fix aports branches not being recognized (MR 2252)
Fix for:
  $ pmbootstrap pull
  [21:09:43] aports_upstream (branch: 3.20-stable): not on one of the official branches (master), skipping pull!

Replace the get_branches_official() function that according to git log
was never able to tell if an aports branch was official. The new
function branch_looks_official() can do that by just checking if the
branch follows the typical naming pattern.

The comment I had put into get_branches_official earlier was not true
anymore, by now this did not only get used by "pmbootstrap status", but
also by "pmbootstrap pull".
2024-06-23 12:38:42 +02:00
Oliver Smith
ef594ddf24
pmb/commands/pull: fix path error with f-string (MR 2252)
Fix for:
  TypeError: can only concatenate str (not "PosixPath") to str
2024-06-23 12:38:42 +02:00
Oliver Smith
9e2bbf80e0
pull: move to pmb/commands/pull.py (MR 2252)
Fixes:
  TypeError: pull() takes 0 positional arguments but 1 was given
2024-06-23 12:38:42 +02:00
Oliver Smith
e2461f00a2
Set version to 3.0.0_alpha (MR 2252)
Let pmbootstrap display 3.0.0_alpha on crash, so it's clear when people
are using the new codebase.

  Before you report this error, ensure that pmbootstrap is up to date.
  Find the latest version here: https://gitlab.com/postmarketOS/pmbootstrap/-/tags
  Your version: 3.0.0_alpha
2024-06-23 12:38:42 +02:00
Caleb Connolly
9fdf49ef0e
flasher: f-string fixes (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
8ecbf1a7de
chroot: apk: don't filter out local packages (MR 2252)
This turns out to have been wrong, since we need to give apk the package
name AND the apk file, it installs the apk and then uninstalls it but
leaves the package installed by name.
2024-06-23 12:38:42 +02:00
Caleb Connolly
fe079c8676
qemu: don't try and install stuff in rootfs (MR 2252)
Don't try and install random packages in the rootfs chroot when starting
QEMU. The chroot is not initialised and this just adds latency.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2472e4c612
Revert "build: init: install pigz" (MR 2252)
Nope, abuild doesn't get along with it at all.

This reverts commit 91a4cfdc9a.
2024-06-23 12:38:42 +02:00
Caleb Connolly
d4d6382897
helpers: pkgrel_bump: fix formatting (MR 2252)
f-strings

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
099bd2f4fa
treewide: remove rogue leading slashes (MR 2252)
One flaw of pathlib is that Path("/tmp/") / "/some/relative/path"
results in Path("/some/relative/path")....

Fix the places in the codebase where we get this wrong.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
9fff9033f2
build: init: install pigz (MR 2252)
faster, better..

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ac148da791
chroot: apk: recurse depends for locally built packages (MR 2252)
Turns out this is necessary, *sigh*. Since we could have one locally
built package depend on another locally built package.

So yeah, re-implement this, but in a slightly lazier / hopefully faster
way.

Bah

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
6f12c0c902
chroot: apk: check all channels for local package (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
139fd2b3e0
install: losetup: fix mount retry (MR 2252)
The retry mechanism here go lost along the way. Bring it back.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ceaad5c291
build: move run_abuild and friends to a new file (MR 2252)
_package.py was getting pretty big. split the actual build steps out.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
700fe32cf5
build: init: de-dup apk add steps (MR 2252)
We install abuild and then immediately install more packages, make this
one step.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
883ab8311e
qemu: fix --display none (MR 2252)
Don't specify the virtio vga device with --display none is set.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
0857f1aa4e
build: package: improve logging (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
bc29bc9d63
tests: chroot: xfail chroot types that aren't tested (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
30fde53279
vermin: don't use StrEnum (MR 2252)
It was only introduced in 3.11. Instead just define __str__() manually
for the Enums where it's needed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4108b95d72
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
5c652e0602
parse: apkindex: handle empty APKINDEX (MR 2252)
Sometimes the local binary repo will have no packages but still an
APKINDEX. It will be empty in this case. Handle this gracefully.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4a52f46bf0
build: fail if remote checksums mismatch (MR 2252)
This is a potential security risk. Tighten things up so we only allow
local checksums to mismatch but remote ones must be correct.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2babb61778
build: package: support incremental builds with --src (MR 2252)
Avoid deleting build artifacts when building with --src.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
91d29a1b2a
build: package: don't copy files that are in .gitignore (MR 2252)
This avoids copying local build artifacts and other things, if they're
in .gitignore then abuild shouldn't need them anyway.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
19a54ee0d6
build: package: fix --src (MR 2252)
Make sure rsync is actually installed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
f331b95824
chroot: allow mounting the device rootfs (MR 2252)
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".

For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.

The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
d1a86fe702
qemu: use OVMF EFI firmware from chroot (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
7d85bb31d5
build: ignore invalid checksums unless --strict (MR 2252)
We can fairly easily patch abuild by appending anything we want to the
APKBUILD file after copying it to the chroot.

Leverage this to override the verify() function so that it doesn't die
when checksums don't match.

Instead let's build the package anyway, but set a flag and print a
warning with instructions on how to generate the checksums.

We should continue to require APKBUILDs in pmaports to have valid
checksums.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
d75f1cf525
tests: add a test for packages_get_locally_built_apks() (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c89b53a87e
chroot: apk: fix installing local packages (MR 2252)
This got broken during rework. Fix it up, make sure we filter out
packages from to_add that are specified in to_add_local.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
fc950100ba
Revert "pmb.chroot.apk: fastpath when all packages are already installed" (MR 2252)
This reverts commit becb6b9e740a0fc0c5d4ea0eeecd148da3bcb530.
2024-06-23 12:38:41 +02:00
Caleb Connolly
3034768a1a
chroot: apk: fix local repos with apk.static (MR 2252)
It turns out that apk with the --root argument will still treat local
repositories relative to the host. This makes sense considering the
intended usecase, however as a result the recent change to use
apk.static for all apk commands inside the target root will break the
use of local packages.

To fix this, adjust how we populate /etc/apk/repositories, changing the
default to no longer include the local user repo. Instead we pass
--repository to apk.static with the host path to the repo.

The only time we want to add the user repo to /etc/apk/repositories
inside the chroot is when the user enters the chroot, so a special case
is added to helpers/frontend.py for the chroot command.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
77f66eb43d
build: package: more dependency resolution fixes (MR 2252)
It was the case that packages were being queued in the wrong order,
since the dependency resolver queues the package before descending into
it's dependencies. This was a bit of a goof since the top level package
was always added last. If we add it first then we can just reverse the
queue and now everything is fine...

Additionally, the logic on when a dependency should be built was a bit
wonky. A case is added for when a dependency exists only in the source
repo and the requisite package isn't marked for build. The solver will
now build the dependency regardless. This is surely an edgecase but
somehow I ran into it, I suspect due to a bug elsewhere...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
7f258516db
build: package: fix cache (MR 2252)
Big blunder lol, now it actually caches...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
30f175073e
qemu: init chroot (MR 2252)
We need to make sure the chroot is initialised before use.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c97ea203a4
chroot: shutdown: simplify (MR 2252)
Remove some of the duplicated parts, make sure we actually shut down
loop devices.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c061ef8123
install: losetup: add detach_all() (MR 2252)
Add a function that just detaches all loop devices that are backed by a
file inside the pmbootstrap work dir.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c4b4c41b8d
helpers: logging: fixes + details_to_stdout in color (MR 2252)
* Fix details_to_stdout which was broken due to a previous commit.
* Improve support for custom colors with "@COLOR@" (e.g. "@BLUE@"), stop
  it from being printed unformatted to the logfile.
* Rework how the logfd is handled so that --details-to-stdout won't
  disable all color output.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
53bdb523e3
checksum: remove args (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c1aa5acdaa
config: add auto_zap_misconfigured_chroots (MR 2252)
This option will make pmbootstrap automatically zap chroots that are
initialized for the wrong channel, making it much faster to switch
between edge and systemd.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
af826c671d
config: file: add usage note to save() (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
8fbc3a450d
build: strict: fix call to chroot.zap (MR 2252)
args lol

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00