Commit graph

2994 commits

Author SHA1 Message Date
Oliver Smith
18fa4e58a3
Ruff: fix typing.Xxx is deprecated, use xxx instead (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
61cb4f4abb
Enable the UP family of lints (MR 2327)
These are based on the originally from pyupgrade. They replace older
syntax with newer (and more concise) syntax. Some of the replaced syntax
is deprecated.
2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
cebd784d84
Prefer f-strings (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
8057715e83
Use Python 3.8-style type annotations (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
acd938c6b2
Remove super() syntax from Python 2.0 (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
f3f392ef66
Use simplified Python 3.8 syntax (MR 2327)
This commit was generated with:

    ruff check --fix --extend-select=UP .
2024-06-23 19:13:52 +02:00
Hugo Osvaldo Barrera
fa2a7c502d
Remove exception for lint (MR 2328)
There are no violations of this lint any more. This lint only checks
that an import is the beginning of a block; it will not be a problem if
a test needs to include an import.
2024-06-23 17:44:24 +02:00
Hugo Osvaldo Barrera
8398bc1218
Do not use bare except (MR 2325)
A bare except catches things like ctrl+c (KeyboardInterrupt). If don't
want to catch this exception. Remove the bare except and restore the
default lint.
2024-06-23 15:40:13 +02:00
Hugo Osvaldo Barrera
cb4aa809e1
Check python formatting in CI (MR 2325)
Fixes: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2324
2024-06-23 15:40:13 +02:00
Hugo Osvaldo Barrera
e421bb2d41
Auto-format codebase with ruff (MR 2325)
See: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2324
Closes: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2278
2024-06-23 15:40:13 +02:00
Hugo Osvaldo Barrera
5a8e2c6cad
Converge supported Python onto 3.9 (MR 2326)
Python 3.9 was required in pmb/__init__.py already, but some references
still said 3.8 and 3.7.

Co-Developed-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-06-23 15:37:19 +02:00
Clayton Craft
a7f288634f
pmb.qemu.run: only set show-cursor when display!=none (MR 2309) 2024-06-23 13:30:29 +02:00
Andrew Robbins
1c3572dee2
Correct btrfs subvolume creation in install step (MR 2319)
btrfs can only create a single subvolume at a time.
2024-06-23 13:10:47 +02:00
Oliver Smith
6ebd4cccb9
README: mention 3.0.0_alpha 2024-06-23 12:49:25 +02:00
Caleb Connolly
0fe6244f3c
sideload: fix double semicolon (MR 2252)
flat_cmd and everything really needs cleaned up.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
b89e300c26
tests: fix tests (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
e6c737f342
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
cb6cd3bc4c
chroot: apk: safety net when chroot unitialized (MR 2252)
We recently changed how we manage chroots, requiring the user (of the
chroot) to initialize it before doing stuff like installing packages.

There are however still certain edgecases in pmbootstrap where this
doesn't get done (for example qemu/run.py would attempt to install the
kernel package for the device, but we don't initialize the rootfs chroot
in QEMU since that doesn't make sense to do). Check for and catch this
situation explicitly so we don't ruin someones day, but still be loud
about it so we can hopefully catch the remaining instances of this.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Oliver Smith
794048e2d5
pull: fix aports branches not being recognized (MR 2252)
Fix for:
  $ pmbootstrap pull
  [21:09:43] aports_upstream (branch: 3.20-stable): not on one of the official branches (master), skipping pull!

Replace the get_branches_official() function that according to git log
was never able to tell if an aports branch was official. The new
function branch_looks_official() can do that by just checking if the
branch follows the typical naming pattern.

The comment I had put into get_branches_official earlier was not true
anymore, by now this did not only get used by "pmbootstrap status", but
also by "pmbootstrap pull".
2024-06-23 12:38:42 +02:00
Oliver Smith
ef594ddf24
pmb/commands/pull: fix path error with f-string (MR 2252)
Fix for:
  TypeError: can only concatenate str (not "PosixPath") to str
2024-06-23 12:38:42 +02:00
Oliver Smith
9e2bbf80e0
pull: move to pmb/commands/pull.py (MR 2252)
Fixes:
  TypeError: pull() takes 0 positional arguments but 1 was given
2024-06-23 12:38:42 +02:00
Oliver Smith
e2461f00a2
Set version to 3.0.0_alpha (MR 2252)
Let pmbootstrap display 3.0.0_alpha on crash, so it's clear when people
are using the new codebase.

  Before you report this error, ensure that pmbootstrap is up to date.
  Find the latest version here: https://gitlab.com/postmarketOS/pmbootstrap/-/tags
  Your version: 3.0.0_alpha
2024-06-23 12:38:42 +02:00
Caleb Connolly
9fdf49ef0e
flasher: f-string fixes (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
8ecbf1a7de
chroot: apk: don't filter out local packages (MR 2252)
This turns out to have been wrong, since we need to give apk the package
name AND the apk file, it installs the apk and then uninstalls it but
leaves the package installed by name.
2024-06-23 12:38:42 +02:00
Caleb Connolly
fe079c8676
qemu: don't try and install stuff in rootfs (MR 2252)
Don't try and install random packages in the rootfs chroot when starting
QEMU. The chroot is not initialised and this just adds latency.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2472e4c612
Revert "build: init: install pigz" (MR 2252)
Nope, abuild doesn't get along with it at all.

This reverts commit 91a4cfdc9a.
2024-06-23 12:38:42 +02:00
Caleb Connolly
d4d6382897
helpers: pkgrel_bump: fix formatting (MR 2252)
f-strings

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
099bd2f4fa
treewide: remove rogue leading slashes (MR 2252)
One flaw of pathlib is that Path("/tmp/") / "/some/relative/path"
results in Path("/some/relative/path")....

Fix the places in the codebase where we get this wrong.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
9fff9033f2
build: init: install pigz (MR 2252)
faster, better..

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ac148da791
chroot: apk: recurse depends for locally built packages (MR 2252)
Turns out this is necessary, *sigh*. Since we could have one locally
built package depend on another locally built package.

So yeah, re-implement this, but in a slightly lazier / hopefully faster
way.

Bah

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
6f12c0c902
chroot: apk: check all channels for local package (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
139fd2b3e0
install: losetup: fix mount retry (MR 2252)
The retry mechanism here go lost along the way. Bring it back.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ceaad5c291
build: move run_abuild and friends to a new file (MR 2252)
_package.py was getting pretty big. split the actual build steps out.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
700fe32cf5
build: init: de-dup apk add steps (MR 2252)
We install abuild and then immediately install more packages, make this
one step.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
883ab8311e
qemu: fix --display none (MR 2252)
Don't specify the virtio vga device with --display none is set.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
0857f1aa4e
build: package: improve logging (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
bc29bc9d63
tests: chroot: xfail chroot types that aren't tested (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
30fde53279
vermin: don't use StrEnum (MR 2252)
It was only introduced in 3.11. Instead just define __str__() manually
for the Enums where it's needed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4108b95d72
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
5c652e0602
parse: apkindex: handle empty APKINDEX (MR 2252)
Sometimes the local binary repo will have no packages but still an
APKINDEX. It will be empty in this case. Handle this gracefully.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4a52f46bf0
build: fail if remote checksums mismatch (MR 2252)
This is a potential security risk. Tighten things up so we only allow
local checksums to mismatch but remote ones must be correct.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2babb61778
build: package: support incremental builds with --src (MR 2252)
Avoid deleting build artifacts when building with --src.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
91d29a1b2a
build: package: don't copy files that are in .gitignore (MR 2252)
This avoids copying local build artifacts and other things, if they're
in .gitignore then abuild shouldn't need them anyway.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
19a54ee0d6
build: package: fix --src (MR 2252)
Make sure rsync is actually installed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
f331b95824
chroot: allow mounting the device rootfs (MR 2252)
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".

For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.

The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
d1a86fe702
qemu: use OVMF EFI firmware from chroot (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
7d85bb31d5
build: ignore invalid checksums unless --strict (MR 2252)
We can fairly easily patch abuild by appending anything we want to the
APKBUILD file after copying it to the chroot.

Leverage this to override the verify() function so that it doesn't die
when checksums don't match.

Instead let's build the package anyway, but set a flag and print a
warning with instructions on how to generate the checksums.

We should continue to require APKBUILDs in pmaports to have valid
checksums.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
d75f1cf525
tests: add a test for packages_get_locally_built_apks() (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c89b53a87e
chroot: apk: fix installing local packages (MR 2252)
This got broken during rework. Fix it up, make sure we filter out
packages from to_add that are specified in to_add_local.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
fc950100ba
Revert "pmb.chroot.apk: fastpath when all packages are already installed" (MR 2252)
This reverts commit becb6b9e740a0fc0c5d4ea0eeecd148da3bcb530.
2024-06-23 12:38:41 +02:00