Commit graph

329 commits

Author SHA1 Message Date
Newbyte
1aa0b05014
pmb.install.blockdevice: Avoid reusing size_mb variable in create_and_mount_image() (MR 2464)
Reusing this variable might lead to confusing bugs later if this code is
changed in the future, and also leads to mypy being confused about the
type of size_mb as it then gets assigned both str and int values.
2024-10-29 14:21:01 +01:00
Vikram Alagh
8ac2e48a21
install: losetup: check if the back-file attribute for a loop device is null (MR 2456)
Seems to be ChromeOS specific:

> this happens every time on my machine. I am on ChromeOS 129.0.6668.112
> using a Debian bookworm chroot through Crouton. I am not sure why
> this is happening nor which layer is responsible for this. I deleted
> the chroot and created another one but the problem persists.

Commit msg tweaked by Oliver.
2024-10-27 16:03:17 +01:00
Newbyte
3c47a52f91
pmb.install.losetup: Remove debug print (MR 2450)
Looks like an unintended leftover from debugging.

---

This was needlessly reverted by
1ec55fc11a. While revreting
0975d06437 was good, this commit did not
cause any harm.
2024-10-27 13:14:11 +01:00
Newbyte
bbf20a030b
pmb.install.losetup: Annotate return types (MR 2430) (MR 2450)
And fix type errors.

---

This was needlessly reverted by
1ec55fc11a. While revreting
0975d06437 was good, this commit did not
cause any harm.
2024-10-27 13:14:11 +01:00
Newbyte
37ec73c07c
pmb.flasher: Remove use of args (MR 2441)
Also adapt pmb/install/recovery.py to new API for variables.py.

[ci:skip-build]: already built successfully in CI
2024-10-22 15:12:40 +02:00
Clayton Craft
47dc493701
pmb.parse.depends.package_provider: add type hinting and fix references (MR 2439)
pmb.parse.apkindex.providers now returns a dict of string:ApkindexBlock,
probably from 71772b9b6

This adds type hinting to package_provider to reflect this, and fixes
code that calls it to properly deal with the possible return types.
Without this, building FDE images is broken.
2024-10-18 13:45:57 +02:00
Clayton Craft
1ec55fc11a
Revert "pmb.install.losetup: Run kpartx, losetup in chroot (MR 2430)"
This series breaks pmbootstrap, and that's not good[1]. A bug was filed
about this (#2465), and there have been multiple attempts to fix it (!
2435 and !2436). It kinda seems like we don't have time to fix/test this
for a while longer, which is fine, but given the impact this bug has I
think we should be revert this series until this issue is solved/tested.

1. pmb is broken in some specific cases, which means some workflows are
broken. One example is that this breaks the pmaports CI, so no work can
be done in pmaports.
2024-10-16 11:35:36 -07:00
Newbyte
a7e2592f1a
pmb.install.losetup: Remove debug print (MR 2430)
Looks like an unintended leftover from debugging.
2024-10-16 00:07:48 +02:00
Newbyte
0975d06437
pmb.install.losetup: Run kpartx, losetup in chroot (MR 2430)
I tested $ pmbootstrap shutdown with this and observed no problems. All
the mounts disappear in lsblk as expected.

Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2423
2024-10-16 00:07:48 +02:00
Newbyte
6455441464
pmb.install.losetup: Annotate return types (MR 2430)
And fix type errors.
2024-10-16 00:07:44 +02:00
Stefan Hansson
71772b9b6b
pmb.parse.apkindex: Introduce proper typing (MR 2425)
And adjust other code.

Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2455
2024-10-13 20:12:34 +02:00
Aster Boese
8335d050bf
pmb.install: switch to gpt with discoverable partitions (MR 2426) 2024-10-13 19:59:26 +02:00
Oliver Smith
eb747ad21f
find_providers: tweak default matching (MR 2301)
The previous version would fail if the suffix contained a dash as well,
e.g. postmarketos-base-ui-audio-some-new-thing. Fix it and add example
comments.
2024-09-30 18:41:09 +02:00
JustSoup321
56a4329f4f
pmb.{config, helpers, install}: add _pmb_default support (MR 2301) 2024-09-30 18:37:01 +02:00
Clayton Craft
36c86cd971
install: create_device_rootfs: install pmos-base-systemd first (MR 2383)
When building a systemd image, this patch installs pmos-base-systemd
(and its dependencies) first before building the rest of the device
chroot. This is a workaround for...

The pmos-base-systemd package provides distro-wide systemd presets for
services, and when it's added to the `install_packages` list then
there's no guarantee that apk will install it *before* service packages
execute `systemctl preset`. In other words: `preset` may be run before
the distro presets are actually installed, and crushing expectations. By
making sure pmos-base-systemd is installed in the chroot first, we can
provide some guarantee that presets will be there when services are
installed a bit later.
2024-09-02 10:27:38 -07:00
JustSoup321
375814b9d2
pmb.{chroot, install}: modify for bsdutils (MR 2315)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-09-01 15:45:06 +02:00
JustSoup321
3559ee4030
pmb.init: make default hostnames pass regex (MR 2393) 2024-09-01 15:01:02 +02:00
Caleb Connolly
f6f503035b
install: add a --sector-size flag (MR 2391)
This can be used when building images for generic device targets that
support devices with different sector size requirements.

For example, trailblazer prebuilts are currently expected to be flashed
to a USB drive where a 4096 sector size would be unsuitable since the
bootloader wouldn't detect it. But when building for a Qualcomm phone,
one would use --split and --sector-size to build the root and boot
partitions with a 4k sector size which is appropriate to the UFS
storage.

This flag could also be used by BPO to build both variants.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-08-25 22:06:17 +02:00
Caleb Connolly
0b0172bafc
install: add flash_boot for fastboot-bootpart (MR 2378)
When using fastboot-bootpart flash method to split root/boot partitions
out, it was no longer possible to flash the Android boot partition.

Add a new flash_boot action to handle this.

This will allow us to migrate devices like the OnePlus 6 to have the
boot partition (ESP) on system, and the rootfs on userdata rather than
the current subpartition method.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 12:53:28 +02:00
Caleb Connolly
c23a9ea9cd
install: fix root partition path in print (MR 2377)
When we output the root partition path, it says rootfs.img but it should
be root.img fix it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-20 17:33:02 +02:00
Oliver Smith
36dd53f402
Run ruff check --fix (MR 2357)
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
2024-07-16 00:26:35 +02:00
HenriDellal
2b577a8694
Fix pmbootstrap install --android-recovery-zip (MR 2368)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-07-16 00:04:15 +02:00
Caleb Connolly
0e62eaac67
ruff: format (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:09:42 +02:00
Caleb Connolly
65d23905bc
treewide: another round of mypy fixes (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:13 +02:00
Caleb Connolly
6301212d99
install: partition: drop unused args (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:12 +02:00
Oliver Smith
18fa4e58a3
Ruff: fix typing.Xxx is deprecated, use xxx instead (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
cebd784d84
Prefer f-strings (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
f3f392ef66
Use simplified Python 3.8 syntax (MR 2327)
This commit was generated with:

    ruff check --fix --extend-select=UP .
2024-06-23 19:13:52 +02:00
Hugo Osvaldo Barrera
e421bb2d41
Auto-format codebase with ruff (MR 2325)
See: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2324
Closes: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2278
2024-06-23 15:40:13 +02:00
Andrew Robbins
1c3572dee2
Correct btrfs subvolume creation in install step (MR 2319)
btrfs can only create a single subvolume at a time.
2024-06-23 13:10:47 +02:00
Caleb Connolly
099bd2f4fa
treewide: remove rogue leading slashes (MR 2252)
One flaw of pathlib is that Path("/tmp/") / "/some/relative/path"
results in Path("/some/relative/path")....

Fix the places in the codebase where we get this wrong.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
139fd2b3e0
install: losetup: fix mount retry (MR 2252)
The retry mechanism here go lost along the way. Bring it back.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ceaad5c291
build: move run_abuild and friends to a new file (MR 2252)
_package.py was getting pretty big. split the actual build steps out.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4108b95d72
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
f331b95824
chroot: allow mounting the device rootfs (MR 2252)
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".

For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.

The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c061ef8123
install: losetup: add detach_all() (MR 2252)
Add a function that just detaches all loop devices that are backed by a
file inside the pmbootstrap work dir.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
f2ca9c618e
ruff: run check --fix (MR 2252)
Get rid of unused imports and such

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
6087a9df8f
core: don't re-export get_context (MR 2252)
This makes testing a lot more annoying.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
6857882cf0
build: finish builder rework (MR 2252)
Rename build.package() to build.packages() and take a list of packages
to build, since every caller was inside a for loop this simplifies
usage and let's us give nicer log output by doing all the builds first,
so log messages don't get lost in the middle.

Behaviour is cleaned up so this shouuuuld work pretty well now. It
properly descends into dependencies and will build dependencies even if
the package given doesn't need building. Technically this was only done
before during install where the dependencies were recursed in
chroot.apk.install().

It probably makes the most sense to have a mode where it doesn't build
dependencies but warns the user about it, at least when invoked via
pmbootstrap build.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
e00d2a8e6d
install: don't call build in create_device_rootfs() (MR 2252)
The call to chroot.apk.install() will automatically build packages, so
we don't need to call it again beforehand.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
dbd0e97895
install: get_selected_providers: demote logging (MR 2252)
verbose is probably more suitable here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
cc85ef70f5
install: losetup.mount() remove args (MR 2252)
not needed here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
ef38c12e39
install: handle chroot unmount/mount properly (MR 2252)
We used to called pmb.chroot.shutdown() with "only_install_related=True"
here to unmount everything in the chroot so we could reliably "df" it to
figure out the rootfs image size.

Since ("pmb.chroot: only init once"), the chroot is no longer
automatically spun back up when we run mkinitfs, and the in-pmbootstrap
marker isn't recreated.

We only actually need to unmount the chroot anyway, so let's restrict it
to just that and remount things again afterwards.

This avoids weird side effects (like the native chroot being
shutdown?!), but this is something we should look into more in the
future since we're still doing a finnicky balancing act to manage chroot
state here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
53a2953a4a
chroot: mount: drop args from remove_mnt_pmbootstrap (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
866e5bcfab
core: add an Arch type (MR 2252)
Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.

The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
450bc6873d
core: move Config type (MR 2252)
Keeping the Config class in types seemed kinda weird and was just done
as a workaround to some cyclical imports. But now things are more in
shape let's move it to core.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
97bd8b96ec
parse: deviceinfo: make Deviceinfo a class (MR 2252)
Introduce a Deviceinfo class and use it rather than the dictionary. This
gives us sweet sweet autocomplete, and lays the foundation for having a
proper deviceinfo validator in the future.

Additionally, continue refactoring out args...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
b51d31acab
flasher: variables: rename vars to fvars (MR 2252)
vars is a reserved keyword in python, use fvars instead.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
de4c912692
WIP: 2024-06-05: args hacking and more (MR 2252)
Continue removing args and do some other optimisations.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
48cd886401
more wip (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00