Commit graph

335 commits

Author SHA1 Message Date
Dominik Bayerl
87515529cf
pmbootstrap: run partprobe after repartitioning (MR 2473)
On some kernels (notably, WSL2) parted fails to inform the kernel
about the new partition layout, leading to errors when trying to
create the new filesystems.

This change runs an additional `partprobe` after updating the
partition table, informing the kernel about the new layout.

Fixes: #2422
Signed-off-by: Dominik Bayerl <dominik.bayerl@carissma.eu>
[caleb: move call so it runs after partition_cgpt() as well]
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-11-04 08:09:37 +01:00
Newbyte
225d8b30a0
pmb: Add lots of type hints (MR 2464) 2024-10-30 12:39:45 +01:00
Newbyte
d05d57b37e
pmb.install.recovery: Assert that partitions are not None (MR 2464)
They should definitely not be None here, thus assert it so we don't need
to keep checking for None later.
2024-10-29 23:08:37 +01:00
Newbyte
b49da9ad82
pmb.install.recovery: Check for None before going through blacklist (MR 2464)
It doesn't really make sense to check the blacklist against None, so
just skip the value if it's None. Appeases mypy.
2024-10-29 23:08:37 +01:00
Newbyte
8bc848fa77
pmb.install._install: Use bool(...) when figuring out whether to use cgpt (MR 2464)
Otherwise, if pmb.parse.deviceinfo().cgpt_kpart is an empty string, for
example, we end up passing an empty string to get_partition_layout()
instead of False.
2024-10-29 23:08:37 +01:00
Newbyte
b0526128fe
pmb.install._install: Round root size to nearest int in get_subpartitions_size() (MR 2464)
This makes the behaviour match the docstring which specifies that this
function returns integers.
2024-10-29 23:08:36 +01:00
Newbyte
1aa0b05014
pmb.install.blockdevice: Avoid reusing size_mb variable in create_and_mount_image() (MR 2464)
Reusing this variable might lead to confusing bugs later if this code is
changed in the future, and also leads to mypy being confused about the
type of size_mb as it then gets assigned both str and int values.
2024-10-29 14:21:01 +01:00
Vikram Alagh
8ac2e48a21
install: losetup: check if the back-file attribute for a loop device is null (MR 2456)
Seems to be ChromeOS specific:

> this happens every time on my machine. I am on ChromeOS 129.0.6668.112
> using a Debian bookworm chroot through Crouton. I am not sure why
> this is happening nor which layer is responsible for this. I deleted
> the chroot and created another one but the problem persists.

Commit msg tweaked by Oliver.
2024-10-27 16:03:17 +01:00
Newbyte
3c47a52f91
pmb.install.losetup: Remove debug print (MR 2450)
Looks like an unintended leftover from debugging.

---

This was needlessly reverted by
1ec55fc11a. While revreting
0975d06437 was good, this commit did not
cause any harm.
2024-10-27 13:14:11 +01:00
Newbyte
bbf20a030b
pmb.install.losetup: Annotate return types (MR 2430) (MR 2450)
And fix type errors.

---

This was needlessly reverted by
1ec55fc11a. While revreting
0975d06437 was good, this commit did not
cause any harm.
2024-10-27 13:14:11 +01:00
Newbyte
37ec73c07c
pmb.flasher: Remove use of args (MR 2441)
Also adapt pmb/install/recovery.py to new API for variables.py.

[ci:skip-build]: already built successfully in CI
2024-10-22 15:12:40 +02:00
Clayton Craft
47dc493701
pmb.parse.depends.package_provider: add type hinting and fix references (MR 2439)
pmb.parse.apkindex.providers now returns a dict of string:ApkindexBlock,
probably from 71772b9b6

This adds type hinting to package_provider to reflect this, and fixes
code that calls it to properly deal with the possible return types.
Without this, building FDE images is broken.
2024-10-18 13:45:57 +02:00
Clayton Craft
1ec55fc11a
Revert "pmb.install.losetup: Run kpartx, losetup in chroot (MR 2430)"
This series breaks pmbootstrap, and that's not good[1]. A bug was filed
about this (#2465), and there have been multiple attempts to fix it (!
2435 and !2436). It kinda seems like we don't have time to fix/test this
for a while longer, which is fine, but given the impact this bug has I
think we should be revert this series until this issue is solved/tested.

1. pmb is broken in some specific cases, which means some workflows are
broken. One example is that this breaks the pmaports CI, so no work can
be done in pmaports.
2024-10-16 11:35:36 -07:00
Newbyte
a7e2592f1a
pmb.install.losetup: Remove debug print (MR 2430)
Looks like an unintended leftover from debugging.
2024-10-16 00:07:48 +02:00
Newbyte
0975d06437
pmb.install.losetup: Run kpartx, losetup in chroot (MR 2430)
I tested $ pmbootstrap shutdown with this and observed no problems. All
the mounts disappear in lsblk as expected.

Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2423
2024-10-16 00:07:48 +02:00
Newbyte
6455441464
pmb.install.losetup: Annotate return types (MR 2430)
And fix type errors.
2024-10-16 00:07:44 +02:00
Stefan Hansson
71772b9b6b
pmb.parse.apkindex: Introduce proper typing (MR 2425)
And adjust other code.

Closes https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2455
2024-10-13 20:12:34 +02:00
Aster Boese
8335d050bf
pmb.install: switch to gpt with discoverable partitions (MR 2426) 2024-10-13 19:59:26 +02:00
Oliver Smith
eb747ad21f
find_providers: tweak default matching (MR 2301)
The previous version would fail if the suffix contained a dash as well,
e.g. postmarketos-base-ui-audio-some-new-thing. Fix it and add example
comments.
2024-09-30 18:41:09 +02:00
JustSoup321
56a4329f4f
pmb.{config, helpers, install}: add _pmb_default support (MR 2301) 2024-09-30 18:37:01 +02:00
Clayton Craft
36c86cd971
install: create_device_rootfs: install pmos-base-systemd first (MR 2383)
When building a systemd image, this patch installs pmos-base-systemd
(and its dependencies) first before building the rest of the device
chroot. This is a workaround for...

The pmos-base-systemd package provides distro-wide systemd presets for
services, and when it's added to the `install_packages` list then
there's no guarantee that apk will install it *before* service packages
execute `systemctl preset`. In other words: `preset` may be run before
the distro presets are actually installed, and crushing expectations. By
making sure pmos-base-systemd is installed in the chroot first, we can
provide some guarantee that presets will be there when services are
installed a bit later.
2024-09-02 10:27:38 -07:00
JustSoup321
375814b9d2
pmb.{chroot, install}: modify for bsdutils (MR 2315)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-09-01 15:45:06 +02:00
JustSoup321
3559ee4030
pmb.init: make default hostnames pass regex (MR 2393) 2024-09-01 15:01:02 +02:00
Caleb Connolly
f6f503035b
install: add a --sector-size flag (MR 2391)
This can be used when building images for generic device targets that
support devices with different sector size requirements.

For example, trailblazer prebuilts are currently expected to be flashed
to a USB drive where a 4096 sector size would be unsuitable since the
bootloader wouldn't detect it. But when building for a Qualcomm phone,
one would use --split and --sector-size to build the root and boot
partitions with a 4k sector size which is appropriate to the UFS
storage.

This flag could also be used by BPO to build both variants.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-08-25 22:06:17 +02:00
Caleb Connolly
0b0172bafc
install: add flash_boot for fastboot-bootpart (MR 2378)
When using fastboot-bootpart flash method to split root/boot partitions
out, it was no longer possible to flash the Android boot partition.

Add a new flash_boot action to handle this.

This will allow us to migrate devices like the OnePlus 6 to have the
boot partition (ESP) on system, and the rootfs on userdata rather than
the current subpartition method.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 12:53:28 +02:00
Caleb Connolly
c23a9ea9cd
install: fix root partition path in print (MR 2377)
When we output the root partition path, it says rootfs.img but it should
be root.img fix it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-20 17:33:02 +02:00
Oliver Smith
36dd53f402
Run ruff check --fix (MR 2357)
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
2024-07-16 00:26:35 +02:00
HenriDellal
2b577a8694
Fix pmbootstrap install --android-recovery-zip (MR 2368)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-07-16 00:04:15 +02:00
Caleb Connolly
0e62eaac67
ruff: format (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:09:42 +02:00
Caleb Connolly
65d23905bc
treewide: another round of mypy fixes (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:13 +02:00
Caleb Connolly
6301212d99
install: partition: drop unused args (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:12 +02:00
Oliver Smith
18fa4e58a3
Ruff: fix typing.Xxx is deprecated, use xxx instead (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
cebd784d84
Prefer f-strings (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
f3f392ef66
Use simplified Python 3.8 syntax (MR 2327)
This commit was generated with:

    ruff check --fix --extend-select=UP .
2024-06-23 19:13:52 +02:00
Hugo Osvaldo Barrera
e421bb2d41
Auto-format codebase with ruff (MR 2325)
See: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2324
Closes: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2278
2024-06-23 15:40:13 +02:00
Andrew Robbins
1c3572dee2
Correct btrfs subvolume creation in install step (MR 2319)
btrfs can only create a single subvolume at a time.
2024-06-23 13:10:47 +02:00
Caleb Connolly
099bd2f4fa
treewide: remove rogue leading slashes (MR 2252)
One flaw of pathlib is that Path("/tmp/") / "/some/relative/path"
results in Path("/some/relative/path")....

Fix the places in the codebase where we get this wrong.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
139fd2b3e0
install: losetup: fix mount retry (MR 2252)
The retry mechanism here go lost along the way. Bring it back.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ceaad5c291
build: move run_abuild and friends to a new file (MR 2252)
_package.py was getting pretty big. split the actual build steps out.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4108b95d72
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
f331b95824
chroot: allow mounting the device rootfs (MR 2252)
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".

For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.

The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
c061ef8123
install: losetup: add detach_all() (MR 2252)
Add a function that just detaches all loop devices that are backed by a
file inside the pmbootstrap work dir.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
f2ca9c618e
ruff: run check --fix (MR 2252)
Get rid of unused imports and such

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
6087a9df8f
core: don't re-export get_context (MR 2252)
This makes testing a lot more annoying.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
6857882cf0
build: finish builder rework (MR 2252)
Rename build.package() to build.packages() and take a list of packages
to build, since every caller was inside a for loop this simplifies
usage and let's us give nicer log output by doing all the builds first,
so log messages don't get lost in the middle.

Behaviour is cleaned up so this shouuuuld work pretty well now. It
properly descends into dependencies and will build dependencies even if
the package given doesn't need building. Technically this was only done
before during install where the dependencies were recursed in
chroot.apk.install().

It probably makes the most sense to have a mode where it doesn't build
dependencies but warns the user about it, at least when invoked via
pmbootstrap build.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
e00d2a8e6d
install: don't call build in create_device_rootfs() (MR 2252)
The call to chroot.apk.install() will automatically build packages, so
we don't need to call it again beforehand.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
dbd0e97895
install: get_selected_providers: demote logging (MR 2252)
verbose is probably more suitable here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
cc85ef70f5
install: losetup.mount() remove args (MR 2252)
not needed here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
ef38c12e39
install: handle chroot unmount/mount properly (MR 2252)
We used to called pmb.chroot.shutdown() with "only_install_related=True"
here to unmount everything in the chroot so we could reliably "df" it to
figure out the rootfs image size.

Since ("pmb.chroot: only init once"), the chroot is no longer
automatically spun back up when we run mkinitfs, and the in-pmbootstrap
marker isn't recreated.

We only actually need to unmount the chroot anyway, so let's restrict it
to just that and remount things again afterwards.

This avoids weird side effects (like the native chroot being
shutdown?!), but this is something we should look into more in the
future since we're still doing a finnicky balancing act to manage chroot
state here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
53a2953a4a
chroot: mount: drop args from remove_mnt_pmbootstrap (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00