Debug is the default loglevel, demote the "already visited" message to
verbose, as it isn't generally useful.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We use a custom verbose log level in pmbootstrap, unfortunately it isn't
possible to correctly type this due to some limitations in the logging
library [1], [2].
Given that our usecase is fairly simple, we can just wrap the module
with our own so we only have to tell mypy to ignore the error once
instead of at every callsite.
[1]: https://github.com/cryptax/droidlysis/issues/15
[2]: https://github.com/python/typing/discussions/980
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a new module: pmb.core to contain explicitly typed pmbootstrap
API. The first component being Suffix and SuffixType. This explicitly
defines what suffixes are possible, future changes should aim to further
constrain this API (e.g. by validating against available device
codenames or architectures for buildroot suffixes).
Additionally, migrate the entire codebase over to using pathlib.Path.
This is a relatively new part of the Python standard library that uses a
more object oriented model for path handling. It also uses strong type
hinting and has other features that make it much cleaner and easier to
work with than pure f-strings. The Chroot class overloads the "/"
operator the same way the Path object does, allowing one to write paths
relative to a given chroot as:
builddir = chroot / "home/pmos/build"
The Chroot class also has a string representation ("native", or
"rootfs_valve-jupiter"), and a .path property for directly accessing the
absolute path (as a Path object).
The general idea here is to encapsulate common patterns into type hinted
code, and gradually reduce the amount of assumptions made around the
codebase so that future changes are easier to implement.
As the chroot suffixes are now part of the Chroot class, we also
implement validation for them, this encodes the rules on suffix naming
and will cause a runtime exception if a suffix doesn't follow the rules.
linux-lts and linux-edge in alpine use "utf-8", that might change, but
add the options regardless, so we can be safe (people can always edit them!)
Fixes https://gitlab.com/postmarketOS/pmaports/-/issues/2782
[ci:skip-build]: already built successfully in CI
We don't want executables, suid executables, or devices. We neither
want symbolic links in fat partitions. These have been taken from
systemd
Ref: pmaports#2782
If the device name does not make a valid hostname, fall back to some
simple valid hostname. Some device names (e.g. 'generic-x86_64') are
valid names for apk packages and such, but are not valid host names. It
assumes that the user (real, or CI) doesn't care if it's unset and just
expects pmb to work and not conditionally crash based on the device they
selected.
Situations where the user sets an invalid hostname are still validated
separately and cause an exception so that they know their config was
wrong and to fit it.
This fixes an issue that came up in CI when doing `pmb config device
generic-x86_64` followed by `pmb install`.
I considered trying to convert invalid host names into valid ones, but I
didn't feel like it was worth the trouble of adding stuff to convert and
re-validate when we can just set a static, boring, but definitely valid
default if the device name is not valid.
This drops the prompt for using non-free firmware in images. The logic
for searching/installing non-free fw subpackages for devices is kept,
and will always be installed. This is to support the many device
packages in pmaports that still have nonfree-firmware subpackages. Going
forward, device packages can list firmware in `depends=` (for required
fw) or `pmb_recommends` (for optional fw).
nonfree-userland wasn't used in pmaports as far as I could find.
Sometimes dependencies of packages we are explicitly adding to world
define selected providers that we should account for. This improves
get_selected_providers to discover/add these selected providers in
dependencies too, recursively.
Fixes#2306
Why
Btrfs has some goodness (snapshots, switching between different rw snapshot)
which plays particularly well with certain "subvolume layouts".
What
This MR seeks to implement such a layout, namely a flat btrfs layout,
where the top level subvolume (i.e. the btrfs filesystem/partition itself)
remains unmounted in all situations,
except when making changes to direct child subvolumes of the filesystem.
- rename all subvols to follow the common @* btrfs subvol naming scheme.
- add subvol @root, because roots home directory shouldn't be rolled back.
- make subvol @var not Copy-on-Write (nodatacow) to avoid write
- multiplication on logs, VMs, databases, containers and flatpaks.
- add subvol @snapshots because that lets us change the root subvol to a
read-write snapshot of @ without affecting snapshots.
- add subvol @srv because it contains data for Web and FTP servers,
which shouldn't roll back together with root subvol.
- add subvol @tmp because we don't want to snapshot temporary files.
This subvol remains unmounted on the device,
unless conditions as laid out in pmaports!4737 are met.
- add check and error for btrfs when using rsync installation.
I zap chroots a lot, since I've found that it often "fixes" a lot of
weird issues that come about if you have stale chroots laying around.
So a common pattern I do is "pmb zap && pmb install ...". Having an
option to pmb install let's me simplify this.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231214234051.4673-1-clayton@craftyguy.net%3E
Make sure the user has at least 256 MiB set as their installation size,
refuse to start the installation otherwise. The default was changed in
2021, 03e9fb05 ("pmb.config.init.boot_size: set to 256 MiB (MR 2037)").
If the user ran "pmbootstrap init" before that commit, the pmbootstrap
config will have the old default set. It is very annoying when you do an
installation with it and only realize it when you run into errors, e.g.
while upgrading. I had that when testing the upgrade to the v23.12
release and also adjusted postmarketos-release-upgrade to warn if the
boot partition is smaller than expected.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231208222714.4601-1-ollieparanoid@postmarketos.org%3E
Having AppStream data is great for installation that make use of GNOME
Software or KDE Discover, however it's not a must. The generation of
AppStream data in alpine (which we maintain) is still certainly
improvable, and we have bumped into problems, unreliabilities and such
from time to time. Installations without AppStream data are
totally functional, even if the first experience on those apps is not
the best. Still, users are one refresh away on Software or Discover
from getting such data downloaded. So failing to generate installations
for this reason is unnecessarily breaking.
Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231205160205.13916-1-pabloyoyoista@postmarketos.org%3E
This makes sure that the "/etc/localtime" file being created points to
the tzdata directory (/usr/share/zoneinfo), instead of to the custom
"/etc/zoneinfo" created by alpine to save space. This is relevant
because it otherwise will point to a directory that contains
incomplete tzdata, and can produce unexpected results in some circumnstances.
Signed-off-by: Pablo Correa Gómez <ablocorrea@hotmail.com>
Tested-by: Clayton Craft <clayton@craftyguy.net>
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230611150743.23310-3-ablocorrea@hotmail.com%3E
Deviceinfo variables are empty strings by default if they are unset
in the deviceinfo file, so the "is not None" check currently enables
the sm_sparse_image_tool for all existing devices using sparse format.
Simplify the check to convert the string to a bool. Empty strings
evaluate to False so this should work as originally intended.
Stock bootloader on these devices boots kernel (it is intended to boot
kernel, but i place secondary bootloader there) from special ChromeOS
kernel partition on special GPT which is created with cgpt utility.
This MR adds initial support for it introducing new deviceinfo options:
- cgpt_kpart - path to file to be flashed to ChromeOS partition;
- cgpt_kpart_start - offset from the start in sectors;
- cgpt_kpart_size - partition size in sectors.
For example:
deviceinfo_cgpt_kpart="/usr/share/u-boot/google-peach-pit/u-boot-dtb.img.kpart"
deviceinfo_cgpt_kpart_start="8192"
deviceinfo_cgpt_kpart_size="32768"
cgpt requires start and size values of partition, so these values
are calculated for each partition.
Reserved size and on-device installer are not yet supported.
Reference: https://archlinuxarm.org/platforms/armv7/samsung/samsung-chromebook
"Keeping track of which partition number is what is hard to understand now.
I think this should be refactored, so we have it defined only in one
place, and easy to read. Since this merge request increases the complexity
of the partitions again, let's do it here before merging." - Oliver
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
When running `pmbootstrap install`, pmbootstrap does not unlock root
when it completes. This patch allows `pmbootstrap install` to run two or
more times (without zap).
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
This removes the hard-coded installation of osk-sdl when specifying
--fde (or using the on-device installer) and instead determines the
unlocker package by taking the most suitable provider of
postmarketos-fde-unlocker (factoring provider priority and packages
selected for installation).
With this change applied, one can manually select an unlocker package to
be installed via
pmbootstrap install --fde --add other-unlocker
Relates to: postmarketOS/pmaports#1309
So I can finally run `pmbootstrap install --password 147147` and go and
make a cup of tea.
Based on MR 1919.
Co-Authored-By: Oliver Smith <ollieparanoid@postmarketos.org>
