For UI packages that have pmb:systemd-never set:
* do not ask in "pmbootstrap init" whether to use systemd or not
* do not install systemd, even if the systemd option is set to "always"
The point of this is to not add maintenance burden to Sxmo (and possibly
other UIs who not wish to support systemd with postmarketOS), so users
don't install with this unsupported use case and report issues with it.
Upgrade packages in the chroot, in case alpine-base, apk etc. have been
built from source with pmbootstrap. We build it from source for systemd
currently, and sometimes it is useful to do that to debug apk.
Add this argument, so we can disable the pmOS repository during
"pmbootstrap repo_bootstrap" for the chroot we build in.
If building natively, it will be disabled in the "native" chroot.
If building for a foreign arch, it will be disabled in the
"buildroot_$arch" chroot, but still be enabled in the native chroot so
we have the cross compilers available.
The name of the argument is a bit long, but it is consistent with the
argument of the same name in pmb.helpers.repo.urls() (to which it gets
passed).
Add the following question to "pmbootstrap init":
[22:12:57] Based on your UI selection, 'default' will result in installing systemd.
[22:12:57] Install systemd? (default/always/never) [default]:
Determine whether the UI prefers to have systemd or not, based on
"pmb:systemd" in the UI package's APKBUILD.
Determine whether the currently selected branch supports systemd, by
checking for a "[repo:systemd]" section in pmaports.cfg. This section
will also contain bootstrap information, to be used in future patches.
We have been discussing making device package versioning more
consistent and ended up settling on using integer versioning for device
packages[1]. As such, let's start by tackling this at the source and
start generating new device packages with 1 as the starting pkgver
instead of 0.1.
[1]: https://gitlab.com/postmarketOS/pmaports/-/issues/2610#note_1796178414
If there are multiple sections to a subpackage declaration, the middle one
(item 1 in the sequence in this case) is the custom function name which we
should use instead of the deduced one.
For example:
$pkgname-something-separate:something_separate:noarch
Here, the subpackage is called $pkgname-something-separate, but function names
cannot contain hyphens, so it's instead given the custom name
something_separate where the hyphen is replaced by an underscore. At the end,
the architecture of the subpackage is also overriden as noarch instead of
whatever architecture the main package has.
However, it is also possible to only override architecture of a subpackage and
not the function name.
In such cases, we get this:
$pkgname-something::noarch
We still have a "middle section" (item 1 in the sequence), but it will be
empty. As such, we not only need to check whether there are more than 1
subpackage part, but also whether that extra subpackage part is an empty string
or not.
This fixes an issue where pmbootstrap would not end up just setting the
properties of subpackages with a set arch but no custom function as
None instead of giving them proper depends, install, pkgdesc, et cetera
properties.
Highlight messages like the following from pmbootstrap install:
[19:45:59] *** (1/4) PREPARE NATIVE CHROOT ***
[ci:skip-build]: already built successfully in CI
If the device name does not make a valid hostname, fall back to some
simple valid hostname. Some device names (e.g. 'generic-x86_64') are
valid names for apk packages and such, but are not valid host names. It
assumes that the user (real, or CI) doesn't care if it's unset and just
expects pmb to work and not conditionally crash based on the device they
selected.
Situations where the user sets an invalid hostname are still validated
separately and cause an exception so that they know their config was
wrong and to fit it.
This fixes an issue that came up in CI when doing `pmb config device
generic-x86_64` followed by `pmb install`.
I considered trying to convert invalid host names into valid ones, but I
didn't feel like it was worth the trouble of adding stuff to convert and
re-validate when we can just set a static, boring, but definitely valid
default if the device name is not valid.
Due to that logging.verbose is "monkeypatched" into logging and not
present by default in the module, Mypy isn't able to pick up that it
exists. As such, just disable the check altogether in this file for
now.
The return value of this function is never used. Additionally, the type
hint for it was incorrect as it may return None. As such, don't bother
returning anything both to appease Mypy and also to remove unused
and consequently presumably untested functionality.
Repurposing argcomplete as a boolean telling whether the module
argcomplete was found results in a variable with two distinct purposes
and types, which is bad practice and confuses Mypy. As such, don't
assign False to argcomplete if the module is not found and instead just
check for whether it exists in sys.modules to determine whether it was
imported successfully.
This drops the prompt for using non-free firmware in images. The logic
for searching/installing non-free fw subpackages for devices is kept,
and will always be installed. This is to support the many device
packages in pmaports that still have nonfree-firmware subpackages. Going
forward, device packages can list firmware in `depends=` (for required
fw) or `pmb_recommends` (for optional fw).
nonfree-userland wasn't used in pmaports as far as I could find.
Sometimes dependencies of packages we are explicitly adding to world
define selected providers that we should account for. This improves
get_selected_providers to discover/add these selected providers in
dependencies too, recursively.
Fixes#2306
Only install the abuild-apk wrapper if cpu emulation is required. This
fixes building for x86 on x86_64.
Fix for:
>>> postmarketos-base-ui-gnome: Analyzing dependencies...
/usr/local/bin/abuild-apk: line 11: /native/usr/bin/abuild-apk: not found
Fixes: c5ca06d5 ("pmb: only enable abuild-apk wrapper for buildroot (MR 2246)")
we're not in the nftables only world yet, things like docker and
tailscale only work with the `iptables` command. those programs
expect iptable-modules, which we make available to nftables over
the nftables/iptables compat layer.
Drop the weird flag file stuff for state management, and just always
mount the source code in, and always unmount it on exit - including in
the error path.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When running build --force, if a later package is a dependency of an
earlier package, it will be "visited" by the dependency resolver,
erroneously causing it to be skipped when later in the session it's
visited to force build. This is because previously visited packages are
marked as such in a hashmap, but skip_already_built() assumes that a
package which has been visited would have already been built.
Fix this by overriding skip_already_built() if doing a force build. This
works because package dependencies are only built if the APKBUILD
version is newer than the binary repo, even when --force is specified.
So there is no risk of an infinite loop here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Some packages like kernels are cross compiled from the native chroot.
the apk_wrapper won't work here! So only enable it for buildroot chroots
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Why
Btrfs has some goodness (snapshots, switching between different rw snapshot)
which plays particularly well with certain "subvolume layouts".
What
This MR seeks to implement such a layout, namely a flat btrfs layout,
where the top level subvolume (i.e. the btrfs filesystem/partition itself)
remains unmounted in all situations,
except when making changes to direct child subvolumes of the filesystem.
- rename all subvols to follow the common @* btrfs subvol naming scheme.
- add subvol @root, because roots home directory shouldn't be rolled back.
- make subvol @var not Copy-on-Write (nodatacow) to avoid write
- multiplication on logs, VMs, databases, containers and flatpaks.
- add subvol @snapshots because that lets us change the root subvol to a
read-write snapshot of @ without affecting snapshots.
- add subvol @srv because it contains data for Web and FTP servers,
which shouldn't roll back together with root subvol.
- add subvol @tmp because we don't want to snapshot temporary files.
This subvol remains unmounted on the device,
unless conditions as laid out in pmaports!4737 are met.
- add check and error for btrfs when using rsync installation.
Speed up using abuild to install build dependencies by introducing a
wrapper which invokes abuild-apk in the native chroot with
LD_PRELOAD_PATH
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Fix "pmbootstrap chroot" and others not passing the proxy environment
variables correctly. Thanks to notfound405 for pointing this out!
Instead of only preserving proxy environment variables in
pmb.helpers.run_core, which should never be called directly, do it in
the calling functions:
* pmb.helpers.run.user
* pmb.helpers.run.root
* pmb.chroot.root
* pmb.chroot.user
This fixes that the environment variables were only really passed by
pmb.helpers.run.user, because the other functions would result in
something like:
HTTP_PROXY=mytestproxy sudo env -i /usr/bin/sh -c '…'
This is needed to either elevate to root, or to elevate to root first
and then enter the chroot as root or user. Due to the "env -i", the
environment intentionally gets cleaned, but unintentionally also removes
the proxy environment variables that were explicitly set.
By adjusting the functions, they now run a variant of:
sudo env -i /usr/bin/sh -c 'HTTP_PROXY=mytestproxy …'
The escaping is simplified in this example, run "pmbootstrap -v" to see
the not very readable, but proper escaping with shutil.quote().
Remove the previous test for preserving the environment variables in
pmb.helpers.run_core (as it should never be called directly), and test
instead the new behavior.
Fixes: issue 2299
Fixes: 13c4ac42 ("pmb.helpers.run_core: fix proxy env var logic")
find_kbuild_output_dir() searches for certain path patterns, which
won't be in the APKBUILD if it uses downstreamkernel_package. Guess
the default kbuild out dir in that case.
Theoretically this might break if some APKBUILD passes weird paths to
downstreamkernel_package, but no kernel package in pmaports today does
that.
