Since cross-native2 was introduced, pmbootstrap now tells the user which
host and which build dependencies are getting installed. This makes
sense if they get installed into separate chroots, but not if they get
installed into the same chroot:
$ pmbootstrap build hello-world-wrapper --force
[16:47:19] Building 1 package
[16:47:19] * edge/hello-world-wrapper
[16:47:19] => (1/1) edge/hello-world-wrapper: Installing dependencies
[16:47:19] *** Install host dependencies
[16:47:19] (native) install hello-world
[16:47:20] *** Install build dependencies
[16:47:20] (native) install hello-world
[16:47:22] => edge/hello-world-wrapper: Building package
Don't list the depends separately if we are installing them into the
same chroot:
$ pmbootstrap build hello-world-wrapper --force
[16:51:35] Building 1 package
[16:51:35] * edge/hello-world-wrapper
[16:51:35] => (1/1) edge/hello-world-wrapper: Installing dependencies
[16:51:35] (native) install hello-world
[16:51:36] => edge/hello-world-wrapper: Building package
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2589
In strict mode, pmbootstrap uninstalls packages between builds. This was
broken since we switched from installing dependencies via abuild to
pmbootstrap since our code doesn't use "--virtual .makedepends-…" when
installing the makedepends:
(088776) [15:32:34] (native) % cd /home/pmos/build; busybox su pmos -c SUDO_APK='abuild-apk --no-progress' HOME=/home/pmos abuild undeps ;
ERROR: No such package: .makedepends-systemd-stage0
Instead of restoring the logic and worrying about uninstalling deps in
the right chroots, just zap all chroots in strict mode.
I've also considered implementing "--virtual .makedepends-…" again, but
running zap on the chroots makes the logic much simpler, is easier to
maintain and makes so little speed difference that it is worth the
trade-off. We don't need to pass additional parameters fore the 2nd
chroot to zap in cross-native2, no need to figure out the virtual
package name in two places, etc.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2587
When an APKBUILD has no makedepends, depends_build would be empty so we
would skip adding rsync when building with --src.
Fix this logic so we always add rsync even when there are no other
makedepends.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2581
Splitting makedepends in _host and _build only makes a difference when
building with cross-native2, as these packages end up in different
chroots.
For QEMU only and crossdirect we install all makedepends into the
foreign chroot. For cross-native (v1) we install everything into the
native chroot.
Show the warning only for cross-native2 so people don't think they have
to adjust their APKBUILDs here. Most of Alpine's APKBUILDs currently
don't have this split either.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2571
* Make CrossCompile a proper enum type rather than a string literal,
* Introduce methods to get the correct host/build chroots depending on the
cross compile type and target architecture.
* Remove autodetect.chroot() since it doesn't do what we expect, adjust
all users to use cross.build_chroot() instead.
* Refactor package building to correctly use cross.host_chroot() and
cross.build_chroot().
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2568
Replace None with the three things it is actually used for to make the
code easier to follow:
* "autodetect"
* "unnecessary"
* "qemu-only"
I've used the term "unnecessary" instead of "native", because "native"
was previously used for "cross-native", then "cross-native2" and it
still sounds similar to these.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2568
Fixes a bug where checkdepends are not being installed, and packages
that run tests that depend on something in checkdepends fail because
it's missing. For example, the `plasma-workspace` forked pkg in pmaports
tells abuild to run `xwfb-run` in `check()`, this binary comes from
`xwayland-run`, which is listed in the `checkdepends`. However this
isn't being installed and so `pmb build` ultimately fails to build the
package.
Set things up so that we can run abuild on the native chroot and use
it's cross compilation features rather than running it and the build
system through QEMU. This massively speeds up building when it works.
cross-native used to be quite limited in functionality and didn't
integrate into abuild itself, this commit fixes that.
Packages can opt-in to this by adding pmb:cross-native2 to their options
and configuring makedepends_host and makedepends_build.
This also speeds up building packages like postmarketos-initramfs since
it entirely avoids running commands through QEMU (usually abuild itself
would be run through QEMU).
Lastly, we preserve the old pmb:cross-kernel options, this can be used to
enable the old cross compiler behaviour which is used for cross
compiling kernels in pmaports. This allows them to keep being supporting
while we adapt them to the new cross-native2.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Replace aports -> pmaports in these messages.
$ pmbootstrap build systemd
[17:37:02] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:37:02] ERROR: Could not find package 'systemd' in pmaports
Some packages in aports have their own subpackages in their
dependencies, for some reason. This causes our resolver to assume there
is a cyclical dependency.
Adjust the resolver to safely ignore these dependencies, since they'll
be built as a part of the package that depends on them.
For example, modemmanager depends on libmm-glib which is a subpackage of
modemmanager. Currently to build modemmanager in pmaports it would be
necessary to manually modify the APKBUILD after copying it from aports.
This change avoids that.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
It may be desirable to test older versions of packages, allow them to be
built when --force is specified and print a warning that they won't be
installed automatically.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Here's the problem: Imagine a queue with 2 packages in it, the 1st
package uses crossdirect and the second uses cross-native. When building
the 1st package, pmb will configure the native chroot for crossdirect
as expected. When it gets to the 2nd package, the chroot env/config
might not be appropriate for actually doing a native cross compile.
This re-inits the cross compiler stuff if the cross compile method
changes while processing the queue.
Another approach that might solve this problem is to not re-use chroots
when building packages... I didn't think this was a good way to go
because it would greatly increase runtime (having to recreate chroots
multiple times)
When a dependency cannot be found in source or binary packages of the
selected architecture, then don't look for binary packages in other
architectures package indexes.
This leads to a confusing error down the line when building packages for
extra_repos_systemd (and e.g. stable branches), because we don't have
binary packages for all arches that pmbootstrap supports, and so it
tries to fetch a non-existing APKINDEX.
Instead, print a nice error about a non-existing dependency.
Adjust the dependency resolver, so it uses the main pkgnames in
dependency resolution, instead of subpackages. This fixes for example
that it couldn't find -dev packages even though the binary packages for
those existed.
> systemd: missing dependency linux-pam-dev, trying to queue other packages first
> linux-pam: missing dependency systemd-dev, trying to queue other packages first
> dbus: missing dependency systemd-dev, trying to queue other packages first
Have messages like the following in "pmbootstrap log" to better
understand why resolving packages fails. The bug with -dev pkgs not
being recognized will be fixed in a follow-up patch.
> systemd: missing dependency linux-pam-dev, trying to queue other packages first
> linux-pam: missing dependency systemd-dev, trying to queue other packages first
> dbus: missing dependency systemd-dev, trying to queue other packages first
Print a warning when breaking cyclical dependencies, for example with
current kde-nightly branch:
> WARNING: cyclical build dependency, building kirigami with binary package of qqc2-desktop-style
Set "stuck = False" in the "while disarray and not stuck" loop
directly, instead of setting it in the queue_item() function that gets
called from it. This way the loop logic can be understood without
looking at queue_item().
Most of the time when building it's reasonably safe to fall back on an
older binary package for building in order to break cyclical
dependencies (e.g. dbus and systemd depend on each other at build time,
but it's safe to build both against older versions of the other).
Implement the necessary logic to break cyclical dependencies by
detecting them and checking for the existence of a binary package.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
Decide the order to build packages in by looking through the build queue
and making sure dependencies are queued before any packages that depend
on them, bailing out in the case of cyclical dependencies.
This should handle subpackages too.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
When building with --src or --force, other logic to detect when to build
a package is skipped. However there are no (known) cases when someone
might want to build a package if the resulting binary package version
will be older than an existing binary package. Catch this case and fail
with recommendations for user action (update pmaports, clear local
binary repo).
This should help avoid the situation where you accidentally build an old
version of a package and it never actually gets installed.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
This function is really hard to understand, try to make this a bit
better by adjusting the wording to better reflect what it does.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
build_packages like apk-tools and alpine-base might get picked up and
queued for build as dependencies of other packages. We don't want this
since we always ensure they are queued first. Avoid queuing them in this
case.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
It turns out the current behaviour was not preserving the old behaviour
at all, and was wrong. We want to build ALL outdated or new packages we
encounter via the dependency graph.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We handle this correctly for dependencies of the base package but not
the base package itself.... Fix this.
Should avoid duplication here ideally.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
A previous attempt inserted subpackage depends into the package depends
list returned by get_depends().
This meant that pmb would try to install subpackage depends when
building the package, which breaks for some cases like device packages
with kernel variants since the depends of different subpackages might
conflict with each other.
Let's instead just add the subpackage depends to the build queue, so
they get built like package depends, without being pulled in as build
dependencies for the package.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Bail on empty build queue after checking if build_packages need
building. So if apk-tools / alpine-base / etc are outdated but nothing
else is they'll still be built.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
