Since cross-native2 was introduced, pmbootstrap now tells the user which
host and which build dependencies are getting installed. This makes
sense if they get installed into separate chroots, but not if they get
installed into the same chroot:
$ pmbootstrap build hello-world-wrapper --force
[16:47:19] Building 1 package
[16:47:19] * edge/hello-world-wrapper
[16:47:19] => (1/1) edge/hello-world-wrapper: Installing dependencies
[16:47:19] *** Install host dependencies
[16:47:19] (native) install hello-world
[16:47:20] *** Install build dependencies
[16:47:20] (native) install hello-world
[16:47:22] => edge/hello-world-wrapper: Building package
Don't list the depends separately if we are installing them into the
same chroot:
$ pmbootstrap build hello-world-wrapper --force
[16:51:35] Building 1 package
[16:51:35] * edge/hello-world-wrapper
[16:51:35] => (1/1) edge/hello-world-wrapper: Installing dependencies
[16:51:35] (native) install hello-world
[16:51:36] => edge/hello-world-wrapper: Building package
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2589
In strict mode, pmbootstrap uninstalls packages between builds. This was
broken since we switched from installing dependencies via abuild to
pmbootstrap since our code doesn't use "--virtual .makedepends-…" when
installing the makedepends:
(088776) [15:32:34] (native) % cd /home/pmos/build; busybox su pmos -c SUDO_APK='abuild-apk --no-progress' HOME=/home/pmos abuild undeps ;
ERROR: No such package: .makedepends-systemd-stage0
Instead of restoring the logic and worrying about uninstalling deps in
the right chroots, just zap all chroots in strict mode.
I've also considered implementing "--virtual .makedepends-…" again, but
running zap on the chroots makes the logic much simpler, is easier to
maintain and makes so little speed difference that it is worth the
trade-off. We don't need to pass additional parameters fore the 2nd
chroot to zap in cross-native2, no need to figure out the virtual
package name in two places, etc.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2587
When an APKBUILD has no makedepends, depends_build would be empty so we
would skip adding rsync when building with --src.
Fix this logic so we always add rsync even when there are no other
makedepends.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2581
We currently always build kernel packages for the primary channel, which
will usually be systemd-edge. This will cause sideload to fail since it
correctly only looks in the edge repo.
Update envkernel to get the proper channel name and put the kernel
there, so they will always be in the local edge repo and not the
systemd-edge one.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2572
Splitting makedepends in _host and _build only makes a difference when
building with cross-native2, as these packages end up in different
chroots.
For QEMU only and crossdirect we install all makedepends into the
foreign chroot. For cross-native (v1) we install everything into the
native chroot.
Show the warning only for cross-native2 so people don't think they have
to adjust their APKBUILDs here. Most of Alpine's APKBUILDs currently
don't have this split either.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2571
* Make CrossCompile a proper enum type rather than a string literal,
* Introduce methods to get the correct host/build chroots depending on the
cross compile type and target architecture.
* Remove autodetect.chroot() since it doesn't do what we expect, adjust
all users to use cross.build_chroot() instead.
* Refactor package building to correctly use cross.host_chroot() and
cross.build_chroot().
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2568
Replace None with the three things it is actually used for to make the
code easier to follow:
* "autodetect"
* "unnecessary"
* "qemu-only"
I've used the term "unnecessary" instead of "native", because "native"
was previously used for "cross-native", then "cross-native2" and it
still sounds similar to these.
Part-of: https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/merge_requests/2568
Cross compiling go programs with cross-native2 fails for armv7, riscv64
and armhf unless we use CGO_ENABLED=1 in the APKBUILD (see related
aports issue).
Add --sysroot to CGO_CFLAGS, so cgo can find the includes and doesn't
fail with:
go build -buildmode=pie -modcacherw -trimpath -buildvcs=false -ldflags "-s -w -X main.Version=2.6.1" -o mkinitfs ./cmd/mkinitfs
# runtime/cgo
_cgo_export.c:3:10: fatal error: stdlib.h: No such file or directory
Add it to LDFLAGS too for linking against libraries from the sysroot.
Related: https://gitlab.alpinelinux.org/alpine/aports/-/issues/15809
Related: https://stackoverflow.com/a/38835938
Fixes a bug where checkdepends are not being installed, and packages
that run tests that depend on something in checkdepends fail because
it's missing. For example, the `plasma-workspace` forked pkg in pmaports
tells abuild to run `xwfb-run` in `check()`, this binary comes from
`xwayland-run`, which is listed in the `checkdepends`. However this
isn't being installed and so `pmb build` ultimately fails to build the
package.
Set things up so that we can run abuild on the native chroot and use
it's cross compilation features rather than running it and the build
system through QEMU. This massively speeds up building when it works.
cross-native used to be quite limited in functionality and didn't
integrate into abuild itself, this commit fixes that.
Packages can opt-in to this by adding pmb:cross-native2 to their options
and configuring makedepends_host and makedepends_build.
This also speeds up building packages like postmarketos-initramfs since
it entirely avoids running commands through QEMU (usually abuild itself
would be run through QEMU).
Lastly, we preserve the old pmb:cross-kernel options, this can be used to
enable the old cross compiler behaviour which is used for cross
compiling kernels in pmaports. This allows them to keep being supporting
while we adapt them to the new cross-native2.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
I tried my best to get the right author based on git commit history, but
in the case of code being moved from one file to make another I might
have gotten something wrong.
the kernel has pending patches in -next which adjust the generated
Makefile in the output directory to include the source tree makefile
with an absolute path, this breaks envkernel which relies on it being a
relative path.
Fix this by mounting our own Makefile instead using the relative path.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Otherwise abuild will try and fail to create /dev/null
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
[ci:skip-build]: already built successfully in CI
recent changes to the kernels build infra in -next seem to have exposed
some issues with "pmbootstrap build --envkernel", specifically in the
case where the APKBUILD doesn't build out-of-tree.
We used to rely on the fact that the kernel output directory contains a
Makefile which points to the source directory, however this Makefile
now(?) contains an absolute path on the host, which won't match what's
in the chroot.
As a result, it's now necessary to build with the same output directory
as the APKBUILD.
We probably need some smarter code here... and/or more consistent
APKBUILDs.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Replace aports -> pmaports in these messages.
$ pmbootstrap build systemd
[17:37:02] NOTE: The package 'systemd' exists in extra-repos/systemd, but systemd is currently disabled
[17:37:02] ERROR: Could not find package 'systemd' in pmaports
There was an oversight when this API was originally created and it
implicitly assumed that the pmaports repository was always named
"pmaports". This unfortunately broke some peoples workflows.
Introduce a new "pkgrepo_name()" function and adjust the codebase to use
it, as well as adjusting pkgrepo internally to special case the
"pmaports" repo so that it's always named pmaports no matter what the
directory itself is named.
This is probably more complexity than we should be dealing with here, we
should probably create a new type to encode this behaviour.
Fixes: #2412
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
At some point, this function was updated to use the Arch type instead of
plain strings, but the return type was left omitted and the docstring
outdated.
Some packages in aports have their own subpackages in their
dependencies, for some reason. This causes our resolver to assume there
is a cyclical dependency.
Adjust the resolver to safely ignore these dependencies, since they'll
be built as a part of the package that depends on them.
For example, modemmanager depends on libmm-glib which is a subpackage of
modemmanager. Currently to build modemmanager in pmaports it would be
necessary to manually modify the APKBUILD after copying it from aports.
This change avoids that.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
It may be desirable to test older versions of packages, allow them to be
built when --force is specified and print a warning that they won't be
installed automatically.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
I don't think there's any case where the number of jobs would have to be
a string. It's also being assigned an integer elsewhere the code (in
ask_for_additional_options() inside of init.py), so an integer seems
like what we actually want. Also fix type errors resulting of this.
Here's the problem: Imagine a queue with 2 packages in it, the 1st
package uses crossdirect and the second uses cross-native. When building
the 1st package, pmb will configure the native chroot for crossdirect
as expected. When it gets to the 2nd package, the chroot env/config
might not be appropriate for actually doing a native cross compile.
This re-inits the cross compiler stuff if the cross compile method
changes while processing the queue.
Another approach that might solve this problem is to not re-use chroots
when building packages... I didn't think this was a good way to go
because it would greatly increase runtime (having to recreate chroots
multiple times)
