In order to get cross-compilers, we generate a few aports (e.g.
binutils-armhf, gcc-armhf) automatically from Alpine's aports.
pmbootstrap was already able to perform a git checkout of Alpine's
aports repository. But it needed to be manually updated. Otherwise
the `pmbootstrap aportgen` command could actually downgrade the aport
instead of updating it to the current version.
After thinking about adding a dedicated pmbootstrap command for
updating git repositories, I thought it would be better to not open
that can of worms (pmbootstrap as general git wrapper? no thanks).
The solution implemented here compares the upstream aport version of
the git checkout of a certain package (e.g. gcc for gcc-armhf) with the
version in Alpine's binary package APKINDEX. When the aport version is
lower than the binary package version, it shows the user how to update
the git repository with just one command:
pmbootstrap chroot --add=git --user -- \
git -C /mnt/pmbootstrap-git/aports_upstream pull
Changes:
* `pmb.aportgen.core.get_upstream_aport()`: new function, that returns
the absolute path to the upstream aport on disk, after checking the
version of the aport against the binary package.
* Use that new function in pmb.aportgen.gcc and pmb.aportgen.binutils
* New function `pmb.helpers.repo.alpine_apkindex_path()`: updates the
APKINDEX if necessary and returns the absolute path to the APKINDEX.
This code was basically present already, but not as function, so now
we have a bit less overhead there.
* `pmbootstrap chroot`: new `--user` argument
* `pmb.parse.apkbuild`: make pkgname check optional, as it fails with
the official gcc APKBUILD before we modify it (the current APKBUILD
parser is not meant to be perfect, as this would require a full shell
parsing implementation).
* Extended `test_aportgen.py` and enabled it by default in
`testcases_fast.sh`. Previously it was disabled due to traffic
concerns (cloning the aports repo, but then again we do a full KDE
plasma mobile installation in Travis now, so that shouldn't matter
too much).
* `testcases_fast.sh`: With "test_aport_in_sync_with_git" removed
from the disabled-by-default list (left over from timestamp based
rebuilds), there were no more test cases disabled by default. I've
changed it, so now the qemu_running_processes test case is disabled,
and added an `--all` parameter to the script to disable no test
cases. Travis runs with the `--all` parameter while it's useful to
do a quick local test without `--all` in roughly 2 minutes instead of
10.
* `aports/cross/binutils-*`: Fix `_mirror` variable to point to current
default Alpine mirror (so the aportgen testcase runs through).
### Only download APKINDEX for relevant architectures
We're downloading the APKINDEX files for all architectures supported by
postmarketOS currently (x86, x86_64, armhf, aarch64). Most of the time,
we only need it for the native and device arch, so this PR reduces the
downloaded files to what is really necessary.
### Intuitive pmbootstrap update logic
* pmb.helpers.repo.update():
* Default is updating all arches where the APKBUILD files exist
* Add existing_only parameter
* Return True when files have been downloaded
* Properly print which arches will be updated
* Print update reason only in verbose log
* Add and improve comments
* pmb.parse.arguments(), update action:
* Add --non-existing parameter
* Default for --arch is None (instead of arch.native)
* pmb.helpers.frontend.update():
* Inform about --non-existing if no APKBUILDs have been updated
Here are the changes necessary in pmbootstrap to make proprietary
software installed onto the device (firmware and userspace drivers)
optional (#756). To full close the issue, we need to apply this concept
to all device packages we already have in a follow-up PR.
Changes:
* New config file options nonfree_firmware and nonfree_userland, which
we ask for during "pmbootstrap init" if there are non-free components
for the selected device.
* We find that out by checking the APKBUILD's subpakages: The non-free
packages are called $pkgname-nonfree-firmware and
$pkgname-nonfree-userland.
* During "pmbootstrap init" we also show the pkgdesc of these
subpackages. Parsing that is implemented in
pmb.parse._apkbuild.subpkgdesc(). It was not implemented as part of
the regular APKBUILD parsing, as this would need a change in the
output format, and it is a lot *less* code if done like in this
commit.
* pmb/parse/apkbuild.py was renamed to _apkbuild.py, and
pmb/install/install.py to _install.py: needed to call the function in
the usual way (e.g. pmb.parse.apkbuild()) but still being able to
test the individual functions from these files in the test suite.
We did the same thing for pmb/build/_package.py already.
* Install: New function get_nonfree_packages() returns the non-free
packages that will be installed, based on the user's choice in
"pmbootstrap init" and on the subpackages the device has.
* Added test cases and test data (APKBUILDs) for all new code,
refactored test/test_questions.py to have multiple functions for
testing the various questions / question types from
"pmbootstrap init" instead of having it all in one big function.
This allows to use another aport folder for testing the new
non-free related questions in init.
* The APKINDEX parser used to return a dictionary with one package for
a given package name. This works for the installed packages database,
because there can only be one provider for a package. But when
parsing packages from binary repositories, we need to support
multiple providers for one package. It is now possible to get a
dictionary with either multiple providers, or just a single provider
for each package.
* Dependency parsing logic has been adjusted, to support multiple
providers. For multiple providers, the one with the same package
name as the package we are looking up is prefered. If there is none
(eg. "so:libEGL.so.1" is provided by "mesa-egl"), it prefers packages
that will be installed anyway, and after that packages that are
already installed. When all else fails, it just picks the first one
and prints a note in the "pmbootstrap log".
* Added testcases for all functions in pmb.parse.apkindex and
pmb.parse.depends
* pmbootstrap chroot has a new "--add" parameter to specify packages
that pmbootstrap should build if neccessary, and install in the
chroot. This can be used to quickly test the depencency resolution
of pmbootstrap without doing a full "pmbootstrap install".
Fixes#1122.
Add qt5-qtbase with OpenGL ES2 enabled and adjust the
upstream compatibility test case.
* Test case: don't get the qt5-qtbase version from any APKINDEX, but
only from Alpine's community APKINDEX
* Test case: If the pkgver is 9999, look at _pkgver
* Upgrade Qt libs to 5.9.3
* Add test case to make sure we are always in sync with Alpine's QT
and update qt5-qtsensors, too.
* Remove qt5-qtquickcontrols2 (it's in Alpine now)
