Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.
The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a Deviceinfo class and use it rather than the dictionary. This
gives us sweet sweet autocomplete, and lays the foundation for having a
proper deviceinfo validator in the future.
Additionally, continue refactoring out args...
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.
This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
With the new chroot type, we can now write fancy paths in the pythonic
way. Convert most of the codebase over, as well as adding various other
type hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We use a custom verbose log level in pmbootstrap, unfortunately it isn't
possible to correctly type this due to some limitations in the logging
library [1], [2].
Given that our usecase is fairly simple, we can just wrap the module
with our own so we only have to tell mypy to ignore the error once
instead of at every callsite.
[1]: https://github.com/cryptax/droidlysis/issues/15
[2]: https://github.com/python/typing/discussions/980
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a new module: pmb.core to contain explicitly typed pmbootstrap
API. The first component being Suffix and SuffixType. This explicitly
defines what suffixes are possible, future changes should aim to further
constrain this API (e.g. by validating against available device
codenames or architectures for buildroot suffixes).
Additionally, migrate the entire codebase over to using pathlib.Path.
This is a relatively new part of the Python standard library that uses a
more object oriented model for path handling. It also uses strong type
hinting and has other features that make it much cleaner and easier to
work with than pure f-strings. The Chroot class overloads the "/"
operator the same way the Path object does, allowing one to write paths
relative to a given chroot as:
builddir = chroot / "home/pmos/build"
The Chroot class also has a string representation ("native", or
"rootfs_valve-jupiter"), and a .path property for directly accessing the
absolute path (as a Path object).
The general idea here is to encapsulate common patterns into type hinted
code, and gradually reduce the amount of assumptions made around the
codebase so that future changes are easier to implement.
As the chroot suffixes are now part of the Chroot class, we also
implement validation for them, this encodes the rules on suffix naming
and will cause a runtime exception if a suffix doesn't follow the rules.
Often I find myself confused and subsequently frustrated by the
behaviour of --fork-alpine as usually I apply whatever changes I want to
the aports package before forking. This is due to that in the proper
aports repo I can leverage the full power of git and apply patches or
revert commits, whereas after the package has been forked to pmaports
the context git needs to do this is lost.
To fix that, this commit introduces --fork-alpine-retain-branch which
works the same way as --fork-alpine except it doesn't change the aports
branch to match the current channel.
We have been discussing making device package versioning more
consistent and ended up settling on using integer versioning for device
packages[1]. As such, let's start by tackling this at the source and
start generating new device packages with 1 as the starting pkgver
instead of 0.1.
[1]: https://gitlab.com/postmarketOS/pmaports/-/issues/2610#note_1796178414
Sometimes I want to build an older version of a package from Alpine, and
since package upgrades can involve things like patches and other
externalities just changing the pkgver and running checksum in pmaports
may not be enough. As such, it tends to be easier to revert the change
in the local aports repo and then fork than forking and then trying to
manually revert the changes yourself (since you can't have git do that
for you given that they are distinct repositories).
Prior to this patch, that was not possible since pmbootstrap would
assume older aport version equals outdated aports in general and as such
cancel the whole operation. Instead, just print a warning and helpful
information to make this workflow possible while also warning users that
they may want to update their local aports.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231209111813.37756-1-newbyte@postmarketos.org%3E
Instead of building cross packages for the native architecture, add a
new get_cross_packages_arches() function that returns the desired
architectures no matter what the current native architecture is.
So gcc-x86_64 will always have arch="aarch64", and gcc-aarch64 will
always have arch="x86_64".
We can tweak the function in the future if e.g. it becomes viable to
cross compile from riscv64. Or even better, remove this and the cross
compile packages upstreamed into alpine (see pmb#551).
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231126104823.3723-1-ollieparanoid@postmarketos.org%3E
Currently, pmbootstrap checks if either the kernel or the ramdisk in a boot.img
contains the MediaTek header, and if one does, it assumes both do. It hardcodes
the label KERNEL for the kernel and ROOTFS for the ramdisk.
My Amazon Echo Dot (gen 2) has a boot.img where only the kernel has the header,
but not the ramdisk. These changes (as well as those in my new boot-deploy MR)
account for that situation (and any possible label an image has) by splitting
bootimg_mtk_mkimage into two variables for the kernel and the ramdisk labels.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/patches/46351
Change the defaults in the new device port wizard:
* arch: armv7 -> aarch64
New devices are more likely to be aarch64.
* flash method: 0xffff -> none
The only device that uses 0xffff is the N900, it's extremely unlikely
that it will get used for a new port. Use the neutral "none" instead.
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230802172542.4861-2-ollieparanoid@postmarketos.org%3E
A long time ago we renamed the flash_system action into flash_rootfs.
Since we still kept some variables around, it's finally time to clean
that up.
Keep backwards compatibility for now since we cannot update pmaports at
the same time since the new deviceinfo names won't be supported in older
pmbootstrap versions.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230605220329.14328-3-luca@z3ntu.xyz%3E
The typical workflow for upgrading cross/gcc-* is:
$ cd pmaports/cross
$ pmbootstrap aportgen gcc-*
Currently this is failing because the APKBUILD has been updated in
Alpine for gcc, but gcc has not been built for all arches yet. This
shouldn't prevent us from generating the proper updated APKBUILDs in
pmaports so just print a note and don't fail here.
I'm committing this directly to master as this currently breaks
test/test_aportgen.py::test_aportgen.
Device packages shouldn't depend on it anymore, in order to end up with
a smaller installation for ui=none this package only gets pulled in by
postmarketos-base if mesa is installed.
Related: pmaports MR 3478
abuild traces shared library dependencies for binaries in built
packages, and adds all of them to depends. This is needed for the cross
gcc builds, as otherwise it will not add the isl dependency (currently
isl25).
With the crossdirect compilation method we mount the cross-gcc from the
native chroot in the foreign arch chroot. Due to the missing dependency,
isl was not installed in the native chroot, and so when attempting to
use that native cross gcc from the foreign arch chroot, it would try to
load the foreign arch isl instead of the native one. Resulting in lots
of the following errors (and of course not a working gcc):
Error relocating /usr/lib/libisl.so.23: unsupported relocation type 1026
I had probably added '!tracedeps' because abuild complains about it
otherwise... but as noted in [1] these complaints are safe to ignore.
Keep !strip for now even though it's not in Alpine's APKBUILD. I've
tried to build without and the build fails with:
strip: Unable to recognise the format of the input file `./usr/aarch64-alpine-linux-musl/lib/libgcc_s.so.1'
[1]: 0415ebf6f9
Related: pmaports issue 1732
Add the subpackage, because g++-$arch depends on it since
12.1.1_git20220630-r6. This fix was already applied to pmaports, thanks
to ungeskriptet! Adjust the aportgen code to make sure this is added in
the future as well.
Related: pmaports MR 3464
Related: f1044b1dae
Do not build the foreign arch libgcc. We are using the one provided by
Alpine's binary repository instead, and when building it here and trying
to use it, we actually run into a conflict.
I tried setting BOOTSTRAP="nolibc" first since there is already logic to
then disable libgcc in the APKBUILD. But that also disables g++ which we
do want to build, so we can't use this.
Fixes: issue 2168
Remove it from depends, as the package has been removed in Alpine and
since quite some time we shouldn't have depended on it anyway.
Related: issue 2167
Set CTARGET and CTARGET_ARCH on top of the generated APKBUILD and use
the existing code in the APKBUILD for build(), package(), makedepends,
subpackages. This is now possible, because the pmbootstrap parser for
APKBUILDs has been improved a lot since this was originally written and
because the Alpine APKBUILD does everything that was hacked in here,
like removing man, info directories and bfd-plugins if doing a cross
build.
This is now very similar to bootstrapping binutils for another
architecture via scripts/bootstrap.sh in aports.git.
Most importantly this sets --disable-gprofng, which fails to build
against musl and is the reason why our cross/binutils-* packages
generated from binutils 2.39.x didn't build anymore.
Related: https://git.alpinelinux.org/aports/tree/main/binutils/APKBUILD
Related: https://sourceware.org/bugzilla/show_bug.cgi?id=29477
