Using pmbootstrap v3 with the old config can cause problems, for example
when having $WORK in the pmaports dir instead of the actual work path.
This is not supported anymore by v3 to reduce complexity. The format of
how mirrors are stored in the config also has changed.
Use a separate config file, so users can go back from v3 to 2.3.x if
they need to (for figuring out a regression) and so users won't run into
bugs when moving from 2.3.x to v3.
Add the missing runtime conversion when --config is given on the
cmdline.
This expects a pathlib.Path object as the argument, but args.config is
only a string.
Remove a bunch of commented out code:
* debug leftovers
* update_work(): a function that used to replace $WORK in all kinds of
arguments to pmbootstrap, with the actual work dir. This seemed like a
good idea when I made this initially, but in hindsight it's clear that
this made args just much more complex. I'm glad that Caleb removed
this. Now $WORK only gets removed in one place, for mount points,
which makes much more sense.
* args.from_argparse and related comment: pmbootstrap used to make a
full copy of its args, so in some cases it could go back before $WORK
was replaced IIRC. This isn't great, but even worse is that this
caused each stack trace to say something like infinite recursion on
args. I'm also glad that Caleb got rid of this :)
Prepare to move the kconfigcheck config into pmaports, so it can be
different per branch and so we can adjust it at the same time as
adjusting the kernel configs.
Once it is moved, pmbootstrap will not know the cateogry names in
advance anymore, so remove the logic for having category-specific
argument names.
In a future patch, we could add an option to manually override the
categories to be tested with one specific category (taking a string
argument), if there is demand for that.
Python 3.10.12 fails without this patch:
File "/media/gompa/73d88639-a730-456c-a428-6d500b4020b7/pmbootstrap/pmb/parse/arguments.py", line 844 help="Alpine Linux mirror, default: " f"{default_config.mirrors["alpine"]}",
SyntaxError: f-string: unmatched '['
Related: https://peps.python.org/pep-0701/
(Commit message written by Oliver)
Add a new flag --image which can be used to mount the rootfs generated
with "pmbootstrap install".
For now this is quite limited in scope. But it's enough to allow for
building a package, updating it in the QEMU image, and then booting it.
The major "gotcha" with this is that the QEMU uses the kernel and
initramfs from the device chroot unless you run it with --efi.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This is to serve as a place to dump useful internal tests, starting with
one that simply parses all available APKINDEX files.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Get rid of config.mirror_alpine and mirrors_postmarketos and make sure
they get migrated over for existing users.
mirrors_postmarketos being a list was always a bit off, but now we have
per-aports mirrors which make a lot more sense for what we're trying to
do with systemd.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add a new config section "mirrors", to replace the mirrors_alpine and
mirrors_postmarketos options. This will allow for more flexibility since
we can then handle the systemd staging repo (and others like plasma
nightly) with relative ease.
The loading/saving is fixed and now properly avoids writing out default
values, this way if the defaults are changed the user won't be stuck
with old values in their pmbootstrap.cfg.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.
The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Keeping the Config class in types seemed kinda weird and was just done
as a workaround to some cyclical imports. But now things are more in
shape let's move it to core.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Use a different binary repo depending on the source repository for the
package. This makes it possible to split out systemd packages into their
own repository.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.
This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
With the new chroot type, we can now write fancy paths in the pythonic
way. Convert most of the codebase over, as well as adding various other
type hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The init_cache() function just assigned some default constants, simplify
this by just declaring it that way to begin with, as well as adding type
hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This is needed for sphinx autoprogram since that expects an
argparse.ArgumentParser, and arguments() returns some argparse
"Namespace" obj. Useful for sphinx/autoprogram and maybe other things
later that want to get at pmb's full args.
Reimplement "pmbootstrap status" to be just a simple and useful status
overview. The previous version ran a bunch of checks every time, and
would fail on these even if pmaports was used for normal development:
* "non-official" branch checked out in pmaports
* pmaports.git is not clean
The information about aports.git was also considered not so useful upon
revisiting this command, since it is only used for "pmbootstrap
aportgen". Most users don't need this, and if the user runs this
command, it will tell if aports.git is outdated.
All of the above made the previous version unpleasant to use and I
suspect most people stopped using the command after trying it out a few
times and seeing the irrelevant but loud NOK complaints.
New version:
$ pmbootstrap status
Channel: edge (pmaports: master_staging_systemd)
Device: qemu-amd64 (x86_64, kernel: virt)
UI: console
systemd: no (default for selected UI)
Old version (without --details it only shows NOK checks):
$ pmbootstrap status --details
[00:55:20] *** CONFIG ***
[00:55:20] Device: qemu-amd64 (x86_64, "QEMU amd64")
[00:55:20] Kernel: virt
[00:55:20] User Interface: console
[00:55:20]
[00:55:20] *** GIT REPOS ***
[00:55:20] Path: /home/user/.local/var/pmbootstrap/cache_git
[00:55:20] - aports_upstream (master)
[00:55:20] - pmaports (master)
[00:55:20]
[00:55:20] *** CHECKS ***
[00:55:20] [OK ] Chroots zapped recently (or non-existing)
[00:55:20] [OK ] aports_upstream: on official channel branch
[00:55:20] [OK ] aports_upstream: workdir is clean
[00:55:20] [OK ] aports_upstream: tracking proper remote branch 'origin/master'
[00:55:20] [OK ] aports_upstream: up to date with remote branch
[00:55:20] [OK ] aports_upstream: remote information updated recently (via git fetch/pull)
[00:55:20] [OK ] pmaports: on official channel branch
[00:55:20] [OK ] pmaports: workdir is clean
[00:55:20] [OK ] pmaports: tracking proper remote branch 'origin/master'
[00:55:20] [OK ] pmaports: up to date with remote branch
[00:55:20] [OK ] pmaports: remote information updated recently (via git fetch/pull)
[00:55:20]
[00:55:20] NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
[00:55:20] DONE!
Instead of assuming the architecture of the foreign device to be what
the user selected in pmbootstrap init, actually query the architecture
from the device and use that.
This does mean that one extra ssh connection is necessary, which does
slow down the procedure somewhat. However, I think that is worth the
user experience improvement this brings.
Also, the deduction process can be skipped by manually specifying
--arch in the sideload invocation should it fail, or if one really wants
to skip that extra ssh roundtrip.
Related: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2317
Often I find myself confused and subsequently frustrated by the
behaviour of --fork-alpine as usually I apply whatever changes I want to
the aports package before forking. This is due to that in the proper
aports repo I can leverage the full power of git and apply patches or
revert commits, whereas after the package has been forked to pmaports
the context git needs to do this is lost.
To fix that, this commit introduces --fork-alpine-retain-branch which
works the same way as --fork-alpine except it doesn't change the aports
branch to match the current channel.
Repurposing argcomplete as a boolean telling whether the module
argcomplete was found results in a variable with two distinct purposes
and types, which is bad practice and confuses Mypy. As such, don't
assign False to argcomplete if the module is not found and instead just
check for whether it exists in sys.modules to determine whether it was
imported successfully.
I zap chroots a lot, since I've found that it often "fixes" a lot of
weird issues that come about if you have stale chroots laying around.
So a common pattern I do is "pmb zap && pmb install ...". Having an
option to pmb install let's me simplify this.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231214234051.4673-1-clayton@craftyguy.net%3E
It can be quite useful to check the configs for a list of packages, so
adjust the argument parsing code to support that.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Tested-by: Oliver Smith <ollieparanoid@postmarketos.org>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20231028085901.35205-1-luca@z3ntu.xyz%3E
With this code path, pmbootstrap would start a distccd + sshd in the
native chroot, and configure it so it runs the cross compiler. The
foreign arch chroots would then call this cross compiler from localhost
by calling the distcc client instead of gcc.
This code has been obsoleted by the much simpler crossdirect in 2019.
Let's finally remove it.
Fixes: issue 2179
Reviewed-by: Luca Weiss <luca@z3ntu.xyz>
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230613161437.570196-4-ollieparanoid@postmarketos.org%3E
A long time ago we renamed the flash_system action into flash_rootfs.
Since we still kept some variables around, it's finally time to clean
that up.
Keep backwards compatibility for now since we cannot update pmaports at
the same time since the new deviceinfo names won't be supported in older
pmbootstrap versions.
Reviewed-by: Oliver Smith <ollieparanoid@postmarketos.org>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230605220329.14328-3-luca@z3ntu.xyz%3E
Always set GOCACHE during build and let it point to pmbootstrap's work
dir. This has a similar effect as using ccache for C/C++.
Set GOMODCACHE conditionally: this is for caching the go modules (git
repositories) that get cloned during a build if they are not bundled
with the source. Usually APKBUILDs should cache them, but when using
pmbootstrap build --src, they would get downloaded over and over again.
Set GOMODCACHE automatically for --src and allow enabling/disabling this
manually with new --go-mod-cache / --no-go-mod-cache.
This speeds up multiple iterations of building the same package
significantly. I'm using it for:
$ pmbootstrap build postmarketos-mkinitfs --src=...
I've verified that using the same GOCACHE dir for multiple architectures
doesn't result in build artifacts for the wrong arch getting used.
Reviewed-by: Clayton Craft <clayton@craftyguy.net>
Link: https://lists.sr.ht/~postmarketos/pmbootstrap-devel/%3C20230223064743.1773-1-ollieparanoid@postmarketos.org%3E
