Commit graph

2679 commits

Author SHA1 Message Date
Caleb Connolly
53a2953a4a
chroot: mount: drop args from remove_mnt_pmbootstrap (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
759fdd655b
chroot: mount: skip mounted targets (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
b447c20545
build: fix arch formatting in check_build_for_arch (MR 2252)
Use f-strings so it's actually converted.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
656ff3f5bb
commands: move shutdown command to pmb/commands (MR 2252)
Another one bites the dust

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
a99ae02068
build: slightly less verbose logging (MR 2252)
Don't log about cached packages or on every call to package().

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
945d6dd063
types: PmbArgs: remove devicesdhbfvhubsud (MR 2252)
Holdover from cleaning up args.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
084309e756
parse: arguments: typing improvements (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
efeb09bf5d
config: pmaports: log all_channels (MR 2252)
helps debugging

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
133091248f
repo: use new config.mirrors (MR 2252)
Use the new config.mirrors section to handle repository URLs instead of
the old mirror_alpine / mirrors_postmarketos options. This let's us
add the systemd staging repo automatically when on the systemd staging
branch / systemd is enabled.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
1a01738d50
config: sanity check via types (MR 2252)
Replace the "sanity_check" code with type checking built into the Config
__setattr__ operator.

This keeps all the Config related code in one place.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
7a8deb0f5e
config: clean up parsing and add mirrors (MR 2252)
Add a new config section "mirrors", to replace the mirrors_alpine and
mirrors_postmarketos options. This will allow for more flexibility since
we can then handle the systemd staging repo (and others like plasma
nightly) with relative ease.

The loading/saving is fixed and now properly avoids writing out default
values, this way if the defaults are changed the user won't be stuck
with old values in their pmbootstrap.cfg.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
8e18b16370
aportgen: more type hinting (MR 2252)
Thanks WhyNotHugo for the recommendations.
2024-06-23 12:38:39 +02:00
Caleb Connolly
866e5bcfab
core: add an Arch type (MR 2252)
Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.

The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
505165dc13
parse: apkindex: fix cache keys (MR 2252)
Using Paths as cache keys is not really ideal, plus it's the absolute
path on the host, weird.

For now, add a cache_key() function to get just the part of the path we
care about and use it as the key. This probably ought to be revisisted
later.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
450bc6873d
core: move Config type (MR 2252)
Keeping the Config class in types seemed kinda weird and was just done
as a workaround to some cyclical imports. But now things are more in
shape let's move it to core.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
b43724fee4
build: multiple binary repos (MR 2252)
Use a different binary repo depending on the source repository for the
package. This makes it possible to split out systemd packages into their
own repository.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
560cea46ea
treewide: migrate to pkgrepo API (MR 2252)
Make use of the new pmb.core.pkgrepo API to handle multiple aports and
extra-repos.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
8a61d67053
chroot: run: support running multiple commands with one call (MR 2252)
Building the command strings and entering the chroot is a
not-insubstantial amount of overhead. Implement support for running
multiple commands with a new pmb.chroot.rootm() function.

TODO: add alternative for chroot.user and run.root/user.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
3a74018f89
chroot: fix shutdown (MR 2252)
Something was going wrong here, confused

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:39 +02:00
Caleb Connolly
3473422639
chroot: fix args.user (MR 2252)
config.user and args.user are distinct, don't combine them.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
bc0b6e39e8
config: fix load/save/ask (MR 2252)
Fix all the config code so pmbootstrap init works again...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
3a796d10f8
core: add pkgrepo helpers (MR 2252)
Introduce helpers for interacting with pkgrepos (e.g. pmaports, and
others). These allow pmbootstrap to handle multiple source repositories.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
0a73bc298b
parse: deviceinfo: default flash_method (MR 2252)
Not sure if we should allow flash_method to be unset, but eh, simplify
usage by adding the none fallback here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
d8d55c3e52
aportgen: fix fork_alpine flag (MR 2252)
Broke during rework

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Oliver Smith
34ee1cff69
pmb.helpers.pmaports.get/find: deduplicate code (MR 2299) (MR 2252)
find() gets the pmaports directory of a package, get() parses the
APKBUILD inside that directory.

Instead of having the extra code path in get() for not searching
subpackages, which duplicated part of find(), add a subpackages
parameter to get().

Besides the deduplication done now, this is in preparation for code that
will extend get() to handle the extra-repos/systemd directory - without
this patch, the changes would need to be duplicated in both functions.
2024-06-23 12:38:38 +02:00
Caleb Connolly
f422b5c7ca
commands: port repo_bootstrap (MR 2252)
The repo_bootstrap command is totally standalone and has it's own state,
making it a good candidate for the pmb.commands submodule. Port it over
and move the require_bootstrap() helper function over to
pmb/helpers/pmaports.py

We also fix the call to pmb.build.package() which broke during rework.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
97bd8b96ec
parse: deviceinfo: make Deviceinfo a class (MR 2252)
Introduce a Deviceinfo class and use it rather than the dictionary. This
gives us sweet sweet autocomplete, and lays the foundation for having a
proper deviceinfo validator in the future.

Additionally, continue refactoring out args...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
b51d31acab
flasher: variables: rename vars to fvars (MR 2252)
vars is a reserved keyword in python, use fvars instead.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
852ace63c4
commands: move index command to pmb/commands (MR 2252)
Use the new command runner for pmbootstrap index.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
d4070c0e9e
chroot: zap: remove unneeded call to pmb.chroot.shutdown() (MR 2252)
We don't need to call this twice, and we boot the chroot up again to zap
packages anyway.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
d9e1da98b2
more n more (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
de4c912692
WIP: 2024-06-05: args hacking and more (MR 2252)
Continue removing args and do some other optimisations.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
5bb2390d98
parse: deviceinfo: cache deviceinfo (MR 2252)
add a cache key for deviceinfo and cache it after parsing.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
968cf81df4
helpers: args: validate arch in sanity_check() (MR 2252)
We did the arch check when building args for some reason. Move it to the
sanity checks.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
1f7e352278
test: drop all tests (MR 2252)
(Commit message written by Oliver)

The tests relied on a lot of specifics of the old codebase. We decided
that it makes more sense to just remove all tests for now to get this
large patchset of refactorings and improvements done, and then gradually
add new tests / possibly adjust some of the old ones and bring them back
in future patches.

Future tests will not be in the "test" directory, Caleb's reasoning:
> Easier to find relevant tests for a module, and avoid the big flat
> list of test files like we had before. Honestly was just a vibes
> thing after reading the docs.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
48cd886401
more wip (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
34dd9d42ba
WIP: start ripping out args (MR 2252)
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.

This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
bfea00e03a
config: pmaports: drop legacy checks (MR 2252)
It's been 6(?) years since pmaports moved out of the pmbootstrap repo.
Let's finally drop these checks.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
e547bb7f9c
helpers: mount: drop args (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
1371525c2b
helpers: ui: replace args.aports with context.aports (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
66c1d20f8e
helpers: pmaports: drop args (MR 2252)
continue replacing args.aports with context.aports

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
0a0cba51e1
helpers: ui: rename list() to list_ui() (MR 2252)
list is a reserved keyword...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
1be8653935
drop args from helpers.git and chroot.apk (MR 2252)
be gone!

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
52338c5e76
args: drop config_channels (MR 2252)
This doesn't seem to be ever used, drop it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
fa804c9453
chroot: replace arch.from_chroot_suffix() with chroot.arch (MR 2252)
Another usage of args dropped!

Although the device_arch variable thing is not very ideal...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
8526631589
commands: new submodule (MR 2252)
The current mechanism of invoking pmbootstrap actions can be a bit
confusing, and relies heavily on args being a big namespace with lots of
properties (which may or may not be valid depending on the context).

To aid in slowly removing args from the codebase, introduce a new
mechanism for running commands. This works by having a class for each
command, where the arguments are passed in as parameters to the
constructor.

No doubt this won't scale very far, but it's a skeleton we can continue
to build on as we migrate more commands over to not require args in
order to run.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
52cf0e0fe3
test: drop args from run (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:38 +02:00
Caleb Connolly
05c86be11c
helpers: drop args from helpers.run functions (MR 2252)
Now we can run commands without needs args available!

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:37 +02:00
Caleb Connolly
02f04ba3a8
core: move args.log to context.log (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:37 +02:00
Caleb Connolly
838beb270d
config: combine save and load (MR 2252)
These are both pretty related, put them in the same file.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:37 +02:00