Introduce helpers for interacting with pkgrepos (e.g. pmaports, and
others). These allow pmbootstrap to handle multiple source repositories.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Not sure if we should allow flash_method to be unset, but eh, simplify
usage by adding the none fallback here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
find() gets the pmaports directory of a package, get() parses the
APKBUILD inside that directory.
Instead of having the extra code path in get() for not searching
subpackages, which duplicated part of find(), add a subpackages
parameter to get().
Besides the deduplication done now, this is in preparation for code that
will extend get() to handle the extra-repos/systemd directory - without
this patch, the changes would need to be duplicated in both functions.
The repo_bootstrap command is totally standalone and has it's own state,
making it a good candidate for the pmb.commands submodule. Port it over
and move the require_bootstrap() helper function over to
pmb/helpers/pmaports.py
We also fix the call to pmb.build.package() which broke during rework.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce a Deviceinfo class and use it rather than the dictionary. This
gives us sweet sweet autocomplete, and lays the foundation for having a
proper deviceinfo validator in the future.
Additionally, continue refactoring out args...
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
(Commit message written by Oliver)
The tests relied on a lot of specifics of the old codebase. We decided
that it makes more sense to just remove all tests for now to get this
large patchset of refactorings and improvements done, and then gradually
add new tests / possibly adjust some of the old ones and bring them back
in future patches.
Future tests will not be in the "test" directory, Caleb's reasoning:
> Easier to find relevant tests for a module, and avoid the big flat
> list of test files like we had before. Honestly was just a vibes
> thing after reading the docs.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
Cease merging pmbootstrap.cfg into args, implement a Context type to let
us pull globals out of thin air (as an intermediate workaround) and rip
args out of a lot of the codebase.
This is just a first pass, after this we can split all the state that
leaked over into Context into types with narrower scopes (like a
BuildContext(), etc).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
It's been 6(?) years since pmaports moved out of the pmbootstrap repo.
Let's finally drop these checks.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The current mechanism of invoking pmbootstrap actions can be a bit
confusing, and relies heavily on args being a big namespace with lots of
properties (which may or may not be valid depending on the context).
To aid in slowly removing args from the codebase, introduce a new
mechanism for running commands. This works by having a class for each
command, where the arguments are passed in as parameters to the
constructor.
No doubt this won't scale very far, but it's a skeleton we can continue
to build on as we migrate more commands over to not require args in
order to run.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We can't totally escape the need for some runtime state defined by args.
To make the migration easier, introduce a global "Context" class and
move some of the read-only global options there.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Testing by building postmarketos-initramfs (which installs >100 packages
but is very fast to build, so a worst-case scenario) this results in a
~15-20% speedup (which everything cached and doing multiple back to back
runs). From 32 seconds down to 25.
Doing a full install with --no-image, this takes us from 70 seconds on
my laptop down to 40s!
This also lets us drastically simplify pmb/helpers/apk.py!
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We currently lazily initialize the chroot's on first use, plus a few
bonus calls to init. However, there are some instances where we actually
don't want the chroot to be initialised (mostly to break recursion
loops).
Simplify the codebase by removing all of this, and just calling
pmb.chroot.init() where it's needed.
In addition, print a warning if init() is called multiple times for one
chroot. This should help us catch these instances if they crop up again.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This was used for testing the on device installer, but doesn't seem to
be super useful anymore. Let's deprecate it and remove it at some point.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Defaulting to the native chroot isn't necessarily intuitive. Let's
require this be specified in full.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We currently do stuff in the chroot before merging /usr, this could lead
to weird side effects due to the recursive nature of chroot.init being
called every time we run a command. Let's reorder a bit to reduce the
risk of weirdness here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We don't need to re-init the chroot this often, cache on first init and
skip all subsequent ones. Even though we take a shortcut if the chroot
already exists we still do a bunch of additional checks which we only
really need to do once.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This is multithreaded, and by consequence much much faster than default
gzip. Since we switched to running gzip from the native chroot we also
changed from running it in the fastest mode, now we're running it with
-9 it can be pretty slow on any kind of mobile hardware.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
With the new chroot type, we can now write fancy paths in the pythonic
way. Convert most of the codebase over, as well as adding various other
type hints.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Avoid passing in a boolean to decide if this is the first time the
function has been called and instead just initialise the globals at the
top of the file.
We can figure out the state management if/when we want
to handle doing multiple installs in a single invocation of pmb.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Debug is the default loglevel, demote the "already visited" message to
verbose, as it isn't generally useful.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We use a custom verbose log level in pmbootstrap, unfortunately it isn't
possible to correctly type this due to some limitations in the logging
library [1], [2].
Given that our usecase is fairly simple, we can just wrap the module
with our own so we only have to tell mypy to ignore the error once
instead of at every callsite.
[1]: https://github.com/cryptax/droidlysis/issues/15
[2]: https://github.com/python/typing/discussions/980
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
According to the type hints for os.cpu_count(), it might return None...
To be safe, add a warning and a fallback in this case (though it seems
incredibly unlikely this would ever be hit)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
