re-introduce pytest, add a conftest.py with some useful fixtures and
basic tests for config loading.
This just checks that we can load the config and migrate it from the old
2.3.x format to the new 3.0 format with the new mirrors section.
Testing anything that requires args or Context should probably wait
until we can properly model state (since global state like in
get_context() really doesn't jive with pytest).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
To support multiple aports we changed how we configure
/home/pmos/packages in the buildroot so it now points to whichever local
binary repo makes sense for the package being built.
Copy over the code to envkernel run_abuild() so the packages actually
get installed to $WORK/packages
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Get rid of config.mirror_alpine and mirrors_postmarketos and make sure
they get migrated over for existing users.
mirrors_postmarketos being a list was always a bit off, but now we have
per-aports mirrors which make a lot more sense for what we're trying to
do with systemd.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We now have support for having mirrors per-aports repo, drop the
mirrors_postmarketos arg from chroot.init and instead have
repo_bootstrap call apk.update_repository_list() explicitly to exclude
the mirrors for the repository we're going to update.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
pmaports.get() can return None usually, but check_arch() has no handling
for it. Set must_exist=True so we properly error out in this case.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Let's make pmbootstrap more colourful! Add some templates to enable
arbitrary colors in log messages.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Rename build.package() to build.packages() and take a list of packages
to build, since every caller was inside a for loop this simplifies
usage and let's us give nicer log output by doing all the builds first,
so log messages don't get lost in the middle.
Behaviour is cleaned up so this shouuuuld work pretty well now. It
properly descends into dependencies and will build dependencies even if
the package given doesn't need building. Technically this was only done
before during install where the dependencies were recursed in
chroot.apk.install().
It probably makes the most sense to have a mode where it doesn't build
dependencies but warns the user about it, at least when invoked via
pmbootstrap build.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The call to chroot.apk.install() will automatically build packages, so
we don't need to call it again beforehand.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We used to recursively descend into the dependencies of the packages we
mark for install, this was needed so that we build all the packages we
want to.
However, the new package builder now does this for us. So simplify and
speed this up a bit by not doing it in apk.install().
Additionally, soft-remove support for passing in packages to delete by
having the package name start with a "!". This was useful at some point,
but we now handle this much better in pmaports.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Generalise pmb.helpers.other.cache with a more python decorator.
The Cache decorator takes a list of function arguments to use as cache
keys, keyword args can be used to restrict caching so that it is skipped
entirely unless the attribute has a specific value.
For example, pmb.helpers.pmaports.get() has the decorator:
@Cache("pkgname", subpackages=True)
This means the return value will be cached only when subpackages is
True, otherwise it will always miss.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Seems to be having trouble unpacking archives...
pigz: skipping: /var/cache/distfiles/postmarketos-mkinitfs-2.5.0.tar.gz ends with .gz
tar: This does not look like a tar archive
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The package builder has long been a pain point since it recurses the
entire dependency tree.
Convert it to run in two stages, first it walks through the package
dependencies, descending into each one and processing them in a queue.
If a package is determined to need building then it gets pushed onto the
build_queue.
Then, it pops each package off the build queue (which is actually a
stack..) and builds it.
This avoids recursion entirely and should open the door to optimisations
in the future.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We used to called pmb.chroot.shutdown() with "only_install_related=True"
here to unmount everything in the chroot so we could reliably "df" it to
figure out the rootfs image size.
Since ("pmb.chroot: only init once"), the chroot is no longer
automatically spun back up when we run mkinitfs, and the in-pmbootstrap
marker isn't recreated.
We only actually need to unmount the chroot anyway, so let's restrict it
to just that and remount things again afterwards.
This avoids weird side effects (like the native chroot being
shutdown?!), but this is something we should look into more in the
future since we're still doing a finnicky balancing act to manage chroot
state here.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Use the new config.mirrors section to handle repository URLs instead of
the old mirror_alpine / mirrors_postmarketos options. This let's us
add the systemd staging repo automatically when on the systemd staging
branch / systemd is enabled.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Replace the "sanity_check" code with type checking built into the Config
__setattr__ operator.
This keeps all the Config related code in one place.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add a new config section "mirrors", to replace the mirrors_alpine and
mirrors_postmarketos options. This will allow for more flexibility since
we can then handle the systemd staging repo (and others like plasma
nightly) with relative ease.
The loading/saving is fixed and now properly avoids writing out default
values, this way if the defaults are changed the user won't be stuck
with old values in their pmbootstrap.cfg.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Move pmb/parse/arch.py over to core and refactor it as an Arch type,
similar to how Chroot was done. Fix all the uses (that I can find) of
arch in the codebase that need adjusting.
The new Arch type is an Enum, making it clear what architectures can be
represented and making it much easier to reason about. Since we support
~5 (kinda) different representations of an Architecture (Alpine, Kernel,
target triple, platform, and QEMU), we now formalise that the Alpine
format is what we represent internally, with methods to convert to any
of the others as-needed.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Using Paths as cache keys is not really ideal, plus it's the absolute
path on the host, weird.
For now, add a cache_key() function to get just the part of the path we
care about and use it as the key. This probably ought to be revisisted
later.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Keeping the Config class in types seemed kinda weird and was just done
as a workaround to some cyclical imports. But now things are more in
shape let's move it to core.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Use a different binary repo depending on the source repository for the
package. This makes it possible to split out systemd packages into their
own repository.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Building the command strings and entering the chroot is a
not-insubstantial amount of overhead. Implement support for running
multiple commands with a new pmb.chroot.rootm() function.
TODO: add alternative for chroot.user and run.root/user.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
