This option will make pmbootstrap automatically zap chroots that are
initialized for the wrong channel, making it much faster to switch
between edge and systemd.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Correctly denote whether the chroot is for the systemd channel or not so
users are warned when switching from edge to systemd staging without
zapping.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Add some exhaustive unit testing to validate that these types behave as
expected.
And fix a few bugs uncovered by the tests.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Rewrite the parser to be more efficient:
* Avoid using str.startswith()
* Simplify logic by parsing backwards and using lines.pop() rather than
passing around an index.
Testing with "pmbootstrap test apkindex_parse_all" on my x86 laptop:
Before: Parsed 78537 packages from 7 APKINDEX files in 1.131 seconds
After : Parsed 78537 packages from 7 APKINDEX files in 0.609 seconds
That makes for an ~86% improvement, almost twice as fast.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This is to serve as a place to dump useful internal tests, starting with
one that simply parses all available APKINDEX files.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Currently it can be quite annoying to parse a pmbootstrap log containing
multiple commands, since there are no newlines to differentiate between
invocations of pmbootstrap, the arguments it's called with aren't
logged, and neither is information about the system.
Fix all of this by printing some newlines to the log file and logging
the pmbootstrap version, the Python version, and the arguments
pmbootstrap was called with.
Extra care is taken to redact the "--password" option - users should
NOT treat this password as secure (since it's in their shell history)
but that doesn't mean we should go out of our way to leak password.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
re-introduce pytest, add a conftest.py with some useful fixtures and
basic tests for config loading.
This just checks that we can load the config and migrate it from the old
2.3.x format to the new 3.0 format with the new mirrors section.
Testing anything that requires args or Context should probably wait
until we can properly model state (since global state like in
get_context() really doesn't jive with pytest).
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
To support multiple aports we changed how we configure
/home/pmos/packages in the buildroot so it now points to whichever local
binary repo makes sense for the package being built.
Copy over the code to envkernel run_abuild() so the packages actually
get installed to $WORK/packages
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Get rid of config.mirror_alpine and mirrors_postmarketos and make sure
they get migrated over for existing users.
mirrors_postmarketos being a list was always a bit off, but now we have
per-aports mirrors which make a lot more sense for what we're trying to
do with systemd.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We now have support for having mirrors per-aports repo, drop the
mirrors_postmarketos arg from chroot.init and instead have
repo_bootstrap call apk.update_repository_list() explicitly to exclude
the mirrors for the repository we're going to update.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
pmaports.get() can return None usually, but check_arch() has no handling
for it. Set must_exist=True so we properly error out in this case.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Let's make pmbootstrap more colourful! Add some templates to enable
arbitrary colors in log messages.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Rename build.package() to build.packages() and take a list of packages
to build, since every caller was inside a for loop this simplifies
usage and let's us give nicer log output by doing all the builds first,
so log messages don't get lost in the middle.
Behaviour is cleaned up so this shouuuuld work pretty well now. It
properly descends into dependencies and will build dependencies even if
the package given doesn't need building. Technically this was only done
before during install where the dependencies were recursed in
chroot.apk.install().
It probably makes the most sense to have a mode where it doesn't build
dependencies but warns the user about it, at least when invoked via
pmbootstrap build.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The call to chroot.apk.install() will automatically build packages, so
we don't need to call it again beforehand.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
We used to recursively descend into the dependencies of the packages we
mark for install, this was needed so that we build all the packages we
want to.
However, the new package builder now does this for us. So simplify and
speed this up a bit by not doing it in apk.install().
Additionally, soft-remove support for passing in packages to delete by
having the package name start with a "!". This was useful at some point,
but we now handle this much better in pmaports.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Generalise pmb.helpers.other.cache with a more python decorator.
The Cache decorator takes a list of function arguments to use as cache
keys, keyword args can be used to restrict caching so that it is skipped
entirely unless the attribute has a specific value.
For example, pmb.helpers.pmaports.get() has the decorator:
@Cache("pkgname", subpackages=True)
This means the return value will be cached only when subpackages is
True, otherwise it will always miss.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Seems to be having trouble unpacking archives...
pigz: skipping: /var/cache/distfiles/postmarketos-mkinitfs-2.5.0.tar.gz ends with .gz
tar: This does not look like a tar archive
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
The package builder has long been a pain point since it recurses the
entire dependency tree.
Convert it to run in two stages, first it walks through the package
dependencies, descending into each one and processing them in a queue.
If a package is determined to need building then it gets pushed onto the
build_queue.
Then, it pops each package off the build queue (which is actually a
stack..) and builds it.
This avoids recursion entirely and should open the door to optimisations
in the future.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
