Commit graph

562 commits

Author SHA1 Message Date
Stefan Hansson
742300b090
pmb.commands: Add pkgver_bump (MR 2415)
Similar to pkgrel but for pkgver. Useful when dealing with e.g.
metapackages.

[ci:skip-build]: already built successfully in CI
2024-09-30 12:45:21 +02:00
Stefan Hansson
8a64c9da8f
pmb: Migrate pkgrel_bump to Command (MR 2411)
Also remove args from functions that don't actually need it to
faciliate this.

This does not attempt to fix any of the bugs with pkgrel_bump.
2024-09-26 18:27:00 +02:00
Oliver Smith
c7a7fee909
pmb.helpers.pmaports.get_repo: "systemd" or None (MR 2405)
This function is used by "pmbootstrap repo_missing", which is only used
by bpo. In order to support building the split repository in bpo, this
patch is required.

Return "systemd" for the packages that are in extra-repos/systemd, and
None for all other packages (indicating that they don't get split into a
separate repository).

Checking the parent-parent dir of the APKBUILD is a bit fragile
(assuming we don't have subdirs in the systemd repository), but that
assumption was made with the previous implementation as well (would only
return "systemd" in that case). I don't see a better solution with
reasonable effort right now, given that we also support multiple
pmaports dirs. We can improve this in the future, meanwhile let's just
not use subdirs in the systemd repository.
2024-09-22 21:15:34 +02:00
Oliver Smith
61a9bdc813
pmb.helpers.pmaports.get_repo: drop must_exist (MR 2405)
The function is never called with must_exist=False, so drop it.
2024-09-22 21:15:06 +02:00
Oliver Smith
b25fa5a854
repo_missing: set try_other_arches to False
Don't attempt to find packages of other arches in repo_missing code.
This leads to unexpected results in general (we want to check if a
binary package exists for only one specific architecture!) - and in case
of master_staging_systemd, it current causes errors because pmbootstrap
tries to fetch armhf, riscv etc. APKINDEXes for which we don't build the
staging repositories.
2024-09-22 15:33:12 +02:00
Oliver Smith
476976ba14
helpers.package.get: pass try_other_arches along
Let the get() function pass the try_other_arches parameter when calling
itself, so it does not get lost.
2024-09-22 15:33:08 +02:00
Oliver Smith
0478c30e5c
install: fix --no-local-pkgs
As chroot.glob() now returns a generator, convert it to a list first to
check if it is empty. This may not be the most efficient method, but
since we expect it to be empty here it is fine.

Without this patch, it always shows the error even if there are no
locally built packages:

  ERROR: --no-local-pkgs specified, but locally built packages found. Consider 'pmbootstrap zap -p' to delete them.
2024-09-19 02:09:43 +02:00
Oliver Smith
9cec60e057
lint: raise error if linter fails (MR 2398)
Fix that pmbootstrap exits with 0 on linter failure. Let it raise an
error instead (which results in exit != 0).
2024-09-18 17:02:19 +02:00
Oliver Smith
fcbc96b2f1
build: package: proper missing dep error (MR 2388)
When a dependency cannot be found in source or binary packages of the
selected architecture, then don't look for binary packages in other
architectures package indexes.

This leads to a confusing error down the line when building packages for
extra_repos_systemd (and e.g. stable branches), because we don't have
binary packages for all arches that pmbootstrap supports, and so it
tries to fetch a non-existing APKINDEX.

Instead, print a nice error about a non-existing dependency.
2024-09-17 02:27:45 +02:00
Caleb Connolly
7db312c981
helpers: frontend: fix newapkbuild. args.force -> context.force (MR 2388)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-09-17 02:27:44 +02:00
Oliver Smith
11fbd6d1f3
migrate_work_folder: use NonBugError (MR 2395)
Don't print a trace for the "Sorry we can't migrate that automatically"
message.
2024-09-08 13:42:34 +02:00
Oliver Smith
652d1a1afd
pmb.helpers.other: drop old migration code (MR 2395)
Get rid of 3 years and older migration code, so we don't need to
maintain it anymore. If the user should have run "pmbootstrap init" 3
years ago and suddenly upgrades to pmbootstrap v3, then they will just
need to delete their work dir (as pmbootstrap instructs to do) and start
over. This is fine, the work dir is just a bunch of caches and locally
built packages.
2024-09-08 13:42:34 +02:00
Clayton Craft
9131b910c7
pmb.helpers.lint: fix keyerror (MR 2397)
repo is a Path obj now, and its name is used as a key in the dest_paths dict.

Fixes this failure:

    working_dir=dest_paths[repo],
                ~~~~~~~~~~^^^^^^
KeyError: PosixPath('/home/clayton/src/pmaports')
2024-09-06 16:36:06 -07:00
JustSoup321
3559ee4030
pmb.init: make default hostnames pass regex (MR 2393) 2024-09-01 15:01:02 +02:00
Caleb Connolly
895187cd51
sideload: fix --user (MR 2391)
Was using config.user and ignoring --user arg

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-08-25 22:06:17 +02:00
Hugo Osvaldo Barrera
84c512ea6a
Add some missing type hints (MR 2338)
Functions with no explicit return type are considered to return `Any`.
2024-08-21 15:47:26 +02:00
Oliver Smith
ae4094c1db
Fix "pmbootstrap build_init" (MR 2386)
Do not fail with:
  ERROR: 'Namespace' object has no attribute 'image'

Adjust the similar check for args.rootfs to be in the same style while
at it.
2024-07-30 00:41:58 +02:00
Oliver Smith
86d38d451b
Fix pmbootstrap config mirrors.… <NEW_VALUE> (MR 2385)
Don't fail with:
  type object 'Config' has no attribute 'mirrors.pmaports'

Fixes: issue 2420
2024-07-28 21:01:26 +02:00
Oliver Smith
ebfda16d6d
args: remove -m and -mp (MR 2371)
Remove arguments to set postmarketOS and Alpine mirrors.

The current implementation is broken with the way the mirrors are now
configured: The current implementation fills a list, but the mirror
configuration code expects a string. It is not longer possible to set
an arbitrary amount of mirrors for the postmarketOS mirror. The benefit
is that we don't have comma separated values in the config file anymore
but instead just simple strings. Now it is possible to have one proper
mirror for aports, pmaports and pmaports_systemd, and one _custom one
to override each of these.

We already have too many options in "pmbootstrap -h", and changing a
mirror is more of a base configuration that you do once and maybe change
a few times, but don't need to be able to set it with each pmbootstrap
run. Users are not going to write this out manually on their
command-lines, it only makes sense in scripts and wrappers for
pmbootstrap, and there you could as well use another method (as now
described in docs/mirrors.md) to set the mirrors.

Less important, but another reason is that using "-mp" (two letters as
short argument) isn't really elegant.
2024-07-22 11:54:45 +02:00
Oliver Smith
b14a872fad
Fix "pmbootstrap config args /new/path" (MR 2374)
Fix writing config options that are stored as CSV-list. Without this
patch:

$ pmbootstrap config aports /tmp/pmaports
[23:43:58] Config changed: aports='/tmp/pmaports'
$ pmbootstrap config aports
[PosixPath('/'), PosixPath('t'), PosixPath('m'), PosixPath('p'), PosixPath('/'), PosixPath('p'), PosixPath('m'), PosixPath('a'), PosixPath('p'), PosixPath('o'), PosixPath('r'), PosixPath('t'), PosixPath('s')]

Storing strings as CSV list isn't great and we probably want to
implement this more elegantly, see the related issue. But let's fix this
first.

Related: pmbootstrap issue 2412
2024-07-17 23:59:18 +02:00
Oliver Smith
591a737733
Support setting custom mirrors + disabling mirrors (MR 2361)
Allow setting _custom mirrors in the config:
* alpine_custom
* pmaports_custom
* systemd_custom

When these are set, they are added to /etc/apk/repositories before real
repositories. This is used by bpo to build packages with a WIP
repository enabled, in addition to the final repository.

All mirrors can also be set to "none" to be disabled. This is important
for bootstrapping from pure Alpine without any binary repository, and
the bpo testsuite also uses this.

I've discussed with Caleb whether to name it _wip instead of _custom,
but the latter is more generic and people may also use this for other
use cases than the bpo wip repository thing.
2024-07-16 00:34:06 +02:00
Oliver Smith
7b164acb70
pmb.helpers.repo.urls: remove commented out print (MR 2361) 2024-07-16 00:34:06 +02:00
Oliver Smith
7b451b25de
Run ruff format (MR 2357) 2024-07-16 00:26:35 +02:00
Oliver Smith
36dd53f402
Run ruff check --fix (MR 2357)
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
2024-07-16 00:26:35 +02:00
HenriDellal
2b577a8694
Fix pmbootstrap install --android-recovery-zip (MR 2368)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-07-16 00:04:15 +02:00
Caleb Connolly
1af44fb159
helpers: args: don't announce config options overwritten by args (MR 2366)
Was left in for debugging

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-13 18:49:45 +02:00
Oliver Smith
9dfa89c58c
Use pmbootstrap_v3.cfg as config file (MR 2350)
Using pmbootstrap v3 with the old config can cause problems, for example
when having $WORK in the pmaports dir instead of the actual work path.
This is not supported anymore by v3 to reduce complexity. The format of
how mirrors are stored in the config also has changed.

Use a separate config file, so users can go back from v3 to 2.3.x if
they need to (for figuring out a regression) and so users won't run into
bugs when moving from 2.3.x to v3.
2024-07-10 20:06:25 +02:00
Caleb Connolly
ebef789f7c
Bump work version to zap for pmb 3.0.0 (MR 2352)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:45:27 +02:00
Caleb Connolly
8da12a98a7
helpers: other: drop args from migrate_work_folder() (MR 2352)
Was erronesously passing args to cli.confirm.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:45:23 +02:00
Caleb Connolly
9bd5860381
helpers: frontend: checksum: make sure chroot is mounted (MR 2354)
chroot's aren't automatically initialized when running a command in them
anymore. Add the explicit chroot.init() call before checksum.

Fixes #2395

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:12:30 +02:00
Caleb Connolly
f512242073
TEMP: chroot: apk: run apk through QEMU again (MR 2351)
This revers b82c4eb167 ("chroot: always run apk static (MR 2252)")
since the current iteration of the apk-tools /usr merge patches
don't handle --root properly and will behave differently based on your
host systems directory structure (specifially for /usr merge).

This commit can be reverted once we fix apk-tools.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:54:11 +02:00
Oliver Smith
852b8d6c7d
args: remove dead code (MR 2349)
Remove a bunch of commented out code:
* debug leftovers
* update_work(): a function that used to replace $WORK in all kinds of
  arguments to pmbootstrap, with the actual work dir. This seemed like a
  good idea when I made this initially, but in hindsight it's clear that
  this made args just much more complex. I'm glad that Caleb removed
  this. Now $WORK only gets removed in one place, for mount points,
  which makes much more sense.
* args.from_argparse and related comment: pmbootstrap used to make a
  full copy of its args, so in some cases it could go back before $WORK
  was replaced IIRC. This isn't great, but even worse is that this
  caused each stack trace to say something like infinite recursion on
  args. I'm also glad that Caleb got rid of this :)
2024-07-08 16:22:38 +02:00
Caleb Connolly
0e62eaac67
ruff: format (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:09:42 +02:00
Caleb Connolly
852efe1f8e
helpers: package: overload types (MR 2344)
Correct return types based on arguments

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:13 +02:00
Caleb Connolly
afbecb7184
helpers: mount: fix rootfs mount path (MR 2344)
Subtle...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:13 +02:00
Caleb Connolly
7741f74d44
helpers: toml: fix mypy on import (MR 2344)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:01:13 +02:00
Caleb Connolly
95d68d8854
helpers: args: improve please_i_really_need_args() logging (MR 2344)
dumping a stack trace directly to the console is not nice, keep it to
the log file.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:00:14 +02:00
Caleb Connolly
fce06843fc
helpers: repo: drop cache key in update (MR 2344)
Simplify the caching here and use logging.warn_once.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:00:14 +02:00
Caleb Connolly
55c912d410
helpers: logging: add warn_once() (MR 2344)
Add a warn_once() function using Cache. Since we use f-strings
everywhere we don't really need the varargs.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-08 16:00:14 +02:00
Oliver Smith
a57b338e9e
pmb.helpers.ui.check_option: fix for UI = none (MR 2348)
Don't crash on master_staging_systemd (and in the future the merged
systemd branch) if "none" is selected as UI.

Fix for:
  ERROR: Could not find aport for package: postmarketos-ui-none
2024-07-05 00:48:52 +02:00
Oliver Smith
91aee0b34c
Fix custom pmaports dir (pmbootstrap --args) (MR 2342)
The aports argument is a list of paths now, so iterate over each of them
when checking if they exist.

Fix for:
INFO:root:ERROR: 'list' object has no attribute 'exists'
2024-07-02 17:40:35 +02:00
Oliver Smith
6c4b4c4993
CI: ignore 'import tomllib' in vermin (MR 2333) 2024-06-30 18:04:31 +02:00
Oliver Smith
1dfaafe577
kconfig check: move rules to kconfigcheck.toml (MR 2333)
Move all kconfig rules from pmb.config to a separate toml file
pmb/data/kconfigcheck.toml. This is a fallback, pmbootstrap now prefers
loading kconfigcheck.toml from the currently checked out pmaports branch
if it exists.

This finally allows having separate kconfig check rules per pmaports
branch and makes the workflow of adjusting these rules much more
pleasant as the rules and kernel configs can just be adjusted at the
same time in pmaports!

This patch also moves the definition of what rules should be checked for
community and main devices, those that have pmb:kconfigcheck-community
in their linux APKBUILD, to the new kconfigcheck.toml. This should make
it much more intuitive, previously one needed to find the place in the
pmbootstrap source and edit it there.

Furthermore the "enforce_check" logic is removed. Previously pmbootstrap
would print warnings for failed config checks in some cases, but not
exit with error which was very confusing. Now exit 0 means all checks
passed and exit 1 means, that there is at least one error.

Use toml for the file, as discussed in pmbootstrap issue 2165. Python
3.11 has a native toml reader, use tomli for previous Python versions
for compatibility.
2024-06-30 18:04:31 +02:00
Oliver Smith
b6058e3569
pmb.helpers.frontend.kconfig: modernize (MR 2333)
Create new pmb/commands/kconfig_check.py and _edit.py files and move
related code there.
2024-06-30 18:04:31 +02:00
Oliver Smith
6f6cf1e8da
kconfig check: remove --kconfig-check-{name} args (MR 2333)
Prepare to move the kconfigcheck config into pmaports, so it can be
different per branch and so we can adjust it at the same time as
adjusting the kernel configs.

Once it is moved, pmbootstrap will not know the cateogry names in
advance anymore, so remove the logic for having category-specific
argument names.

In a future patch, we could add an option to manually override the
categories to be tested with one specific category (taking a string
argument), if there is demand for that.
2024-06-30 18:04:31 +02:00
gompa
8390a093d8
Fix quotation reuse (MR 2336)
Python 3.10.12 fails without this patch:

File "/media/gompa/73d88639-a730-456c-a428-6d500b4020b7/pmbootstrap/pmb/parse/arguments.py", line 844 help="Alpine Linux mirror, default: " f"{default_config.mirrors["alpine"]}",
SyntaxError: f-string: unmatched '['

Related: https://peps.python.org/pep-0701/
(Commit message written by Oliver)
2024-06-24 21:55:53 +02:00
Newbyte
8c0bc2e0e8
pmb.helpers.frontend: Print paths as a shell-friendly string (MR 2337)
Instead of

    [PosixPath('/home/newbyte/.local/var/pmbootstrap/cache_git/pmaports')]

you get

    /home/neboula/.local/var/pmbootstrap/cache_git/pmaports

like before.

Fixes envkernel.sh failing during activation.

Closes https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2378
2024-06-24 21:12:31 +02:00
Oliver Smith
18fa4e58a3
Ruff: fix typing.Xxx is deprecated, use xxx instead (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
cebd784d84
Prefer f-strings (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
f3f392ef66
Use simplified Python 3.8 syntax (MR 2327)
This commit was generated with:

    ruff check --fix --extend-select=UP .
2024-06-23 19:13:52 +02:00