Commit graph

2994 commits

Author SHA1 Message Date
Newbyte
816ae5ac6c
pmb.sideload: Run $ uname -m in subshell (MR 2390)
Some shells, in particular Nushell, replace the built-in
POSIX-compatible uname with a custom one that does not support common
flags. I opened an issue in the Nushell repository about this, but it
was rejected on the grounds that POSIX compatibility is not a goal. As
such, run this in a subshell to ensure that we get the expected uname
interface.

Nushell issue: https://github.com/nushell/nushell/issues/12570
2024-08-11 17:22:38 +02:00
Newbyte
8b9648b057
pmb.sideload: Annotate "None" return types (MR 2390)
Otherwise this defaults to Any.
2024-08-11 17:22:24 +02:00
Anri Dellal
170829326c
kconfig: edit,migrate: use single package as argument (MR 2387) 2024-07-30 21:08:36 +03:00
Oliver Smith
ae4094c1db
Fix "pmbootstrap build_init" (MR 2386)
Do not fail with:
  ERROR: 'Namespace' object has no attribute 'image'

Adjust the similar check for args.rootfs to be in the same style while
at it.
2024-07-30 00:41:58 +02:00
Oliver Smith
86d38d451b
Fix pmbootstrap config mirrors.… <NEW_VALUE> (MR 2385)
Don't fail with:
  type object 'Config' has no attribute 'mirrors.pmaports'

Fixes: issue 2420
2024-07-28 21:01:26 +02:00
Anri Dellal
08758bef90
parse: deviceinfo: fix "deviceinfo.dtb missing" error (MR 2381)
Fixes #2416
2024-07-23 11:37:53 +00:00
Oliver Smith
f47115b61b
Add FIXME comment about docs for mirrors.systemd (MR 2371) 2024-07-22 11:54:45 +02:00
Oliver Smith
ebfda16d6d
args: remove -m and -mp (MR 2371)
Remove arguments to set postmarketOS and Alpine mirrors.

The current implementation is broken with the way the mirrors are now
configured: The current implementation fills a list, but the mirror
configuration code expects a string. It is not longer possible to set
an arbitrary amount of mirrors for the postmarketOS mirror. The benefit
is that we don't have comma separated values in the config file anymore
but instead just simple strings. Now it is possible to have one proper
mirror for aports, pmaports and pmaports_systemd, and one _custom one
to override each of these.

We already have too many options in "pmbootstrap -h", and changing a
mirror is more of a base configuration that you do once and maybe change
a few times, but don't need to be able to set it with each pmbootstrap
run. Users are not going to write this out manually on their
command-lines, it only makes sense in scripts and wrappers for
pmbootstrap, and there you could as well use another method (as now
described in docs/mirrors.md) to set the mirrors.

Less important, but another reason is that using "-mp" (two letters as
short argument) isn't really elegant.
2024-07-22 11:54:45 +02:00
Caleb Connolly
1fe7b9931c
build: package: queue subpkg depends for base package too (MR 2380)
We handle this correctly for dependencies of the base package but not
the base package itself.... Fix this.

Should avoid duplication here ideally.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 17:38:04 +02:00
Caleb Connolly
e5dc22b838
build: package: queue subpkg depends for build (MR 2379)
A previous attempt inserted subpackage depends into the package depends
list returned by get_depends().

This meant that pmb would try to install subpackage depends when
building the package, which breaks for some cases like device packages
with kernel variants since the depends of different subpackages might
conflict with each other.

Let's instead just add the subpackage depends to the build queue, so
they get built like package depends, without being pulled in as build
dependencies for the package.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 17:25:31 +02:00
Caleb Connolly
3680a0d7de
Revert "build: package: add subpkg depends (MR 2377)" (MR 2379)
This reverts commit 6002a73329.
2024-07-21 17:08:42 +02:00
Caleb Connolly
0b0172bafc
install: add flash_boot for fastboot-bootpart (MR 2378)
When using fastboot-bootpart flash method to split root/boot partitions
out, it was no longer possible to flash the Android boot partition.

Add a new flash_boot action to handle this.

This will allow us to migrate devices like the OnePlus 6 to have the
boot partition (ESP) on system, and the rootfs on userdata rather than
the current subpartition method.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 12:53:28 +02:00
Henrik Grimler
dd4eb60241
flashers: heimdall-isorec: use heimdall_flash_isorec_kernel.sh (MR 2360)
This wrapper script, which was added in alpinelinux's heimdall 2.1.0
package, replaces heimdall_flash_kernel.sh and takes a fifth arg with
dtb name. The script then concatenates vmlinuz and the dtb into a
vmlinuz-dtb file and flashes it to the kernel partition.

Boot-deploy was previously responsible for creating vmlinuz-dtb, but
to save disk space we have moved away from creating it
unconditionally.

Fixes: https://gitlab.com/postmarketOS/boot-deploy/-/issues/26
2024-07-21 12:50:17 +02:00
Henrik Grimler
32541b2141
flashers: drop heimdall_wait_for_device wrapper script (MR 2360)
Instead, use new --wait arg added in heimdall 2.1.0. Heimdall then
waits until it detects a compatible device and then continues with
flash action.
2024-07-21 12:50:17 +02:00
Caleb Connolly
c23a9ea9cd
install: fix root partition path in print (MR 2377)
When we output the root partition path, it says rootfs.img but it should
be root.img fix it.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-20 17:33:02 +02:00
Caleb Connolly
6002a73329
build: package: add subpkg depends (MR 2377)
Recurse dependencies of subpackages too.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-19 16:58:26 +02:00
Caleb Connolly
f07ceca9c6
chroot: apk: pass in both repos (MR 2377)
Pass in --repository for all channels not just the default.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-19 16:53:46 +02:00
Caleb Connolly
d8e5037336
build: package: build build packages even when nothing else is queued (MR 2376)
Bail on empty build queue after checking if build_packages need
building. So if apk-tools / alpine-base / etc are outdated but nothing
else is they'll still be built.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-18 03:41:05 +02:00
Oliver Smith
b14a872fad
Fix "pmbootstrap config args /new/path" (MR 2374)
Fix writing config options that are stored as CSV-list. Without this
patch:

$ pmbootstrap config aports /tmp/pmaports
[23:43:58] Config changed: aports='/tmp/pmaports'
$ pmbootstrap config aports
[PosixPath('/'), PosixPath('t'), PosixPath('m'), PosixPath('p'), PosixPath('/'), PosixPath('p'), PosixPath('m'), PosixPath('a'), PosixPath('p'), PosixPath('o'), PosixPath('r'), PosixPath('t'), PosixPath('s')]

Storing strings as CSV list isn't great and we probably want to
implement this more elegantly, see the related issue. But let's fix this
first.

Related: pmbootstrap issue 2412
2024-07-17 23:59:18 +02:00
ijiki16
e746c181a7
symlink: fix android-recovery-installer path (MR 2372) 2024-07-16 12:48:30 +04:00
Oliver Smith
37244e1c0e
docs/mirrors: new document (MR 2361) 2024-07-16 00:34:06 +02:00
Oliver Smith
89a3136f08
docs: support markdown (MR 2361)
Let's write all new documentation in markdown, and convert existing ones
at some point. I think we are all much more familiar with markdown than
with the rst format.
2024-07-16 00:34:06 +02:00
Oliver Smith
591a737733
Support setting custom mirrors + disabling mirrors (MR 2361)
Allow setting _custom mirrors in the config:
* alpine_custom
* pmaports_custom
* systemd_custom

When these are set, they are added to /etc/apk/repositories before real
repositories. This is used by bpo to build packages with a WIP
repository enabled, in addition to the final repository.

All mirrors can also be set to "none" to be disabled. This is important
for bootstrapping from pure Alpine without any binary repository, and
the bpo testsuite also uses this.

I've discussed with Caleb whether to name it _wip instead of _custom,
but the latter is more generic and people may also use this for other
use cases than the bpo wip repository thing.
2024-07-16 00:34:06 +02:00
Oliver Smith
7b164acb70
pmb.helpers.repo.urls: remove commented out print (MR 2361) 2024-07-16 00:34:06 +02:00
Oliver Smith
7c3af8c391
pmb.config: remove config file migration logic (MR 2361)
We have decided to let the user generate a pmbootstrap_v3.cfg file
instead of attempting to automatically migrate the file. This way we
don't need to worry about potential migration bugs and users can also
temporarily go back to v2 to compare behavior with v3 without problems.

Remove migration logic related to mirror_alpine and
mirrors_postmarketos, before I add support for using multiple
repositories with these again (as needed for bpo). This reduces
complexity and removes a note about "multiple mirrors are not supported"
that won't be valid anymore.
2024-07-16 00:34:06 +02:00
Oliver Smith
e7af3c49a5
init: mirror selection: set systemd mirror too (MR 2361)
When selecting a different mirror in "pmbootstrap init", then select the
systemd mirror as well. It will always be the normal postmarketOS mirror
plus some path appended. Currently it is the path for the staging
repository, this will change when the systemd changes are merged to
pmaports master and bpo can build it. I'll adjust this before we release
pmbootstrap v3.

I have considered changing the code so that we don't store a separate
systemd mirror in the config. But I think it's useful to be able to
point to systemd mirrors with different paths for debugging purposes,
and also it would make the logic more complex once we re-introduce
support for multiple mirrors of the same type for bpo. So I've
intentionally kept that part as-is.
2024-07-16 00:34:06 +02:00
Oliver Smith
642a374bac
Add a checklist comment for upgrading min py ver (MR 2357) 2024-07-16 00:26:35 +02:00
Oliver Smith
3e0eba8561
pmb/types: manual ruff fixes (MR 2357)
Manual fixes for things that ruff didn't fix automatically.

From python3 docs:
> Union[X, Y] is equivalent to X | Y and means either X or Y.
2024-07-16 00:26:35 +02:00
Oliver Smith
7b451b25de
Run ruff format (MR 2357) 2024-07-16 00:26:35 +02:00
Oliver Smith
36dd53f402
Run ruff check --fix (MR 2357)
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
2024-07-16 00:26:35 +02:00
Oliver Smith
9fd45fb334
Bump minimum python3 version to 3.10 (MR 2357) 2024-07-16 00:26:35 +02:00
HenriDellal
2b577a8694
Fix pmbootstrap install --android-recovery-zip (MR 2368)
Tweaked-By: Oliver Smith <ollieparanoid@postmarketos.org>
2024-07-16 00:04:15 +02:00
Anri Dellal
d8e174b04c
flasher boot: fix missing args (MR 2369)
Closes #2407
Closes #2408
2024-07-15 23:52:33 +02:00
Anri Dellal
b8ca63dc2d
Fix types of pmb arguments (MR 2370)
- Change some arguments' types to bool to avoid type mismatch errors
  after refactoring in the future
- Add more arguments from parse/arguments with corresponding types
- Change type of pmbootstrap log --lines argument to int
2024-07-15 23:50:13 +02:00
Arnav Singh
b98c5e2cab
config: init: fix crash in mirrors prompt (MR 2367)
This was missed in fc010bc7c8

Before this fix:

```
...

[18:34:35] Enable background timer to prevent repeated sudo authorization? (y/n) [n]: y
(020272) [18:34:44] ERROR: 'Config' object has no attribute 'mirrors_postmarketos'
(020272) [18:34:44] See also: <https://postmarketos.org/troubleshooting>
(020272) [18:34:44] Traceback (most recent call last):
  File "/home/arnavion/src/pmbootstrap/pmb/__init__.py", line 72, in main
    return config_init.frontend(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arnavion/src/pmbootstrap/pmb/config/init.py", line 729, in frontend
    ask_for_additional_options(config)
  File "/home/arnavion/src/pmbootstrap/pmb/config/init.py", line 542, in ask_for_additional_options
    logging.info("Selected mirror:" f" {','.join(context.config.mirrors_postmarketos)}")
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/arnavion/src/pmbootstrap/pmb/core/config.py", line 130, in __getattribute__
    return super().__getattribute__(key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Config' object has no attribute 'mirrors_postmarketos'
```
2024-07-15 23:47:40 +02:00
Caleb Connolly
1af44fb159
helpers: args: don't announce config options overwritten by args (MR 2366)
Was left in for debugging

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-13 18:49:45 +02:00
Caleb Connolly
01b53b0b49
config: init: set aports when non-default work dir is picked (MR 2366)
If you init and set a custom work dir it will not be relative to
pmaports anymore. Detect this and reset config.aports to be relative to
the new workdir.

Fixes #2406

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-13 18:49:22 +02:00
Caleb Connolly
2ce06e9791
build: package: properly ensure build-base packages are built first (MR 2363)
In ("build: special case building abuild (MR 2356)") we tried to ensure
that abuild would be built before anything else, however this
implementation failed miserably if the first package given didn't
actually need to be built (since we skip building deps of packages that
aren't flagged for build).

Let's take a different stab at this by actually inserting build_packages
into the build queue if they need building. They're inserted at the end
just before the queue is reversed, so they will always be built first.

This makes the order of pmb.config.build_packages important, enshrine
this with a comment.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
889a1e011d
build: package: fail gracefully if --src and dependencies need building (MR 2363)
If you build a package with --src but some dep is outdated, force you
to build it (or otherwise handle the situation yourself). We can't guess
what to do here.

This could be made configurable (opt-in) in the future.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
d2f40a9a29
build: package: update all build_packages in buildroot after building (MR 2363)
We might have just built some other package which will later be used to
build more packages. Handle all of them and don't special case abuild.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
7544823265
chroot: apk: add quiet option (MR 2363)
Let us install packages without announcing. Useful if there is some more
contextual message display already.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
de268b1b82
chroot: init: use local repos (MR 2363)
Ensure we get our fork of apk straight away without having to run `apk
upgrade -a` immediately afterwards.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:11 +02:00
Caleb Connolly
02f9da9a64
build: init: simplify installing build deps (MR 2363)
No reason to handle abuild separately here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:11 +02:00
Anri Dellal
30d8ea997f
Refactor bootimg_analyze (MR 2359) 2024-07-11 21:54:53 +02:00
Oliver Smith
aeb65f2991
Tweak errors for missing config/work dir (MR 2350)
* Use NonBugError instead of RuntimeError so users don't get a stack
  trace for this.
* Check for the pmbootstrap <= 3.0.0 config, and print a nice message
  asking users to recreate their config.
* Add a separate message if --config was used.
* Reword the generic "no config found" message to suggest 'pmbootstrap
  init' first, and only mention --config at the end. Most users will
  need to just run 'pmbootstrap init'.
2024-07-10 20:06:25 +02:00
Oliver Smith
9dfa89c58c
Use pmbootstrap_v3.cfg as config file (MR 2350)
Using pmbootstrap v3 with the old config can cause problems, for example
when having $WORK in the pmaports dir instead of the actual work path.
This is not supported anymore by v3 to reduce complexity. The format of
how mirrors are stored in the config also has changed.

Use a separate config file, so users can go back from v3 to 2.3.x if
they need to (for figuring out a regression) and so users won't run into
bugs when moving from 2.3.x to v3.
2024-07-10 20:06:25 +02:00
Anri Dellal
3d9607aa99
init: fix bootimg_analyze step (MR 2358) 2024-07-09 17:21:44 +03:00
Caleb Connolly
ebef789f7c
Bump work version to zap for pmb 3.0.0 (MR 2352)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:45:27 +02:00
Caleb Connolly
8da12a98a7
helpers: other: drop args from migrate_work_folder() (MR 2352)
Was erronesously passing args to cli.confirm.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:45:23 +02:00
Caleb Connolly
234673ce03
build: special case building abuild (MR 2356)
Since abuild is an implicit dependency of everything, we need special
case handling for it. Insert it as a dependency of the first package in
the build queue so that it will be built if outdated, then after
building re-install it in the chroot so all subsequent packages will use
the freshly built abuild.

This is currently used for systemd where we use an abuild fork

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:19:57 +02:00