1
0
Fork 1
mirror of https://gitlab.postmarketos.org/postmarketOS/pmbootstrap.git synced 2025-07-13 03:19:47 +03:00

treewide: lint markdown files (MR 2485)

lint all the markdown files so they pass markdownlint.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
This commit is contained in:
Caleb Connolly 2024-11-12 01:51:29 +01:00 committed by Oliver Smith
parent 14d9134d8b
commit 65419c2bbe
No known key found for this signature in database
GPG key ID: 5AE7F5513E0885CB
3 changed files with 235 additions and 172 deletions

View file

@ -1,23 +1,29 @@
## Contributing
# Contributing
pmbootstrap development is being discussed in
[#postmarketOS-devel](https://wiki.postmarketos.org/wiki/Matrix_and_IRC).
### CI scripts
## CI scripts
Use `pmbootstrap ci` inside your `pmbootstrap.git` dir, to run all CI scripts
locally.
### Coding style
## Coding style
A lot of the coding style is enforced by the CI scripts.
#### Python
### Python
* Use [PEP8](https://www.python.org/dev/peps/pep-0008/).
* Max line length: 80-100 characters (use 80 for comments and most code lines
except when 100 makes much more sense; try to keep it consistent with
existing code).
except when 100 makes much more sense; try to keep it consistent with existing
code).
* Use [f-strings](https://peps.python.org/pep-0498/) for any new or modified
code, instead of any of the other string formatting methods.
* pmbootstrap should run on any Linux distribution, so we support all active
Python versions (see [here](https://www.python.org/downloads/)).
* ruff is used to enforce a consistent code style, it is run in CI and can be
easily run locally too (see Linting down below).
* Docstrings below functions are formatted in `reST` style:
```python
@ -31,19 +37,22 @@ This is a reST style.
"""
```
#### Shell scripts
### Shell scripts
* Must be POSIX compliant, so busybox ash can interpret them. (Exception: the
`local` keyword can also be used, to give variables a local scope inside
functions).
### Code patterns
## Code patterns
#### The `args` variable
This contains the arguments passed to pmbootstrap, and some additional data.
See `pmb/helpers/args.py` for details. This is a legacy construct, see
### The `args` variable
This contains the arguments passed to pmbootstrap, and some additional data. See
`pmb/helpers/args.py` for details. This is a legacy construct, see
[#1879](https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/1879).
#### Executing commands
### Executing commands
Use one of the following functions instead of Python's built-in `subprocess`:
* `pmb.helpers.run.user()`
@ -52,14 +61,15 @@ Use one of the following functions instead of Python's built-in `subprocess`:
* `pmb.chroot.root()`
These functions call `pmb.helpers.run_core.core()` internally to write to the
log file (that you can read with `pmbootstrap log`) and timeout when there is
no output. A lot of function parameters are passed through to `core()` as well,
see its docstring for a detailed description of what these parameters do.
log file (that you can read with `pmbootstrap log`) and timeout when there is no
output. A lot of function parameters are passed through to `core()` as well, see
its docstring for a detailed description of what these parameters do.
#### Using shell syntax
##### Using shell syntax
The passed commands do not run inside a shell. If you need to use shell syntax,
wrap your command with `sh -c` and use `shutil.quote` on the parameters (if
they contain untrusted input):
wrap your command with `sh -c` and use `shutil.quote` on the parameters (if they
contain untrusted input):
```py
# Does not work, the command does not run in a shell!
@ -72,22 +82,24 @@ shell_cmd = f"echo {shutil.quote(text)} > {shutil.quote(dest)}"
pmb.chroot.root(["sh", "-c", shell_cmd])
```
If you need to run many commands in a shell at once, write them into a
temporary shell script and execute that with one of the `pmb` command
functions.
If you need to run many commands in a shell at once, write them into a temporary
shell script and execute that with one of the `pmb` command functions.
### Writing files to the chroot
#### Writing files to the chroot
The users in the chroots (`root` and `pmos`) have different user IDs than the
user of the host system. Therefore we can't just write a file to anywhere in
the chroot. Use one of the following methods.
user of the host system. Therefore we can't just write a file to anywhere in the
chroot. Use one of the following methods.
#### Short files
##### Short files
```py
pmb.chroot.user(["sh", "-c", f"echo {shlex.quote(hostname)}"
" > /etc/hostname"], suffix)
```
##### Long files
#### Long files
Write to a temp dir first with python code, then move and chown the file.
```py
@ -99,30 +111,31 @@ pmb.chroot.root(["mv", "/tmp/somefile", "/etc/somefile"])
pmb.chroot.root(["chown", "root:root", "/etc/somefile"], suffix)
```
### Manual testing
## Manual testing
### APKBUILD parser
#### APKBUILD parser
Besides the python tests, it's a good idea to let the APKBUILD parsing code run
over all APKBUILDs that we have in pmaports.git, before and after making
changes. This makes it easy to spot regressions.
```
$ pmbootstrap apkbuild_parse > /tmp/new
$ git checkout master
$ pmbootstrap apkbuild_parse > /tmp/old
$ colordiff /tmp/old /tmp/new | less -R
```sh
pmbootstrap apkbuild_parse > /tmp/new
git checkout master
pmbootstrap apkbuild_parse > /tmp/old
colordiff /tmp/old /tmp/new | less -R
```
### Debugging
## Debugging
### Tab completion
#### Tab completion
When tab completion breaks, commands-line `pmbootstrap build <TAB>` will simply
not return the expected list of packages anymore. Exceptions are not printed.
To change this behavior and get the exceptions, adjust the
`eval "$(register-python-argcomplete pmbootstrap)"` line in your shell's rc
file.
not return the expected list of packages anymore. Exceptions are not printed. To
change this behavior and get the exceptions, adjust the `eval
"$(register-python-argcomplete pmbootstrap)"` line in your shell's rc file.
```
```sh
$ register-python-argcomplete3 pmbootstrap
_python_argcomplete() {
@ -148,6 +161,6 @@ _python_argcomplete() {
complete -o nospace -o default -F _python_argcomplete "pmbootstrap"
```
Copy the whole output of the command to your shell's rc file instead of the
eval line, but remove `1>/dev/null 2>/dev/null`. Then it will print exceptions
to the shell.
Copy the whole output of the command to your shell's rc file instead of the eval
line, but remove `1>/dev/null 2>/dev/null`. Then it will print exceptions to the
shell.

249
README.md
View file

@ -9,13 +9,15 @@ Find the location of the upstream repository for pmbootstrap on the
[postmarketOS homepage](https://postmarketos.org/source-code/).
Run CI scripts locally with:
```
$ pmbootstrap ci
```sh
pmbootstrap ci
```
Run a single test file:
```
$ pytest -vv ./test/test_keys.py
```sh
pytest -vv ./test/test_keys.py
```
## Issues
@ -24,12 +26,14 @@ Issues are being tracked
[here](https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues).
## Requirements
* Linux distribution on the host system (`x86`, `x86_64`, `aarch64` or `armv7`)
* [Windows subsystem for Linux (WSL)](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux)
does **not** work! Please use [VirtualBox](https://www.virtualbox.org/) instead.
* [Windows subsystem for Linux
(WSL)](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux) does
**not** work! Please use [VirtualBox](https://www.virtualbox.org/) instead.
* [Linux kernel 3.17 or higher](https://postmarketos.org/oldkernel)
* Note: kernel versions between 5.8.8 and 6.0 might
[have issues with parted](https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2309).
* Note: kernel versions between 5.8.8 and 6.0 might [have issues with
parted](https://gitlab.postmarketos.org/postmarketOS/pmbootstrap/-/issues/2309).
* Python 3.10+
* For python3 < 3.11: tomli
* OpenSSL
@ -40,220 +44,261 @@ Issues are being tracked
* tar
## Relation to pmaports
For pmbootstrap to be useful, it needs to maintain a local copy of the
[pmaports](https://gitlab.postmarketos.org/postmarketOS/pmaports) repository where
postmarketOS-specific packages are maintained. This is set up automatically, but
the local copy of pmaports does not automatically get updated. To update it, you
can run `$ pmbootstrap pull`.
The latest pmbootstrap version works with currently
[active postmarketOS releases](https://wiki.postmarketos.org/wiki/Releases).
Attempting to use pmbootstrap with old postmarketOS versions (old pmaports
branches) may result in failures and is not supported. See
`pmbootstrap_min_version` in
For pmbootstrap to be useful, it needs to maintain a local copy of the
[pmaports](https://gitlab.postmarketos.org/postmarketOS/pmaports) repository
where postmarketOS-specific packages are maintained. This is set up
automatically, but the local copy of pmaports does not automatically get
updated. To update it, you can run `$ pmbootstrap pull`.
The latest pmbootstrap version works with currently [active postmarketOS
releases](https://wiki.postmarketos.org/wiki/Releases). Attempting to use
pmboostrap with old postmarketOS versions (old pmaports branches) may result in
failures and is not supported. See `pmbootstrap_min_version` in
[pmaports.cfg](https://wiki.postmarketos.org/wiki/Pmaports.cfg_reference) for
the oldest supported pmbootstrap version for a given pmaports revision. The
upper bound is not documented.
## Usage Examples
Please refer to the [postmarketOS wiki](https://wiki.postmarketos.org) for
in-depth coverage of topics such as
[porting to a new device](https://wiki.postmarketos.org/wiki/Porting_to_a_new_device)
or [installation](https://wiki.postmarketos.org/wiki/Installation_guide). The
help output (`pmbootstrap -h`) has detailed usage instructions for every
command. Read on for some generic examples of what can be done with
`pmbootstrap`.
in-depth coverage of topics such as [porting to a new
device](https://wiki.postmarketos.org/wiki/Porting_to_a_new_device) or
[installation](https://wiki.postmarketos.org/wiki/Installation_guide). The help
output (`pmbootstrap -h`) has detailed usage instructions for every command.
Read on for some generic examples of what can be done with `pmbootstrap`.
### Installing pmbootstrap
<https://wiki.postmarketos.org/wiki/Installing_pmbootstrap>
### Basics
Initial setup:
```
$ pmbootstrap init
```sh
pmbootstrap init
```
Run this in a second window to see all shell commands that get executed:
```
$ pmbootstrap log
```sh
pmbootstrap log
```
Quick health check and config overview:
```
$ pmbootstrap status
```sh
pmbootstrap status
```
### Packages
Build `aports/main/hello-world`:
```
$ pmbootstrap build hello-world
```sh
pmbootstrap build hello-world
```
Cross-compile to `armhf`:
```
$ pmbootstrap build --arch=armhf hello-world
```sh
pmbootstrap build --arch=armhf hello-world
```
Build with source code from local folder:
```
$ pmbootstrap build linux-postmarketos-mainline --src=~/code/linux
```sh
pmbootstrap build linux-postmarketos-mainline --src=~/code/linux
```
Update checksums:
```
$ pmbootstrap checksum hello-world
```sh
pmbootstrap checksum hello-world
```
Generate a template for a new package:
```
$ pmbootstrap newapkbuild "https://gitlab.postmarketos.org/postmarketOS/tinydm/-/archive/1.2.0/tinydm-1.2.0.tar.gz"
```sh
pmbootstrap newapkbuild "https://gitlab.postmarketos.org/postmarketOS/tinydm/-/archive/1.2.0/tinydm-1.2.0.tar.gz"
```
#### Default architecture
Packages will be compiled for the architecture of the device running
pmbootstrap by default. For example, if your `x86_64` PC runs pmbootstrap, it
would build a package for `x86_64` with this command:
```
$ pmbootstrap build hello-world
Packages will be compiled for the architecture of the device running pmbootstrap
by default. For example, if your `x86_64` PC runs pmbootstrap, it would build a
package for `x86_64` with this command:
```sh
pmbootstrap build hello-world
```
If you would rather build for the target device selected in `pmbootstrap init`
by default, then use the `build_default_device_arch` option:
```
$ pmbootstrap config build_default_device_arch True
```sh
pmbootstrap config build_default_device_arch True
```
If your target device is `pine64-pinephone` for example, pmbootstrap will now
build this package for `aarch64`:
```
$ pmbootstrap build hello-world
```sh
pmbootstrap build hello-world
```
### Chroots
Enter the `armhf` building chroot:
```
$ pmbootstrap chroot -b armhf
```sh
pmbootstrap chroot -b armhf
```
Run a command inside a chroot:
```
$ pmbootstrap chroot -- echo test
```sh
pmbootstrap chroot -- echo test
```
Safely delete all chroots:
```
$ pmbootstrap zap
```sh
pmbootstrap zap
```
### Device Porting Assistance
Analyze Android
[`boot.img`](https://wiki.postmarketos.org/wiki/Glossary#boot.img) files (also
works with recovery OS images like TWRP):
```
$ pmbootstrap bootimg_analyze ~/Downloads/twrp-3.2.1-0-fp2.img
```sh
pmbootstrap bootimg_analyze ~/Downloads/twrp-3.2.1-0-fp2.img
```
Check kernel configs:
```
$ pmbootstrap kconfig check
```sh
pmbootstrap kconfig check
```
Edit a kernel config:
```
$ pmbootstrap kconfig edit --arch=armhf postmarketos-mainline
```sh
pmbootstrap kconfig edit --arch=armhf postmarketos-mainline
```
### Root File System
Build the rootfs:
```
$ pmbootstrap install
```sh
pmbootstrap install
```
Build the rootfs with full disk encryption:
```
$ pmbootstrap install --fde
```sh
pmbootstrap install --fde
```
Update existing installation on SD card:
```
$ pmbootstrap install --disk=/dev/mmcblk0 --rsync
```sh
pmbootstrap install --disk=/dev/mmcblk0 --rsync
```
Run the image in QEMU:
```
$ pmbootstrap qemu --image-size=1G
```sh
pmbootstrap qemu --image-size=1G
```
Flash to the device:
```
$ pmbootstrap flasher flash_kernel
$ pmbootstrap flasher flash_rootfs --partition=userdata
```sh
pmbootstrap flasher flash_kernel
pmbootstrap flasher flash_rootfs --partition=userdata
```
Export the rootfs, kernel, initramfs, `boot.img` etc.:
```
$ pmbootstrap export
```sh
pmbootstrap export
```
Extract the initramfs
```
$ pmbootstrap initfs extract
```sh
pmbootstrap initfs extract
```
Build and flash Android recovery zip:
```
$ pmbootstrap install --android-recovery-zip
$ pmbootstrap flasher --method=adb sideload
```sh
pmbootstrap install --android-recovery-zip
pmbootstrap flasher --method=adb sideload
```
### Repository Maintenance
List pmaports that don't have a binary package:
```
$ pmbootstrap repo_missing --arch=armhf --overview
```sh
pmbootstrap repo_missing --arch=armhf --overview
```
Increase the `pkgrel` for each aport where the binary package has outdated
dependencies (e.g. after soname bumps):
```
$ pmbootstrap pkgrel_bump --auto
```sh
pmbootstrap pkgrel_bump --auto
```
Generate cross-compiler aports based on the latest version from Alpine's
aports:
```
$ pmbootstrap aportgen gcc-armhf
Generate cross-compiler aports based on the latest version from Alpine's aports:
```sh
pmbootstrap aportgen gcc-armhf
```
Manually rebuild package index:
```
$ pmbootstrap index
```sh
pmbootstrap index
```
Delete local binary packages without existing aport of same version:
```
$ pmbootstrap zap -m
```sh
pmbootstrap zap -m
```
### Debugging
Use `-v` on any action to get verbose logging:
```
$ pmbootstrap -v build hello-world
```sh
pmbootstrap -v build hello-world
```
Parse a single APKBUILD and return it as JSON:
```
$ pmbootstrap apkbuild_parse hello-world
```sh
pmbootstrap apkbuild_parse hello-world
```
Parse a package from an APKINDEX and return it as JSON:
```
$ pmbootstrap apkindex_parse $WORK/cache_apk_x86_64/APKINDEX.8b865e19.tar.gz hello-world
```sh
pmbootstrap apkindex_parse $WORK/cache_apk_x86_64/APKINDEX.8b865e19.tar.gz hello-world
```
`ccache` statistics:
```
$ pmbootstrap stats --arch=armhf
```sh
pmbootstrap stats --arch=armhf
```
### Use alternative sudo
@ -269,16 +314,18 @@ then all files matching the glob `~/.ssh/*.pub` will be placed in
Sometimes, for example if you have a large number of SSH keys, you may wish to
select a different set of public keys to include in an image. To do this, set
the `ssh_key_glob` configuration parameter in the pmbootstrap config file to a
string containing a glob that is to match the file or files you wish to
include.
string containing a glob that is to match the file or files you wish to include.
For example, a `~/.config/pmbootstrap_v3.cfg` may contain:
```ini
[pmbootstrap]
# ...
ssh_keys = True
ssh_key_glob = ~/.ssh/postmarketos-dev.pub
# ...
```
## License
[GPLv3](LICENSE)

View file

@ -1,42 +1,45 @@
# Mirror Configuration
A typical postmarketOS installation has one Alpine Linux mirror configured as well as one
postmarketOS mirror. As Alpine's CDN mirror is used by default, it should be suitable for most
users. The postmarketOS mirror can be configured interactively with `pmbootstrap init`, under
"additional options".
A typical postmarketOS installation has one Alpine Linux mirror configured as
well as one postmarketOS mirror. As Alpine's CDN mirror is used by default, it
should be suitable for most users. The postmarketOS mirror can be configured
interactively with `pmbootstrap init`, under "additional options".
Find the currently selected mirrors in the output of `pmbootstrap status`, as well as in
`/etc/apk/repositories` for initialized chroots and finished installations.
Find the currently selected mirrors in the output of `pmbootstrap status`, as
well as in `/etc/apk/repositories` for initialized chroots and finished
installations.
## Advanced
Some advanced use cases are supported by configuring the mirrors directly, either by running the
non-interactive `pmbootstrap config` command or by editing `pmbootstrap_v3.cfg`. Find the lists of
mirrors at [mirrors.alpinelinux.org](https://mirrors.alpinelinux.org) and
Some advanced use cases are supported by configuring the mirrors directly,
either by running the non-interactive `pmbootstrap config` command or by editing
`pmbootstrap_v3.cfg`. Find the lists of mirrors at
[mirrors.alpinelinux.org](https://mirrors.alpinelinux.org) and
[mirrors.postmarketos.org](https://mirrors.postmarketos.org).
### Change the mirrors non-interactively
```
$ pmbootstrap config mirrors.alpine http://uk.alpinelinux.org/alpine/
$ pmbootstrap config mirrors.pmaports http://postmarketos.craftyguy.net/
$ pmbootstrap config mirrors.systemd http://postmarketos.craftyguy.net/extra-repos/systemd/
```sh
pmbootstrap config mirrors.alpine http://uk.alpinelinux.org/alpine/
pmbootstrap config mirrors.pmaports http://postmarketos.craftyguy.net/
pmbootstrap config mirrors.systemd http://postmarketos.craftyguy.net/extra-repos/systemd/
```
Reset to default works as with all config options:
```
$ pmbootstrap config -r mirrors.alpine
$ pmbootstrap config -r mirrors.pmaports
$ pmbootstrap config -r mirrors.systemd
```sh
pmbootstrap config -r mirrors.alpine
pmbootstrap config -r mirrors.pmaports
pmbootstrap config -r mirrors.systemd
```
### Disable the postmarketOS mirror
This is useful to test bootstrapping from pure Alpine:
```
$ pmbootstrap config mirrors.pmaports none
$ pmbootstrap config mirrors.systemd none
```sh
pmbootstrap config mirrors.pmaports none
pmbootstrap config mirrors.systemd none
```
### Use `_custom` mirrors
@ -49,7 +52,7 @@ to build packages with a WIP repository enabled in addition to the final
repository, but could also be used if you have another custom repository that
you want to use in addition to the postmarketOS binary package repository.
```
$ pmbootstrap config mirrors.pmaports_custom http://custom-repository-here
$ pmbootstrap config mirrors.systemd_custom http://custom-repository-here/extra-repos/systemd
```sh
pmbootstrap config mirrors.pmaports_custom http://custom-repository-here
pmbootstrap config mirrors.systemd_custom http://custom-repository-here/extra-repos/systemd
```