Right now, they appear on screen when using --details-to-stdout. This
does not work well with Travis CI and screws up the log.
Disabling the progress bars in abuild works just like Alpine does it in
their Travis CI script: Exporting SUDO_APK as
"abuild-apk --no-progress" instead of "abuild-apk".
test_check_checksums.py: Run "pmbootstrap build_init" before building
any packages, so it is a bit less verbose (downloading the APKINDEX
files etc.). Later we run the build init code again (because we use
--strict while building the packages), but then the APKINDEX files
are already present. So overall the log is a bit shorter before the
building starts. (It is still logged to the logfile, which gets
printed on error anyway.)
* Testsuite: Run UIs in Qemu and check running processes (and other changes)
* When `pmbootstrap qemu` gets killed, it now takes down the Qemu process with it
* `test/check_checksums.py` got a new optional `--build` parameter, which makes
it build all changed packages instead of just checking the checksums
* We run this before running the testsuite now, so all changed packages get
built before running tests (otherwise tests would hang without any output
while a changed package is building)
* New testcase, that zaps all chroots, installs a specific UI (xfce4 and
plasma-mobile currently, easy to extend), runs it via Qemu and checks the
running processes via SSH.
* Version checking testcase: rewritten to include Alpine's testsuite file in
our source tree, so we don't need to clone their git repo anymore. Now it
is enabled for Travis.
* All this gives us a nice 10% code coverage boost
* Increased the `hello-world` pkgrel to verify that the Travis job is working.
* Various fixes
* Build device-packages for the device arch and don't raise an
exception, but print a note if --ignore-depends is not specified
and therefore the kernel gets installed, too.
* Don't use --force when building in Travis (because abuild doesn't
check the checksums then. Bug report on the way.)
* Don't run the building process in the background, but wait for its
completion
* Exit with 1 when showing usage in check_checksums.py
Follow up to #1162.
* `pmb.build.buildinfo()`: Used to record the build environment. It is
flawed because it scans the repo APKINDEX files instead of using the
actually installed packages list. When it was implemented we were not
able to do the latter. After this is removed, `pmb.parse.depends` can
be simplified (it needs to be rewritten for #1122).
* `pmb.helpers.repo.diff()` and `pmb.helpers.repo.files()`: These were
used exclusively by `pmb.build.buildinfo()`, to learn about which
files have been changed in the local repository folder after a
package was built. The idea was, that we could find subpackages that
way. But this information is present in the installed package list as
well, which is a much cleaner approach.
If you want to build a package without changing the version number,
please use `--force` from now on. For example:
pmbootstrap build --force hello-world
Prior to this commit, changes were detected automatically (timestamp
based rebuilds). However, that feature does not work as expected with
the binary package repository we have now, and depending on how you use
git, it has never worked. Close#1167, close#1156, close#1023 and
close#985. This commit also mentions --force when a package is up to date,
but the user requested to build it.
Preparation for #1122.
* `pmb.parse.apkindex.parse()`, removed strict parameter: This used to raise
an exception when two entries in the apkindex provided the same package.
Turns out this is *not* invalid after all, two packages can provide the same
soname for example (e.g. libhybris, mesa-egl). In an APKINDEX, sonames are
listed as they were packages ("so:libjpeg.so.8" etc.).
* Remove `pmbootstrap challenge` leftover code from reproducible builds effort,
which was a dead end. This code uses the broken strict feature.
Small improvements:
* Allow to specify multiple packages to `pmbootstrap parse_apkbuild`
* Specifying no package will parse all packages (like kconfig_check)
(also `parse_apkbuild`)
* JSON output is sorted of `parse_apkbuild`
* Make pkgver check optional, so we can disable it in the device wizard test case
* Parse_apk* -> apk*_parse
* Don't let the user mess with globs (disallow '*' in pkgname)
Fixes#893. Changes:
* New action: "pmbootstrap pkgrel_bump"
* pmbootstrap detects missing soname depends when trying to install
anyting, and suggests "pkgrel_bump --auto" to fix it
* Testcase test_soname_bump.py checks the pmOS binary package repo
for soname breakage, so we see it when CI runs for new PRs
* libsamsung-ipc: bump pkgrel because of soname bump
* Don't ask for the mesa driver when the Qemu arch is not the
native arch and always use swrast in that case
* qemu-vexpress: use LTS kernel
* qemu-aarch64: use drm-backend for weston
This reverts commit 99d7b58ee5.
People usually add manufacturer name in the phone full name which
results in having manufacturer written double in pkdesc (e.g. `Samsung
Samsung Galaxy Mini 2`)
* Changed usb-shell behavior, it wait for some user action before continue booting
* Rename usb-shell to debug-shell and changed port to 23
* Add `20-debug-shell.sh` script to static code analysis
* Enable eth0 interface in initramfs (qemu)
* Add additional script to run a shell in order to be able to kill it from a telnet session
* Rename deviceinfo variable flash_methods to flash_method
* Update pmb.config.deviceinfo_attributes / add sanity check
* Add test case that parses all deviceinfo files
* ccache: Fix for distcc cross-compiling / various improvements
* Make ccache work when cross-compiling with distcc (fix#716)
* Allow to configure the ccache size in "pmbootstrap init"
* Moved ccache stats code from pmb/build/other.py to
pmb/helpers/frontend.py
* Grouped job count, ccache size and timestamp based rebuilds
together to "build options" and allow to skip them
* Sorted config options that had to be modified anyway
alphabetically
* Improve comment in arch-bin-masquerade APKBUILD
Fixes#955. Previously we did not look through all APKINDEXes while
looking for the package with a given name and the highest version.
This caused pmbootstrap to build packages even if they are in the
binary repo and up to date.
Add qt5-qtbase with OpenGL ES2 enabled and adjust the
upstream compatibility test case.
* Test case: don't get the qt5-qtbase version from any APKINDEX, but
only from Alpine's community APKINDEX
* Test case: If the pkgver is 9999, look at _pkgver
On my system, /proc/mounts sometimes contains a line like
```
udev /media/zhuowei/redhd/pmbootstrap/chroot_native/dev/loop0p2\040(deleted)
devtmpfs rw,relatime,size=1959476k,nr_inodes=489869,mode=755 0 0
```
The "\040(deleted)" text confuses `pmbootstrap shutdown`. Remove the suffix
if we find it in a /proc/mounts entry. This fixes#545.
* Upgrade Qt libs to 5.9.3
* Add test case to make sure we are always in sync with Alpine's QT
and update qt5-qtsensors, too.
* Remove qt5-qtquickcontrols2 (it's in Alpine now)
* Rename pmb/build/package.py to pmb/build/_package.py, so we can
access the functions it contains in testcases, and still use
pmb.build.package()
* Refactor the entire file. Instead of one big function that does
too many things, we have many small ones now, that are tested
in the testsuite and easier to modify
* Whenever building a package, pmbootstrap does not only build and
install the "makedepends" (like we did before), now it does the
same for the "depends". That's required to be compatible with
abuild. The old behavior can still be used with 'pmbootstrap
build --ignore-depends'.
* Because of that change, noarch packages can no longer be built in
the native chroot if we need them for a foreign chroot. A device-
package depending on a kernel would pull in the same kernel for
the native architecture otherwise.
* Running 'pmbootstrap build device-...' without '--ignore-depends'
and without a matching '--arch' displays a note that explains
this change to the user and tells how to use it instead.
* Noarch packages no longer get symlinked. That was only
implemented for packages built in the native chroot, and now that
is not always the case anymore. Symlinking these packages creates
packages with broken dependencies anyway (e.g.
device-samsung-i9100 can't be installed in x86_64, because
linux-samsung-i9100 is armhf only).
* Rename "carch" to "arch" wherever used. Naming it "carch"
sometimes is confusing with no benefit.
* Add a testcase for the aarch64 qemu workaround (because it failed
first and I needed to know for sure if it is working again).
* Improved some verbose logging, which helped with development of
this feature.
* Removed the old "build" test case (which was disabled in
testcases_fast.sh) as the new "build_package" test case covers its
functionallity.
* Only build indexes if the packages folder exists for that arch (Travis
couldn't run a test case otherwise)
* add my own build key
* enable the repo in the config
* update the README file
* Adjust testcase, that validates the keys and enable it in testcases_fast.sh
* Only save/load keys to/from the config file, which we ask for during
'pmbootstrap init', so the binary repo gets used even if a config file
already exists (this also removes a workaround, that deletes the work
folder path from the config dictionary before writing it)
* Download missing APKINDEX.tar.gz files with Python code, before
attempting to build packages (so we know which ones aleady exist in
the binary packages repository)
* Consider APKINDEX files older than 4 hours as outdated and download
them again (also in Python code)
* Provide 'pmbootstrap update' to force-update the APKINDEX files
* Travis: more logging output on failure
* Only allow keys from config_keys to be used by "pmbootstrap config"
The mesa driver, which ends up in the installation image, needs to be known
before the installation is done (in other words: when running the qemu action,
it is to late as the image has already been generated). That's why one can
choose the Qemu mesa driver in `pmbootstrap init` now:
```
Device [qemu-amd64]:
Which mesa driver do you prefer for your Qemu device? Only select something other
than the default if you are having graphical problems (such as glitches).
Mesa driver (dri-swrast/dri-virtio) [dri-virtio]:
```
It is still possible to select `dri-swrast`, because `dri-virtio` may not work
in all cases, and that way we could easily debug it or experiment with other
mesa drivers (e.g. the "vmware" one, which is supported by mesa and Qemu).
Other changes:
* `pmbootstrap qemu` accepts a `--display` variable now, which passes the value
directly to `qemu`'s `display` option. It defaults to `sdl,gl=on` (@PureTryOut
reported that to work best with plasma mobile on his PC). `--display` and
`--spice` (which is still working) are mutually exclusive.
* Removed obsolete telnet port pass-through: We only use the debug telnet port
since osk-sdl has been merged.
* Add show-cursor to the Qemu command line, so it shows a cursor in X11
* Refactored the spice code (`command_spice` only returns the spice command,
because it has all necessary information already) and the spice port can be
specified on the commandline now (previously it was hardcoded in one place and
then always looked up from there).
* Start comments with capital letters.
* Keep the log on the screen a bit shorter (e.g. Qemu command is written to the
"pmbootstrap log" anyway, so there's no need to display it again).
* linux-postmarketos-stable: Adjust kernel configs
x86_64, armhf: enable as modules:
CONFIG_DRM_VIRTIO_GPU, CONFIG_VIRTIO_PCI, CONFIG_VIRTIO_BALLOON
aarch64: all 3 options were already enabled as built-in (no change)
* Set '-vga virtio' for mesa-dri-virtio
* pmbootstrap init: Generate new port device- and linux-package
* adds `pmbootstrap aportgen device-*` and
`pmbootstrap aportgen linux-*`
* ask for confirmation when selecting a non-existing device
* generate the packages directly from init
* refactor aportgen code
* fixed some easy things in the linux- APKBUILD (more to come in
follow-up PRs!)
Testing:
* Test all questions to the user from pmb.config.init and pmb.aportgen.device
(except for the timezone question, because we would need to monkeypatch the
os.path.exists() function, which messes up pytest, so we'd need to refactor
the timezone function to be more testsuite friendly first)
* Run the device wizard in a testcase a few times and check the output, that
pmbootstrap.aportgen.device and pmbootstrap.aportgen.linux create by parsing
the resulting APKBUILDs and deviceinfo and checking its contents.
* Build the generated device package once in the same testcase
Thanks a lot to @drebrez for all the help with this one:
<https://github.com/postmarketOS/pmbootstrap/pull/821>
See also the updated porting guide:
<https://wiki.postmarketos.org/wiki/Porting_to_a_new_device>
* Add architecture argument for the buildroot chroot, defaults to device architecture
* Output pmbootstrap log file after failure to debug Travis failure
* Travis: disable timestamp based rebuilds
Travis suddenly fails, because flake8 has a new detection rule:
Using except to catch all exceptions triggers E722.
However, we only use it once and we really want to catch all exceptions,
as it wraps the whole program and shows a nice error message, with
the full trace in the log.
* Removed obsolete apkindex_files cache testcase (the corresponding
function has been removed in #345 already).
* Fix test_challenge_apk: It failed on Travis, because we're accessing
/etc/abuild.conf, which only exists after initializing the build environment.
It's a random dummy file anyway, so I've replaced it with another file.
* Fix test_folder_size: accept a tolerance in the result
* Allow to specify a custom username in "pmbootstrap init"
* Build chroots have "pmos" instead of "user" as username now
* Installation user UID is 1000 now (as in all other Linux distributions)
* Adjust autologins
* postmarketos-base: enable wheel group for sudo, removed previous sudoers file
* Implement safe upgrade path:
We save the version of the work folder format now, in $WORK/version.
When this file does not exist, it defaults to 0.
In case it does not match the currently required version
(pmb.config.work_version), then ask the user if it should
automatically be upgraded.
We check if origin/HEAD is present. In case that reference is
missing, we show a meaningful error message now, with an explanation
on how to add it. Also moved find_out_of_sync_files_tracked() to
pmb.helpers.git
* The system image size is now calculated as: root size - home size.
* New function in `pmb/helpers/other.py`: `folder_size()`, with a
testcase.
* Instead of copying everything to the system image folder, and
deleting the home folder afterwards, do not copy the home folder
in the first place.
* Added `pmbootstrap -s` to skip generating the initramfs for faster
debugging.
* Set the default value in the "are you sure, that your partition has
at least..." to "y", so we can run `yes '' | pmbootstrap install`
to make it run through the whole installation process.
* Increase full size to 120%, boot partition gets 15 MB free space now
I've replaced all instances in the code of `os.path.abspath`
with `os.path.realpath`, as this does the same as `abspath`
plus resolving symlinks.
See also: https://stackoverflow.com/a/40311142
Previously, distutils.version.LooseVersion was used, because it was
sort of close enough to how Alpine parses versions.
This new version uses the exact same algorithm, as `apk` does, and
it passes *all* of `apk`'s testcases for version checking (previously
we simply skipped the ones, that did not pass).
* Remove pmb/helpers/version.py left-over (it is in parse now)
* Make asserts consistent, do not use unnecessary parenthesis
