Commit graph

1017 commits

Author SHA1 Message Date
Oliver Smith
e94ed7b03d Close #212: Split heimdall in heimdall-isorec and heimdall-bootimg (#254) 2017-07-25 18:56:10 +00:00
clayton craft
924546135c De-couple weston from postmarketos-base (#233)
Thanks, craftyguy!

- UI is selectable with `pmbootstrap init`. Currently only 'weston' and 'none' are options. It'll automatically pick up any new `postmarketos-ui-*` packages added at later dates.
- splits off weston packages install from postmarketos-base and puts
them into postmarketos-ui-weston. Also note that NO weston packages are
installed by "pmbootstrap install" by default unless the user selects a ui in the `init`
- configuration of weston is now in postmarketos-ui-weston.
- the demos have been spun off to `postmarketos-demos`, and `postmarketos-ui-weston` lists this package as a dependency.
2017-07-24 20:55:55 +00:00
clayton craft
fcf593531c Do not store aports directory in config file (#247)
This addresses #246 by not storing the aports directory in the config
file. The default location is still available (from config/__init__.py),
and can be overridden.
2017-07-24 18:47:11 +00:00
cmdr2
b687ca70d5 Update runtime error message thrown for grsec/hardened kernels (#236)
* Update runtime error message thrown for grsec/hardened kernels, saying that they are not supported right now, #107

* Remove unnecessary check for grsec enable/disable status - the build will not work in either case
2017-07-23 12:29:13 +00:00
cmdr2
2afe205243 Ignore directories while searching for loop devices (#235)
Thanks, cmdr2! Here's the text from his PR:

Fixes an issue on alpine-vanilla, where `/dev/loop` is a directory (containing `/dev/loop/0`, `/dev/loop/1` ..etc), and causes `./pmbootstrap install` to fail.

Ignoring the `/dev/loop` directory works because `/dev/` also contains files named `/dev/loop0`, `/dev/loop1` ..etc, which the new code will still pick up.

The `install` command passes after this fix on my alpine-vanilla.
2017-07-23 12:24:56 +00:00
Martijn Braam
14f164b8a2 Merge pull request #218 from postmarketOS/fix/162-install-no-args.add
Install: Small improvements (Fix #162, Close #215)
2017-07-23 12:08:56 +02:00
Oliver Smith
13166f0f79
Fix #230: pmbootstrap flasher flash_system is broken
Regression from the cmdline argument feature.
2017-07-23 00:17:01 +02:00
drebrez
0e5808bed1 Close #198: Add possibility to override the kernel commandline during boot (#217)
Usage example:
pmbootstrap flasher boot --cmdline "PMOS_NO_OUTPUT_REDIRECT"

Thanks, drebrez!
2017-07-22 21:16:51 +00:00
Oliver Smith
66cda810c1
Improve the FDE hint, you need to type in the password twice. 2017-07-22 12:49:43 +02:00
Oliver Smith
4786516a22
Fix #219 for real!
100% tested. Assuming that you don't need to test an obvious change,
because it only changes one line is dangerous. I will learn from this,
sorry for the inconvenience.

`gcc-armhf` compiles fine again with that change, and it is able to
cross-compile packages as it used to.
2017-07-22 12:36:00 +02:00
Oliver Smith
c730326d3c Close #195: Ignore parameters in dependencies (#225)
* Ignore `>`, `<`, `=` and `!` operators, when they are specified in
  the dependencies. This was the desired behavior before, but it was
  not implemented correctly (so it wouldn't ignore them everywhere).
  Of course the real fix would be to honor these operators like apk
  does. But this isn't feasible right now, and it should work for
  most, if not all, our use-cases. I have documented this in the wiki
  under build internals and if we happen to need correct operator
  handling, we should do it then.

Minor other changes:
* `pmbootstrap parse_apkindex`: support optional package parameter to
  only show the parsed content for one package.
* Support building most python APKBUILDs by replacing ${pkgname#py-}
  properly
2017-07-22 09:54:49 +00:00
Oliver Smith
177f6ea592
Fix PEP8 syntax
Spaces around a plus sign.
2017-07-22 11:07:05 +02:00
Oliver Smith
9e5683f151
Fix #219: pmbootstrap could not find g++-armhf/g++-aarch64 2017-07-22 09:49:24 +02:00
Oliver Smith
223f584e26
pmbootstrap shutdown: Make cryptsetup closing slightly more robust
I've had a strange case where `cryptsetup status` did not work
inside the chroot anymore, and only zapping it would resolve it.
But I couldn't zap, because the status check was preventing shutdown
(on which zap depends) from working. This commit works around that.
2017-07-21 23:44:28 +02:00
Oliver Smith
77d7ed1679
Close #215: Write explicity what passwords are for in install 2017-07-21 21:14:16 +02:00
Oliver Smith
5651c28b41
Fix #162: flash_system: meaningful error for missing image 2017-07-21 20:26:56 +02:00
Oliver Smith
56b34212f6 Various distccd related improvements, mostly respect --verbose (#216)
I've done some refactoring while debugging #209.
* Unused file `pmb/build/crosscompiler.py` removed (that was a
  left over from `_pmb_build_in_native_chroot` hack
* Do verbose logging in distccd, when `pmbootstrap --verbose` is
  being invoked
* Restart distccd, when the commandline has changed (e.g. when the
  currently running version was not verbose, and the new one is
  verbose.) Prior to this change, it only got restarted, when the
  architecture changed (so it did not allow changing the job count
  on the fly for example).
* Insert missing whitespace in arguments help.
2017-07-21 16:25:52 +00:00
Oliver Smith
cf0dc2a989
Fix #209: Cross-compiling fails with C++ code 2017-07-21 17:21:32 +02:00
Oliver Smith
9677d89f84 Fix #145: add deviceinfo_flash_offset_base again (#175)
* Fix #145: add deviceinfo_flash_offset_base again

* Add default OFFSET_BASE value in flasher if deviceinfo_flash_offset_base variable is not set
...and set correct default base value in create_bootimg function
2017-07-19 20:44:12 +00:00
Oliver Smith
ca5b8e3367
Fix #200: Parse depends_dev in APKBUILDs 2017-07-19 21:24:03 +02:00
Pablo Castellano
6be7d656ee Two trivial changes (typo + info message) (#179)
Thanks, Pablo Castellano!

* Fixed typo

* Guide user about what are the next steps after flashing kernel and password

* Increased postmarketos-mkinitfs' pkgver and rewording

Changes requested:
https://github.com/postmarketOS/pmbootstrap/pull/179
2017-07-19 18:05:34 +00:00
Oliver Smith
1aae094b50 Close #174: Put frontend functions in extra file (#185) 2017-07-18 19:01:11 +00:00
Clayton Craft
3ca54dc87b [RX51] Export u-boot kernel image with flasher
This properly exports the uImage kernel, which is used by u-boot, when
running 'flasher export'. Note the change to the uImage name to follow
the pattern "uImage-$flavor", which in the case of the RX 51 is
"uImage-nokia-rx51".
2017-07-17 12:14:05 -07:00
Oliver Smith
ec57bd7df1 Merge pull request #144 from postmarketOS/heimdall-partitions-in-deviceinfo
Close #22: deviceinfo: add heimdall partitions / make flasher variables optional
2017-07-14 20:48:38 +00:00
Oliver Smith
69bb66cbd4
Challenge APK files: less output by default 2017-07-14 19:32:57 +02:00
Oliver Smith
50d23e2fd5
Fix #194: ismount() fails on symlinks 2017-07-14 07:05:45 +02:00
Oliver Smith
63aa1eae1b
AutoPEP8 1.3.2 slightly changed automatic formatting 2017-07-14 07:00:07 +02:00
Oliver Smith
f37367c57c
Properly support specifying a local folder as --mirror-pmOS
This is required for developing and testing the binary repository
scripts (see #64). Changes:
* When specified, the local folder gets mounted inside the chroots
  as /mnt/postmarketos-mirror
* The apkindex_files() function outputs the correct path to the local
  repository (it does *not* hash the URL in that case, which would
  be wrong)
* /etc/apk/repositories: when the pmOS mirror is a local folder,
  the path "/mnt/postmarketos-mirror" gets added to that file instead
  of the outside path (so apk finds it properly inside the chroot)
2017-07-12 21:55:47 +02:00
Oliver Smith
8e3e296cbd
Depends parsing: Properly handle empty depends / buildinfo: strict depends 2017-07-12 21:01:40 +02:00
Oliver Smith
4f4588405f
Properly update /etc/apk/repositories when the mirrors change
This is important for the binary repository scripts, so it's feasible
to test the binary package build and challenge process locally without
setting up a new chroot whenever changing the repo URLs.

Also it behaves a bit more intuitively, because it really uses the
repo URL specified on the commandline, even when the chroot is already
set up.
2017-07-11 19:19:39 +02:00
Oliver Smith
34622368d9
Fix #107: Grsec check: read file in /proc as root 2017-07-11 18:50:40 +02:00
Oliver Smith
d0ffe4879d
Challenge APKINDEX.tar.gz: allow .diff.md files in the repository 2017-07-11 18:39:28 +02:00
Oliver Smith
c3a4446166
Challenge APKINDEX.tar.gz: add logging 2017-07-11 18:26:22 +02:00
Oliver Smith
efab009a1f
deviceinfo: add heimdall partitions / make flasher variables optional 2017-07-10 20:27:06 +02:00
Oliver Smith
51bdc24315 Properly rebuild/install packages when something changed (Fix #120, #108, #131) (#129)
TLDR: Always rebuild/install packages when something changed when executing "pmbootstrap install/initfs/flash", more speed in dependency resolution.
---
pmbootstrap has already gotten some support for "timestamp based rebuilds", which modifies the logic for when packages should be rebuilt. It doesn't only consider packages outdated with old pkgver/pkgrel combinations, but also packages, where a source file has a newer timestamp, than the built package has.

I've found out, that this can lead to more rebuilds than expected. For example, when you check out the pmbootstrap git repository again into another folder, although you have already built packages. Then all files have the timestamp of the checkout, and the packages will appear to be outdated. While this is not largely a concern now, this will become a problem once we have a binary package repository, because then the packages from the binary repo will always seem to be outdated, if you just freshly checked out the repository.

To combat this, git gets asked if the files from the aport we're looking at are in sync with upstream, or not. Only when the files are not in sync with upstream and the timestamps of the sources are newer, a rebuild gets triggered from now on.

In case this logic should fail, I've added an option during "pmbootstrap init" where you can enable or disable the "timestamp based rebuilds" option.

In addition to that, this commit also works on fixing #120: packages do not get updated in "pmbootstrap install" after they have been rebuilt. For this to work, we specify all packages explicitly for abuild, instead of letting abuild do the resolving. This feature will also work with the "timestamp based rebuilds".

This commit also fixes the working_dir argument in pmb.helpers.run.user, which was simply ignored before.

Finally, the performance of the dependency resolution is faster again (when compared to the current version in master), because the parsed apkbuilds and finding the aport by pkgname gets cached during one pmbootstrap call (in args.cache, which also makes it easy to put fake data there in testcases).

The new dependency resolution code can output lots of verbose messages for debugging by specifying the `-v` parameter. The meaning of that changed, it used to output the file names where log messages come from, but no one seemed to use that anyway.
2017-07-10 15:23:43 +00:00
Oliver Smith
05e79c1d5c
Fix #107: Meaningful error for grsecurity/chroot_deny_chmod 2017-07-06 21:16:00 +02:00
Oliver Smith
6b08d31652 Merge pull request #136 from postmarketOS/build-all-kernels-natively
[i9070][i9100] Build kernels natively
2017-07-06 16:03:48 +00:00
Oliver Smith
5cd07c44fb
Display device count in 'pmbootstrap init' 2017-07-05 18:49:52 +02:00
Oliver Smith
dede11657d
Get rid of the _pmb_build_in_native_chroot hack 2017-07-05 18:40:00 +02:00
Oliver Smith
4fd1420f09
Add 'pmbootstrap aportgen busybox-static-armhf' (and other archs) 2017-07-05 18:20:13 +02:00
Oliver Smith
9d9dc9e541 Merge pull request #134 from postmarketOS/fix-missing-flavor-variable
Added flavor to the flash_system command for consistency
2017-07-05 14:19:31 +00:00
Martijn Braam
4cdf51f94e Changed flasher.run()s flavor to an optional argument 2017-07-05 14:01:54 +02:00
Martijn Braam
d16c90b39c Added flavor to the flash_system command for consistency, This fixes issue #133 2017-07-05 13:39:50 +02:00
Martijn Braam
2518627579 Fixed typo in new boot.img flasher 2017-07-05 00:48:19 +02:00
Oliver Smith
866333ef6c
Close #77: Implement 'pmbootstrap flasher export' 2017-07-04 23:03:51 +02:00
Oliver Smith
615880e161
pmbootstrap flasher: use boot.img file with fastboot 2017-07-04 22:47:03 +02:00
Pablo Castellano
ba744abae0 Show path to boot.img and description of commands after pmbootstrap install
This change got lost in this PR merge:
https://github.com/postmarketOS/pmbootstrap/pull/125
2017-07-03 23:49:05 +02:00
Oliver Smith
4573863561
Rename args.no_fde to args.full_disk_encryption 2017-06-28 17:22:48 +02:00
Oliver Smith
86cb9f5ec8
Merge branch 'optional_full_disk_encryption' of https://github.com/PabloCastellano/pmbootstrap into optional-fde 2017-06-28 17:16:56 +02:00
Oliver Smith
28a0e10e56
pmb.chroot.apk.installed(): use apk's internal database
...instead of running apk every time to get the list of installed
packages and their versions. The internal package database from
apk has the same format, as the extracted APKINDEX file (except
that it has more key-value pairs, which we ignore/do not need
right now). So the APKINDEX code has been extended to parse both
tar-packed APKINDEX files and regular text files in the APKINDEX
format.

This is required for #108, for a better detection of outdated
packages (because the internal package database  saves the
package's timestamp, too). A nice benefit is, that this is faster
than calling apk every time and it doesn't fill up the log as much.

I've also used this improved function for determining the apk
version (for the outdated version check), and I've deleted
pmb.parse.other.package_split(), as it is not needed anymore.
2017-06-27 17:56:15 +02:00