1
0
Fork 1
mirror of https://gitlab.postmarketos.org/postmarketOS/pmbootstrap.git synced 2025-07-24 21:15:10 +03:00
Commit graph

339 commits

Author SHA1 Message Date
Caleb Connolly
03bccfd037
build: package: bail out if newer binary package exists (MR 2388)
When building with --src or --force, other logic to detect when to build
a package is skipped. However there are no (known) cases when someone
might want to build a package if the resulting binary package version
will be older than an existing binary package. Catch this case and fail
with recommendations for user action (update pmaports, clear local
binary repo).

This should help avoid the situation where you accidentally build an old
version of a package and it never actually gets installed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Tweaked-by: Oliver Smith <ollieparanoid@postmarketos.org>
2024-09-17 02:27:44 +02:00
Caleb Connolly
bdc2c07837
build: package: improve readability of is_cached_or_cache() (MR 2388)
This function is really hard to understand, try to make this a bit
better by adjusting the wording to better reflect what it does.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-09-17 02:27:44 +02:00
Caleb Connolly
b63e7fa594
build: package: don't queue build_packages as dependencies (MR 2388)
build_packages like apk-tools and alpine-base might get picked up and
queued for build as dependencies of other packages. We don't want this
since we always ensure they are queued first. Avoid queuing them in this
case.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-09-17 02:27:44 +02:00
Caleb Connolly
12c14ca68a
build: other: remove unused indexes arg from get_status() (MR 2388)
This is never used so drop it to simplify things a bit.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-09-17 02:27:44 +02:00
Caleb Connolly
69fa230f3f
build: package: build outdated dependencies for packages (MR 2388)
It turns out the current behaviour was not preserving the old behaviour
at all, and was wrong. We want to build ALL outdated or new packages we
encounter via the dependency graph.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-09-17 02:27:39 +02:00
Oliver Smith
8cdaff3289
kconfig edit: update msg for copy kernel cfg back (MR 2384)
Change to "back to pamports dir" instead of "back to aports-folder".
Naming the pmaports dir just "aports" is legacy from before pmaports.git
was split out to a separate git repository.
2024-08-12 19:32:19 +02:00
Caleb Connolly
1fe7b9931c
build: package: queue subpkg depends for base package too (MR 2380)
We handle this correctly for dependencies of the base package but not
the base package itself.... Fix this.

Should avoid duplication here ideally.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 17:38:04 +02:00
Caleb Connolly
e5dc22b838
build: package: queue subpkg depends for build (MR 2379)
A previous attempt inserted subpackage depends into the package depends
list returned by get_depends().

This meant that pmb would try to install subpackage depends when
building the package, which breaks for some cases like device packages
with kernel variants since the depends of different subpackages might
conflict with each other.

Let's instead just add the subpackage depends to the build queue, so
they get built like package depends, without being pulled in as build
dependencies for the package.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-21 17:25:31 +02:00
Caleb Connolly
3680a0d7de
Revert "build: package: add subpkg depends (MR 2377)" (MR 2379)
This reverts commit 6002a73329.
2024-07-21 17:08:42 +02:00
Caleb Connolly
6002a73329
build: package: add subpkg depends (MR 2377)
Recurse dependencies of subpackages too.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-19 16:58:26 +02:00
Caleb Connolly
d8e5037336
build: package: build build packages even when nothing else is queued (MR 2376)
Bail on empty build queue after checking if build_packages need
building. So if apk-tools / alpine-base / etc are outdated but nothing
else is they'll still be built.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-18 03:41:05 +02:00
Oliver Smith
36dd53f402
Run ruff check --fix (MR 2357)
Now that we have target-version = "py310" in [tool.ruff] in
pyproject.toml, ruff check complains about using typing.Optional and
typing.Union instead of newer syntax. Run the tool to fix it.
2024-07-16 00:26:35 +02:00
Caleb Connolly
2ce06e9791
build: package: properly ensure build-base packages are built first (MR 2363)
In ("build: special case building abuild (MR 2356)") we tried to ensure
that abuild would be built before anything else, however this
implementation failed miserably if the first package given didn't
actually need to be built (since we skip building deps of packages that
aren't flagged for build).

Let's take a different stab at this by actually inserting build_packages
into the build queue if they need building. They're inserted at the end
just before the queue is reversed, so they will always be built first.

This makes the order of pmb.config.build_packages important, enshrine
this with a comment.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
889a1e011d
build: package: fail gracefully if --src and dependencies need building (MR 2363)
If you build a package with --src but some dep is outdated, force you
to build it (or otherwise handle the situation yourself). We can't guess
what to do here.

This could be made configurable (opt-in) in the future.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
d2f40a9a29
build: package: update all build_packages in buildroot after building (MR 2363)
We might have just built some other package which will later be used to
build more packages. Handle all of them and don't special case abuild.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:12 +02:00
Caleb Connolly
02f9da9a64
build: init: simplify installing build deps (MR 2363)
No reason to handle abuild separately here.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-12 00:37:11 +02:00
Caleb Connolly
234673ce03
build: special case building abuild (MR 2356)
Since abuild is an implicit dependency of everything, we need special
case handling for it. Insert it as a dependency of the first package in
the build queue so that it will be built if outdated, then after
building re-install it in the chroot so all subsequent packages will use
the freshly built abuild.

This is currently used for systemd where we use an abuild fork

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:19:57 +02:00
Caleb Connolly
c5135ac62a
build: use get_pkgver() again (MR 2353)
During rework this got lost, re-incorporate it so packages built with
--src have a relevant pkgver.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-07-09 00:07:53 +02:00
Arnav Singh
6915c6a17b
pmb.build._package: Fix installing cross-compile deps in native chroot (MR 2341) 2024-07-02 17:37:03 +02:00
Newbyte
235d25618f
pmb.build.kconfig: Annotate None return types (MR 2337)
Otherwise mypy doesn't check the return type
2024-06-24 20:45:21 +02:00
Newbyte
706a853db0
pmb.build.kconfig: Install make and gcc (MR 2337)
Otherwise pmbootstrap can't run these a few lines later. I don't know
how this worked before, but this makes $ pmbootstrap kconfig migrate
function again.
2024-06-24 20:35:51 +02:00
Newbyte
cf35e12c03
pmb.build.kconfig.get_arch: Return Arch instead of str (MR 2337)
crosscompile() in autodetect.py expects <arch> to be an Arch type
instead of a str, which is also probably what we want to move the entire
codebase towards. As such, let's convert this string into an Arch here.
2024-06-24 20:26:47 +02:00
Newbyte
b07fb4b24f
pmb.build._package: Move comment to docstring (MR 2337)
This explains a parameter, so let's put it in the right place.
2024-06-24 20:23:01 +02:00
Newbyte
ca6d0eba28
pmb.build: Properly type cross-compilation types (MR 2337) 2024-06-24 20:23:01 +02:00
Oliver Smith
0155c4227a
kconfig edit: don't run check right afterwards (MR 2329)
This is currently broken, and I'm about to rewrite a lot of related
code. Just don't run the check for now, we can bring this back
afterwards if we want to.
2024-06-23 19:16:26 +02:00
Oliver Smith
5acd280823
pmb.build.kconfig.menuconfig: fix f-string + arch (MR 2329)
Fix for:
  TypeError: can only concatenate str (not "Arch") to str
2024-06-23 19:16:26 +02:00
Oliver Smith
18fa4e58a3
Ruff: fix typing.Xxx is deprecated, use xxx instead (MR 2327) 2024-06-23 19:13:57 +02:00
Hugo Osvaldo Barrera
e421bb2d41
Auto-format codebase with ruff (MR 2325)
See: https://gitlab.com/postmarketOS/pmbootstrap/-/issues/2324
Closes: https://gitlab.com/postmarketOS/pmbootstrap/-/merge_requests/2278
2024-06-23 15:40:13 +02:00
Caleb Connolly
e6c737f342
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2472e4c612
Revert "build: init: install pigz" (MR 2252)
Nope, abuild doesn't get along with it at all.

This reverts commit 91a4cfdc9a.
2024-06-23 12:38:42 +02:00
Caleb Connolly
9fff9033f2
build: init: install pigz (MR 2252)
faster, better..

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
ceaad5c291
build: move run_abuild and friends to a new file (MR 2252)
_package.py was getting pretty big. split the actual build steps out.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
700fe32cf5
build: init: de-dup apk add steps (MR 2252)
We install abuild and then immediately install more packages, make this
one step.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
0857f1aa4e
build: package: improve logging (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
30fde53279
vermin: don't use StrEnum (MR 2252)
It was only introduced in 3.11. Instead just define __str__() manually
for the Enums where it's needed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4108b95d72
ruff: run check --fix (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
4a52f46bf0
build: fail if remote checksums mismatch (MR 2252)
This is a potential security risk. Tighten things up so we only allow
local checksums to mismatch but remote ones must be correct.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
2babb61778
build: package: support incremental builds with --src (MR 2252)
Avoid deleting build artifacts when building with --src.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
91d29a1b2a
build: package: don't copy files that are in .gitignore (MR 2252)
This avoids copying local build artifacts and other things, if they're
in .gitignore then abuild shouldn't need them anyway.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:42 +02:00
Caleb Connolly
19a54ee0d6
build: package: fix --src (MR 2252)
Make sure rsync is actually installed.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
7d85bb31d5
build: ignore invalid checksums unless --strict (MR 2252)
We can fairly easily patch abuild by appending anything we want to the
APKBUILD file after copying it to the chroot.

Leverage this to override the verify() function so that it doesn't die
when checksums don't match.

Instead let's build the package anyway, but set a flag and print a
warning with instructions on how to generate the checksums.

We should continue to require APKBUILDs in pmaports to have valid
checksums.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
77f66eb43d
build: package: more dependency resolution fixes (MR 2252)
It was the case that packages were being queued in the wrong order,
since the dependency resolver queues the package before descending into
it's dependencies. This was a bit of a goof since the top level package
was always added last. If we add it first then we can just reverse the
queue and now everything is fine...

Additionally, the logic on when a dependency should be built was a bit
wonky. A case is added for when a dependency exists only in the source
repo and the requisite package isn't marked for build. The solver will
now build the dependency regardless. This is surely an edgecase but
somehow I ran into it, I suspect due to a bug elsewhere...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
7f258516db
build: package: fix cache (MR 2252)
Big blunder lol, now it actually caches...

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
53bdb523e3
checksum: remove args (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
f2ca9c618e
ruff: run check --fix (MR 2252)
Get rid of unused imports and such

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:41 +02:00
Caleb Connolly
6087a9df8f
core: don't re-export get_context (MR 2252)
This makes testing a lot more annoying.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
efce73df2f
build: adjust logging (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
0e3386e8f3
build: index_repo: fix multiple repo support (MR 2252)
Make index_repo() run for all local binary repos.

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
59ad8fd507
build: kconfig: remove unused args (MR 2252)
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00
Caleb Connolly
50943b9fc5
build: envkernel: fix package output (MR 2252)
To support multiple aports we changed how we configure
/home/pmos/packages in the buildroot so it now points to whichever local
binary repo makes sense for the package being built.

Copy over the code to envkernel run_abuild() so the packages actually
get installed to $WORK/packages

Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
2024-06-23 12:38:40 +02:00