Administration/pmbootstrap-meow
2
0
Fork 0
forked from Mirror/pmbootstrap
Code Pull requests Activity Actions

Only download APKINDEX for relevant architectures (#1201)

Browse source 
### Only download APKINDEX for relevant architectures
We're downloading the APKINDEX files for all architectures supported by
postmarketOS currently (x86, x86_64, armhf, aarch64). Most of the time,
we only need it for the native and device arch, so this PR reduces the
downloaded files to what is really necessary.

### Intuitive pmbootstrap update logic
* pmb.helpers.repo.update():
  * Default is updating all arches where the APKBUILD files exist
  * Add existing_only parameter
  * Return True when files have been downloaded
  * Properly print which arches will be updated
  * Print update reason only in verbose log
  * Add and improve comments
* pmb.parse.arguments(), update action:
  * Add --non-existing parameter
  * Default for --arch is None (instead of arch.native)
* pmb.helpers.frontend.update():
  * Inform about --non-existing if no APKBUILDs have been updated
This commit is contained in:
Oliver Smith 2018-03-04 13:44:27 +00:00 • committed by GitHub
parent e6bb96426b
commit 99127111a1
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
8 changed files with 58 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

39
pmb/helpers/repo.py
View file

@ -106,25 +106,40 @@ def apkindex_files(args, arch=None):
return ret
def update(args, force=False):
"""
Download the APKINDEX files for all URLs and architectures.
:arg force: even update when the APKINDEX file is fairly recent
def update(args, arch=None, force=False, existing_only=False):
"""
Download the APKINDEX files for all URLs depending on the architectures.
architectures = pmb.config.build_device_architectures
:param arch: * one Alpine architecture name ("x86_64", "armhf", ...)
* None for all architectures
:param force: even update when the APKINDEX file is fairly recent
:param existing_only: only update the APKBUILD files that already exist,
this is used by "pmbootstrap update"
:returns: True when files have been downloaded, False otherwise
"""
# Architectures and retention time
architectures = [arch] if arch else pmb.config.build_device_architectures
retention_hours = pmb.config.apkindex_retention_time
retention_seconds = retention_hours * 3600
# Find outdated APKINDEX files. Formats:
# outdated: {URL: apkindex_path, ... }
# outdated_arches: ["armhf", "x86_64", ... ]
outdated = {}
outdated_arches = []
for url in urls(args, False):
for arch in architectures:
# APKINDEX file name from the URL
url_full = url + "/" + arch + "/APKINDEX.tar.gz"
cache_apk_outside = args.work + "/cache_apk_" + arch
apkindex = cache_apk_outside + "/APKINDEX." + hash(url) + ".tar.gz"
# Find update reason, possibly skip non-existing files
reason = None
if not os.path.exists(apkindex):
if existing_only:
continue
reason = "file does not exist yet"
elif force:
reason = "forced update"
@ -133,19 +148,25 @@ def update(args, force=False):
if not reason:
continue
# Update outdated and outdated_arches
logging.debug("APKINDEX outdated (" + reason + "): " + url_full)
outdated[url_full] = apkindex
if arch not in outdated_arches:
outdated_arches.append(arch)
# Bail out or show log message
if not len(outdated):
return
return False
logging.info("Update package index for " + ", ".join(outdated_arches) +
" (" + str(len(outdated)) + " file(s))")
# Show one message only
logging.info("Update package index (" + str(len(outdated)) + "x)")
# Download and move to right location
for url, target in outdated.items():
# Download and move to right location
temp = pmb.helpers.http.download(args, url, "APKINDEX", False,
logging.DEBUG)
target_folder = os.path.dirname(target)
if not os.path.exists(target_folder):
pmb.helpers.run.root(args, ["mkdir", "-p", target_folder])
pmb.helpers.run.root(args, ["cp", temp, target])
return True