This replaces the current loop calling pmbootstrap lint once for each
package with a single invocation for increased performance.
Here are some measurements for linting all packages under main/.
Before this change:
$ time .gitlab-ci/apkbuild-linting.py > /dev/null
real 3m55,840s
user 3m48,592s
sys 0m16,913s
After this change but without postmarketOS/pmbootstrap!2100:
$ time .gitlab-ci/apkbuild-linting.py > /dev/null
real 0m14,359s
user 0m17,994s
sys 0m6,488s
After this change with postmarketOS/pmbootstrap!2100:
$ time .gitlab-ci/apkbuild-linting.py > /dev/null
real 0m6,411s
user 0m13,334s
sys 0m2,417s
Note that postmarketOS/pmbootstrap!2100 is not required for that little
bit of extra performance but rather because it allows to differentiate
between linting errors for different packages in the output.
Depends: postmarketOS/pmbootstrap!2100
Closes: #564
This replaces the mkinitfs script with a re-write (in Go). The
re-written mkinitfs will atomically replace the initfs in /boot, check
for free space before doing so, and try to do whatever it can to not
leave the system in an unbootable state by botching the initfs
creation/installation.
pmb:cross-native with go cross compiling doesn't work exactly, it makes
the correct binary, but on Alpine go uses -buildmode=pie which creates a
dynamic thing, and the interpreter is wrong (e.g. it it set to use the
host arch's interpreter, like /lib/ld-musl-x86_64 even though
GOARCH=arm64)
The init.sh script is no longer a template, "initramfs-extra" is used.
Fixes https://gitlab.com/postmarketOS/pmaports/-/issues/1019
fixes https://gitlab.com/postmarketOS/pmaports/-/issues/660
Missing '/' in aports path will make glob below invalid.
It will find 0 files and thus effectively skip all tests.
Construct full path using f-strings.
Double '/' in path that can be there if args.aports ends
with slash is harmless.
Some devices have a working but non-writable real-time clock (RTC).
This package contains two shell scripts: One writes the offset between
'hwclock' and 'swclock' to a file at shutdown, another one reads the
offset from the file at boot and sets the 'swclock'. This way the system
time in userspace is kept in present time.
[ci:skip-build] already built successfully in CI
To prevent accidental bricks, postmarketos-update-kernel is not
installed by default. If it works for your device and you've added the
deviceinfo variable, you also need to depend on the package for the
updates to actually happen.
[ci:skip-build] already built successfully in CI
This disables ccache in the CI when building packages. I think ccache is
not necessary since the CI will never build a package twice in the same
pipeline, and ccache dir is not preserved for use between pipelines.
There is a real time cost to compiling with ccache enabled on the
initial compile (compressing objects for cache, etc), and some testing
with building the linux-purism-librem5 kernel in the CI shows this cost
can be significant... ~160 minutes when ccache is enabled vs ~105
minutes when it is disabled with this patch.
kernel compile CI runs referenced:
https://gitlab.com/postmarketOS/pmaports/-/jobs/1087602863https://gitlab.com/postmarketOS/pmaports/-/jobs/1087956999
Some testing on my laptop using osk-sdl with and without ccache for
initial compile:
ccache disabled:
$ rm _build -r && CCACHE_DISABLE=1 meson _build >/dev/null && CCACHE_DISABLE=1 time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
7.76
$ rm _build -r && CCACHE_DISABLE=1 meson _build >/dev/null && CCACHE_DISABLE=1 time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
7.89
$ rm _build -r && CCACHE_DISABLE=1 meson _build >/dev/null && CCACHE_DISABLE=1 time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
7.71
ccache enabled:
$ rm /tmp/ccache _build -r && CCACHE_DIR=/tmp/ccache meson _build >/dev/null&& CCACHE_DIR=/tmp/ccache time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
8.62
$ rm /tmp/ccache _build -r && CCACHE_DIR=/tmp/ccache meson _build >/dev/null&& CCACHE_DIR=/tmp/ccache time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
8.46
$ rm /tmp/ccache _build -r && CCACHE_DIR=/tmp/ccache meson _build >/dev/null&& CCACHE_DIR=/tmp/ccache time meson compile -C _build
ninja: Entering directory `_build'
[10/10] Linking target osk-sdl
8.54
We may wish to distribute some files that are explicitly
non-free, and it makes sense to place them in a separate
non-free directory rather than in main.
Unmaintained devices are device packages that:
- Are known to be broken in some way without an active maintainer
who can investigate how to fix it, or
- Have not received any updates for a very long time, or
- Are discouraged from using because they are just intended for testing.
An example for this are ports using the downstream kernel for devices
which have a mainline port that is working quite well.
Unmaintained devices are still built by bpo (otherwise it would not make
sense to keep them), but they do not show up in "pmbootstrap init".
However, it is possible to manually select them by entering the name.
pmbootstrap will warn in that case.
Unmaintained packages should have a # Unmaintained: <reason> comment
in the APKBUILD, this comment is displayed in "pmbootstrap init"
so that the user knows why the device should not be used unless they
know what they are doing.
In this git repository, everything is about aports, this distinction was only
useful back in the day when pmbootstrap and pmaports were in the same git
repository.
Refactor the install code to be generic, so we only need to add the new
path in one place when adding a new file. Put the full path into the
filename to make this possible.
I've tried to mimic the final directory structure in the aport first
(e.g. $pkgdir/sbin/swapfile -> main/postmarketos-base/sbin/swapfile),
but that leads to conflicts as abuild only takes the filename for
checksums (conflict with etc/conf.d/swapfile):
https://gitlab.alpinelinux.org/alpine/abuild/-/issues/10013
We don't really need checksum verification for the files shipped in the
same directory (not downloaded), but nevertheless this is a bug. It
leads to confusing behavior and might be fixed by letting abuild demand
that there is only one source file with the same name, as I suggested in
the issue linked above. So let's avoid this altogether with the flat
file name.
postmarketos-base will soon build its source="" variable with a shell
function. Support this use case by not trying to parse source="" with
pmbootstrap anymore, but rather parsing the trivial sha512sums
variable.
Don't print a message for each APKBUILD that is getting parsed. pytest
will display the stdout below the raised exception, so with the print
one had to scroll up a lot to find the actual error message. Now it is
printed at the bottom.
When using '[ci:skip-build]' without any changed packages, do not
attempt to run "pmbootstrap checksum" without any package arguments.
This may happen, because CI in MRs runs across all patches in the MR,
but CI on branches (master, v20.05) only run on the last commit.
Prevents CI failures like this one:
9a48b41a23
Split the aports-build job into arch-specific build-x86_64,
build-aarch64, ... jobs. Extend build_changed_aports.py to accept the
architecture as argument, and to build all packages for that arch where
possible.
Import and use pmbootstrap code for APKBUILD parsing etc, as we do this
already in various testcases running in this repository.
Co-Authored-By: Bart Ribbers <bribbers@disroot.org>
Verify _pmb_recommends for architectures listed in the APKBUILD of
postmarketos-ui-* packages, not from their binary packages. Otherwise
disabling an architecture in a merge request will not be respected, as
the binary package still has the arch enabled.
Libhybris is failing to build for armhf too. Instead of adding !armhf
everywhere, let's just remove it: as it became clear in #703, nobody is
willing to put in the effort to maintain this properly.
If you would like to continue with libhybris on postmarketOS, please
create your own aports repository and put the aports there.
Related: https://wiki.postmarketos.org/wiki/HybrisCloses: #703, #700, #678
Instead of encouraging the user to add the directory to
test_directory_structure.py, mention that we just moved the firmware to
device/{main,community,testing}/. This will be more helpful for users
who get the message after rebasing on master.
In the rare case that we need to change the directory structure, we will
figure out which file to edit.
[ci skip]: various firmware packages fail when built at the same time
(they use generic source files like "bcmdhd.cal" with different checksums)
Add common.get_upstream_branch() and use it instead of assuming "master"
as upstream branch.
Example: user forks v20.05 to v20.05_some-fix, then creates a merge
request. CI must use v20.05 as upstream branch, when comparing for
modified files etc.
This script is used by multiple repositories by now, let's make sure we
only have one place where we need to maintain it. The version in
ci-common is shellchecked and displays less output for 'pmbootstrap
init' again (unless it fails, then it shows the whole log).
Read kernel module names from files in
/etc/postmarketos-mkinitfs/modules and print out which modules are
required by which of these files as the initramfs gets generated. Put
the default modules into a new 00-default.modules. This allows mkinitfs
hooks to add modules to the initramfs.
Parse the modules from a file similar to modules-load.d, with commented
lines and empty lines ignored. Add a simple test script for that and run
it in check().
Split mkinitfs.sh into two files, so the functions (now in
mkinitfs_functions.sh) can be sourced in a test. Such a test will be
added in the next commit.
Move $BINARIES_EXTRA into the get_binaries_extra() function, so its
$(find...) does not get executed when sourcing the functions file. Move
$BINARIES into get_binaries() too for consistency.
Make it pass shellcheck by fixing obvious mistakes like '==' instead of
'='. Add "shellcheck disable" lines for things that are intentionally done
or would be a bit more complicated to rewrite. It would be nice to
refactor some things like '${outfile/initramfs-/uImage-}', but this should
be done in a separate change (patches welcome).
Remove unused variable "modules_path". While at it, add two new fatal
error messages instead of just "exit 1" without any comment and remove
redundant mkdir in APKBUILD.
