We have access to the fancy runners since GitLab.com donate premium to
us. There's no downside to making the most of them while we can. We
don't depend on them, but they'll make our CI go zoom!
For the next few months until we migrate to self-hosted :D
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
Introduce support for having CI automatically update packages using
project access tokens and scheduled pipelines.
The pipeline schedule is configured with an additional variable
containing a list of packages to update, the autoupdate-fetch.sh script
then calls a handler for each of these, the handler is expected to check
for an update, make the necessary APKBUILD changes and commit them.
After all packages have been updated, the changes are pushed to pmaports
master using a project access token.
[ci:skip-build] takes too long to build in CI
Rules are the new and much more configurable way to handle conditionally
enabling jobs. Replace the old "only" keyword usage with equivalent
rules.
Signed-off-by: Caleb Connolly <caleb@postmarketos.org>
See similar patch in pmbootstrap for rationale:
27618d5ffd
W504 ignore is removed as the rule does not exist in Ruff. But, since it
was ignored anyway, this does not matter.
* Check for space indentation instead of tabs
* Check that an entry actually exists
* Check that a directory ends in a slash, otherwise GitLab seems to
ignore it
Reimplement the distfile-check as python test inside this git
repository, so we don't need to download a tarball of ci-common.git to
run it. This would not have been nice for running the test with
'pmbootstrap ci', as we don't want it to get downloaded every time the
test runs.
This new implementation is done in less than 40 lines of code, very fast
and doesn't need to create a lot of files in /tmp to build a
"distfiletree".
Adjust all CI scripts that make use of .ci/common.py to
'pmbootstrap ci'. Move all scripts that are not direct entry points to
running CI scripts to .ci/lib.
Comment out the dtb check, as it is failing. Apparently it didn't run
properly before. Let's fix this after this CI change is done.
Split out the grep for CHANGEME into a separate .ci/grep.sh script, as
it's not related to shellcheck and also there's another grep check that
I'll move there in a follow-up patch.
Add a wrapper .ci/wiki.sh, that pmbootstrap ci can run and use it in
gitlab-ci.yml. Add a .ci/lib/gitlab_prepare_ci.sh script that makes a
user available with sudo, as expected by the pmbootstrap ci compatible
scripts. Set up binfmt_misc too while at it, it will be needed for
running pmbootstrap.
The next patches will adjust the rest of the CI scripts and move the old
scripts in the .ci/lib dir, so only the scripts that should be called
directly are in the .ci dir.
Related: https://postmarketos.org/pmb-ci
A common issue is that building kernels takes longer than the default of
1h, and then MR authors need to adjust the timeout in their pmaports
fork and run it again. By setting this option this is hopefully no
longer the case.
I set it to 10h because that's what we currently have set in the
postmarketOS gitlab configuration and it would allow building MRs for
service packs with multiple kernels.
pmbootstrap has logic built in to detect stale builds - by default it
will stop a build if there was no output for 15 minutes. So having a
large timeout shouldn't be a problem.
Adjust to shellcheck removal in Alpine. Currently it's still in edge for
x86_64, but installation fails with:
ERROR: unable to select packages:
so:libffi.so.7 (no such package):
required by: shellcheck-0.7.2-r1[so:libffi.so.7]
Use the official static binary release from shellcheck instead of the
Alpine package. Version 0.7.2 is intentionally used instead of latest
8.0.0, since the latter generates a new warning. Let's unbreak CI first
before adjusting to a newer shellcheck version.
Related: https://lists.alpinelinux.org/~alpine/devel/%3C20211021133615.32f08070%40ncopa-desktop.lan%3E
CI is currently failing at the "apk upgrade" line with the following
error. It must be related to the openssl3 transition that is going on in
Alpine:
485B6261A57F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1882:
ERROR: musl-1.2.2-r6: Permission denied
485B6261A57F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1882:
ERROR: busybox-1.34.0-r3: Permission denied
485B6261A57F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1882:
ERROR: musl-utils-1.2.2-r6: Permission denied
So let's get rid of it. It shouldn't be needed to install shellcheck
anyway.
This does not work as expected, CI doesn't run for x86 and armhf on
master too. An attempt to fix this was made in MR 2030, but it didn't
work as expected and it's not worth looking into this further right now.
This reverts commit d695455f08.
Split the aports-build job into arch-specific build-x86_64,
build-aarch64, ... jobs. Extend build_changed_aports.py to accept the
architecture as argument, and to build all packages for that arch where
possible.
Import and use pmbootstrap code for APKBUILD parsing etc, as we do this
already in various testcases running in this repository.
Co-Authored-By: Bart Ribbers <bribbers@disroot.org>
This script is used by multiple repositories by now, let's make sure we
only have one place where we need to maintain it. The version in
ci-common is shellchecked and displays less output for 'pmbootstrap
init' again (unless it fails, then it shows the whole log).
Do not run upstream compatibility checks whenever pushing to master.
This is confusing, because if the upstream compatibility check fails,
then it appears that the last patch was broken in some way although it
isn't related.
I've moved the upstream compatibility checks to a separate repository
already, and added a badge to the pmaports gitlab project that indicates
whether they are currently succeding or not. The checks run hourly now.
Related: https://gitlab.com/postmarketOS/monitoringFixes: #457
Since we copy these aports directly from Alpine and try to keep them in
sync, we don't want to differ from upstream because our CI doesn't pass
on linting. If the APKBUILD should be improved, it should be done
upstream which is then synced back to us.
