Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
5e0c7bf81d
5 changed files with 133 additions and 0 deletions
|
|
@ -0,0 +1,26 @@
|
|||
[[!comment format=mdwn
|
||||
username="ewen"
|
||||
avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
|
||||
subject="timer_create"
|
||||
date="2018-09-11T21:37:24Z"
|
||||
content="""
|
||||
It does very much look like something has removed `timer_create` from the Synology NAS. Eg, if I statically compile the [example `timer_create` program in the man page](http://man7.org/linux/man-pages/man2/timer_create.2.html) on Debian Jessie/Stretch with:
|
||||
|
||||
gcc -o test_timer_create.jessie -static test_timer_create.c -lrt -lpthread
|
||||
|
||||
and try to run it on the Synology NAS now, I get effectively the same symptoms:
|
||||
|
||||
ewen@nas01:~$ ./test_timer_create.jessie 1 100
|
||||
Establishing handler for signal 34
|
||||
Blocking signal 34
|
||||
timer_create: Bad address
|
||||
ewen@nas01:~$
|
||||
|
||||
(Unfortunately I have no easy way to check if with a previous Synology NAS version.)
|
||||
|
||||
The same manpage notes that it *can* be configured out of the kernel: \"Since Linux 4.10, support for POSIX timers is a configurable option that is enabled by default. Kernel support can be disabled via the CONFIG_POSIX_TIMERS option.\" for which [the patch](https://patchwork.kernel.org/patch/9422453/) notes that \"Some embedded systems have no use for them\". Obviously the Synology NAS is an embedded kernel situation. Given that the tripping point was the [release where Synology introduced their Meltdown/Spectre fixes](https://www.synology.com/en-uk/releaseNote/DS216+II#ver_23739-2), I imagine that they backported a *lot* of related fixes from later kernels, and it seems likely that they did deliberately turn off `CONFIG_POSIX_TIMERS`, even if their 1st level/2nd level helpdesk did not know about it, since timing RAM accesses is pretty much key to the common Meltdown/Spectre exploits, and thus various projects removed \"high resolution\" timers. (Unfortunately there's no `/proc/config.gz` or similar that I can see, and while I can find some [Synology open source bits](https://sourceforge.net/projects/dsgpl/files/) I got bored digging to try to find a kernel config file.)
|
||||
|
||||
The fact that later Haskell switched to `poll` probably explains why the later build works. At least that seems to a viable path forward for using `git-annex` on my NAS for now.
|
||||
|
||||
Ewen
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
[[!comment format=mdwn
|
||||
username="ewen"
|
||||
avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
|
||||
subject="LC_TIME"
|
||||
date="2018-09-11T22:23:49Z"
|
||||
content="""
|
||||
From some hunting around I can find reports of that `loadlocale.c` issue with `LC_TIME` throughout 2018. It seems to be more commonly encountered now that things like Ubuntu 18.04 are released with the newer glibc. (There's also a `LC_COLLATE` change somewhere around [mid](https://www.reddit.com/r/archlinux/comments/3g9dzf/loadlocalec130_error_on_everything/)/[late](https://www.linuxquestions.org/questions/slackware-arm-108/login-error-after-upgrade-of-current-4175562326/) 2015, which seems to be the previous change in data format impacting people in this way.)
|
||||
|
||||
The generally understood wisdom seems to be that it's caused by locale data compiled by older (glibc) tool versions than the ones being used to load it (eg, statically linked into the binaries). This is at least the second context in which I've come across it, in the last month, myself. Both being caused by statically linked binaries built on Linux versions older/newer than the one on which they were being run, approximately across the 2017/2018 calendar boundary.
|
||||
|
||||
On the Synology NAS this seems to be the locale tools version:
|
||||
|
||||
ewen@nas01:~$ locale --version
|
||||
locale (crosstool-NG 1.20.0) 2.20-2014.11
|
||||
Copyright (C) 2014 Free Software Foundation, Inc.
|
||||
This is free software; see the source for copying conditions. There is NO
|
||||
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
Written by Ulrich Drepper.
|
||||
ewen@nas01:~$
|
||||
|
||||
which is a tiny bit newer than Debian Jessie:
|
||||
|
||||
ewen@noc:~$ cat /etc/debian_version
|
||||
8.11
|
||||
ewen@noc:~$ locale --version
|
||||
locale (Debian GLIBC 2.19-18+deb8u10) 2.19
|
||||
Copyright (C) 2014 Free Software Foundation, Inc.
|
||||
This is free software; see the source for copying conditions. There is NO
|
||||
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
||||
Written by Ulrich Drepper.
|
||||
ewen@noc:~$
|
||||
|
||||
and somewhat older than Debian Stretch (2.24), Debian Unstable (2.27), or [Ubuntu 18.04](https://packages.ubuntu.com/bionic/locales) which is also 2.27. My guess from the timing is that something in the `lc_time` structure size changed between about glibc 2.24 and 2.27, maybe late 2017 or early 2018.
|
||||
|
||||
Digging in the glibc git tree turns up [commit f301e5334065e93aace667fd4a87bce6fc1dbd13](https://sourceware.org/git/?p=glibc.git;a=commit;h=f301e5334065e93aace667fd4a87bce6fc1dbd13), from 2017-10-27 which foreshadows a change with \"Now when we are about to add alternative month
|
||||
names to LC_TIME (BZ#10871) this will fail again.\". This appears to be [BugZilla #10871](https://sourceware.org/bugzilla/show_bug.cgi?id=10871) which had been open forever, but the patches seem to have been [finally pushed on 2018-01-22](https://sourceware.org/bugzilla/show_bug.cgi?id=10871#c137). Which is about the right timing for the problems seen. Building locales with a glibc after that and loading with an older libc (statically linked), or vice versa, is going to break. (I think this [might be the commit that breaks things](https://sourceware.org/git/?p=glibc.git;a=commit;h=95cb863a1ef7760a11272bbd7ba5fe62dc41be54), and if not, it's very nearby that commit in the history. Applied 2018-01-22.)
|
||||
|
||||
I don't see a work around for statically linked binaries using system locale files, short of building versions with code from on either side of that flag date and suggesting people use the right version depending on their locale files...
|
||||
|
||||
... the best kludge I've thought of so far that *might* work is to turn `LC_ALL` into `LC_...` settings for each individual one, except `LC_TIME=C`, in the hope that maybe that'll cause the (changed) time part not to load. But I haven't tested that myself.
|
||||
|
||||
Ewen
|
||||
|
||||
PS: One might hope that this change could have been done in a backwards compatible manner. But that does not appear to be the case here. glibc seem to have been perfectly fine with creating a flag day, presumably reasoning distros can always force the locales to be recompiled so they'll always stay in sync.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[!comment format=mdwn
|
||||
username="ewen"
|
||||
avatar="http://cdn.libravatar.org/avatar/605b2981cb52b4af268455dee7a4f64e"
|
||||
subject="LC_TIME"
|
||||
date="2018-09-11T23:52:23Z"
|
||||
content="""
|
||||
In case it helps, I think this is the [change to the `lc_time` structure definition](https://sourceware.org/git/?p=glibc.git;a=blobdiff;f=locale/categories.def;h=3cbb4e673882eb4ff827fa2b6d9eec2faafceff4;hp=47947f76862ea77779b821a7ca294a3eff67136e;hb=95cb863a1ef7760a11272bbd7ba5fe62dc41be54;hpb=4612268a0ad8e3409d8ce2314dd2dd8ee0af5269), and this is an example of the [change needed to a program loading them](https://sourceware.org/git/?p=glibc.git;a=blobdiff;f=locale/programs/ld-time.c;h=41864482eccb14e31e0ff6cbe27c7008f687160d;hp=67d055a1c41d859b651e3af60f55b6a1499ed107;hb=95cb863a1ef7760a11272bbd7ba5fe62dc41be54;hpb=4612268a0ad8e3409d8ce2314dd2dd8ee0af5269). Looks like two new arrays (one `char *`, one wide char), each 12 entries long, and maybe a flag for whether the \"alt month\" (names) are present (`alt_mon_defined`), but I'm not clear whether or not that is in the loaded files. Both linked from the commit I mentioned in the previous comment.
|
||||
|
||||
While the parser in that program seems to be able to cope with various formats to some extent, the [loadlocale.c code has an assert check on size](https://sourceware.org/git/?p=glibc.git;a=blob;f=locale/loadlocale.c;h=15f93bec80e1f11c114eaac1c85f77e6ed071712;hb=95cb863a1ef7760a11272bbd7ba5fe62dc41be54#l126) that makes it more brittle. (And at this point I think I've seen it fail both with \"newer locale files, older binary\" and \"older locale files, newer binary\", so I suspect both are at least \"untested\".)
|
||||
|
||||
For the Synology NAS case building on Debian Stretch will *might* still work, and building on Debian Jessie looks like it should still work, but building on Debian Testing/Unstable presumably will not. Presumably this incompatibility issue will just get more common over 2018/2019 as more things are/aren't upgraded. Joy.
|
||||
|
||||
Ewen
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
[[!format sh """
|
||||
(git-annex)hopa:~/datalad/neurovault/snapshots[master]
|
||||
$> git annex whereis 1003/13873.nii.gz
|
||||
whereis 1003/13873.nii.gz git-annex: Cannot run git-annex-remote-datalad-archives -- It is not installed in PATH (/usr/lib/git-annex.linux/bin:/usr/lib/git-core:/home/yoh/gocode/bin:/home/yoh/gocode/bin:/home/yoh/bin:/home/yoh/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/snap/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/lib/git-annex.linux/extra)
|
||||
|
||||
|
||||
$> source ~datalad/datalad-master/venvs/dev/bin/activate
|
||||
|
||||
$> git annex whereis 1003/13873.nii.gz
|
||||
whereis 1003/13873.nii.gz (3 copies)
|
||||
123c73e5-a8dc-4cff-8ffc-679c7ea67f94 -- yoh@smaug:/mnt/datasets/datalad/crawl/neurovault
|
||||
48c1556f-6241-45de-9497-338d437fcb62 -- yoh@falkor:/srv/datasets.datalad.org/www/neurovault/snapshots [origin]
|
||||
af2785da-2538-4346-a6f6-f2f30fc3f025 -- [datalad-archives]
|
||||
|
||||
datalad-archives: dl+archive:SHA256E-s6460020224--710cc05117e2290e2f793271d11e26452cdc111121e09a937dbf5a34b3cc0107.tar/neurovault_snapshot/1003/13873.nii.gz#size=23262
|
||||
|
||||
$> git annex version
|
||||
git-annex version: 6.20180807+git230-gaa291acfe-1~ndall+1
|
||||
"""]]
|
||||
|
||||
IMHO output should be like (since all information is actually available)
|
||||
|
||||
[[!format sh """
|
||||
$> git annex whereis 1003/13873.nii.gz
|
||||
git-annex: ERROR - Cannot run git-annex-remote-datalad-archives
|
||||
whereis 1003/13873.nii.gz (3 copies)
|
||||
123c73e5-a8dc-4cff-8ffc-679c7ea67f94 -- yoh@smaug:/mnt/datasets/datalad/crawl/neurovault
|
||||
48c1556f-6241-45de-9497-338d437fcb62 -- yoh@falkor:/srv/datasets.datalad.org/www/neurovault/snapshots [origin]
|
||||
af2785da-2538-4346-a6f6-f2f30fc3f025 -- datalad-archives {ERROR}
|
||||
|
||||
datalad-archives: dl+archive:SHA256E-s6460020224--710cc05117e2290e2f793271d11e26452cdc111121e09a937dbf5a34b3cc0107.tar/neurovault_snapshot/1003/13873.nii.gz#size=23262
|
||||
"""]]
|
||||
|
||||
and indeed with error code exit may be.
|
||||
|
||||
[[!meta author=yoh]]
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="emanuele.olivetti@47d88ed185b03191e25329caa6fabc2efb3118b2"
|
||||
nickname="emanuele.olivetti"
|
||||
avatar="http://cdn.libravatar.org/avatar/f51cc5c6c3a0eb28faa6491c3cbcfcce"
|
||||
subject="comment 5"
|
||||
date="2018-09-12T10:45:03Z"
|
||||
content="""
|
||||
Thanks Joey for looking into this one. I use git v2.7.4, currently shipped with Ubuntu 16.04, which may not be very recent. I don't use v6 because it is not the default in git-annex, at least till now. Should I?
|
||||
I'll try the recent commit you mention, as soon as it will be available in the Linux standalone version or in Neurodebian (currently 6.20180807-g48d11a5df and 6.20180807+git230-gaa291acfe-1~ndall+1) and I will report here shortly after.
|
||||
"""]]
|
||||
Loading…
Reference in a new issue