Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2019-05-28 10:28:30 -04:00
commit ca5ee26858
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 99 additions and 0 deletions

View file

@ -0,0 +1,15 @@
The current Linux standalone for ARM, used on armv5tel with Debian stretch, does not work:
$ cd git-annex.linux/
$ ./runshell
$ git annex
error: git-annex died of signal 4
$ git-annex
Illegal instruction
I downloaded the arm tarball, unpacked and did as above. Same if do as above as root or use the alternative way of modifying the PATH, as suggested in the installation instructions. The CPU is Feroceon 88FR131 rev 1 (v5l).
Is there a way of getting a working pre-built git-annex on such a system?
Thanks in advance!

View file

@ -0,0 +1,52 @@
I am trying to use git-annex as part of a datalad install on an Ubuntu 18.04 VirtualBox VM to set up a repository linking to some datasets at a public ftp server. When I use git-annex version 7.20190219 (installed with apt-get install git-annex-standalone from NeuroDebian) to add the links, I get the following error:
emmet@emmet-VirtualBox:~/conp-dataset/projects/1KGP-22May2019$ git annex addurl ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz file=1KGP_chr6-vcf.gz
addurl ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz Unsupported url scheme ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
download failed: Unsupported url scheme ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
failed
git-annex: addurl: 1 failed
However when I use git-annex v 6.20180227 (installed from the apt repository that came with Ubuntu 18.04) the link is generated correctly:
emmet@emmet-VirtualBox:~/conp-dataset/projects/1KGP-22May2019$ git annex addurl ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz --file=1KGP_chr6-vcf.gz
addurl ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
ALL.chr6.phase3_shapeit2_mvncall_integrated_ 100%[===========================================================================================>] 953.14M 9.81MB/s in 93s
2019-05-22 16:10:44 URL: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr6.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz [999436830] -> "/home/emmet/conp-dataset/projects/1KGP-22May2019/.git/annex/tmp/URL-s999436830--ftp&c%%ftp.1000genomes.ebi.ac.uk-dda0592051d00d19f6c947a6b3aae6a5" [1]
(to 1KGP_chr6-vcf.gz) ok
(recording state in git...)
Similarly, after publishing the correctly formed links generated with 6.20180227 to github, trying to download them on another Ubuntu VirtualBox VM with git-annex 7.20190219 installed causes the following error:
emmet@emmet-VirtualBox:~/conp-dataset/projects/1KGP-22May2019$ git annex get 1KGP_chr2-vcf.gz
get 1KGP_chr2-vcf.gz (from web...)
download failed: Unsupported url scheme ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr2.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz
Unable to access these remotes: web
Try making some of these repositories available:
00000000-0000-0000-0000-000000000001 -- web
2cc9a182-813f-4740-8157-c7a560560238 -- emmet@emmet-VirtualBox:~/conp-dataset/projects/1KGP-22May2019
(Note that these git remotes have annex-ignore set: origin)
failed
git-annex: get: 1 failed
but downloading them on that same VM with git-annex v 6.20180227 installed instead works:
emmet@emmet-VirtualBox:~/conp-dataset/projects/1KGP-22May2019$ git annex get 1KGP_chr2-vcf.gz
get 1KGP_chr2-vcf.gz (from web...)
ALL.chr2.phase3_sha 100%[===================>] 1.22G 6.99MB/s in 3m 11s
2019-05-22 16:25:44 URL: ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr2.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz [1312735578] -> "/home/emmet/conp-dataset/projects/1KGP-22May2019/.git/annex/tmp/MD5E-s1312735578--774370621affaf125e6994089f975ed7.gz" [1]
(checksum...) ok
(recording state in git...)
Would you know why this is happening? And how I can avoid this failure mode in the more up-to-date version of git-annex?

View file

@ -0,0 +1,10 @@
In an external special remote, if I set annex-readonly=true, I get
[[!format sh """
git annex --verbose --debug checkpresentkey MD5E-s18932--663e2e38c829d7b3fff12fce3a6fdb6d.fasta 380286ac-2e8f-4285-94da-406eca323411
...
(checking dnanexus...) Configuration does not allow accessing dx://file-BXF0vYQ0QyBF509G9J12g927/contaminants.clip_db.fasta
"""]]
Removing the annex-readonly setting lets checkpresentkey work. But isn't checkpresentkey a read-only operation?

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="configuration variable"
date="2019-05-23T19:31:46Z"
content="""
I think then we could add in DataLad ability to record in the repository's configuration custom git annex configuration options to be passed to git annex calls (for that repo) and this way mitigate it where needed
"""]]

View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="ndj"
avatar="http://cdn.libravatar.org/avatar/0e1938953fb670f94a4e65b81a3cc58d"
subject="comment 3"
date="2019-05-28T03:36:52Z"
content="""
I did more digging and I it looks like the issue is that I have a very old version of git (2.2.1) installed via brew that was taking precedence over the system git (2.11.0). Taking that out of the mix made everything work.
Also, by just using `runshell`, it worked as well. I may switch to using that method of adding git-annex to my path, rather than just manually tacking it on the end.
This is on MacOS 10.12.6 and git annex version 7.20190220-g1812ec5da.
Thank you both for your help.
"""]]