Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2017-07-31 14:45:46 -04:00
commit aaa59cce8c
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 2132 additions and 0 deletions

View file

@ -0,0 +1,67 @@
[[!comment format=mdwn
username="ilovezfs"
avatar="http://cdn.libravatar.org/avatar/f2b3954cf2ed0a551de9a49d3b6a64d0"
subject="pipe: resource exhausted (Too many open files)"
date="2017-07-30T16:05:10Z"
content="""
Hi Joey,
Yes, I can work around that particular failure by tweaking constraints.
In particular,
--allow-newer=\"sandi:base\" --allow-newer=\"aws:time\"
I also first need to \"cabal install Cabal<1.25\" because git-annex seems
to have a compatibility problem with Cabal-2.0.0.2. The error with
Cabal-2.0.0.2 is
[[!format sh \"\"\"
Building executable 'git-annex' for git-annex-6.20170520..
Build/BuildVersion.hs:1:1: error:
File name does not match module name:
Saw: Main
Expected: Build.BuildVersion
|
1 | {- Outputs the version of git-annex that was built, for use by
| ^
cabal: Leaving directory '.'
cabal: Error: some packages failed to install:
git-annex-6.20170520-3mbb3kAWTlXI0EMervIo66 failed during the building phase.
The exception was:
ExitFailure 1
\"\"\"]]
But, once I've installed the lower version of Cabal, and tweaked the
constraints for sandi and for aws, git-annex does build successfully.
However, at that point a more serious issue comes up when I run \"git
annex test\" and here is the full log:
https://gist.github.com/ilovezfs/d430f589d87d7f7237591cd51b2b94cb
Here is a snippet:
[[!format sh \"\"\"
From ../../.t/tmprepo53
* [new branch] git-annex -> r2/git-annex
* [new branch] master -> r2/master
* [new branch] synced/master -> r2/synced/master
conflictor/subfile
conflictor.variant-cc12
conflictor/subfile
conflictor.variant-cc12
OK (3.27s)
conflict resolution symlink bit: OK
conflict resolution (uncommitted local file): FAIL (0.49s)
sync failed in r1
conflict resolution (removed file): FAIL
Exception: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files)
conflict resolution (nonannexed file): FAIL
Exception: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files)
conflict resolution (nonannexed symlink): FAIL (0.03s)
git annex init failed
conflict resolution (mixed locked and unlocked file): FAIL
Exception: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files)
map: FAIL
Exception: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files)
\"\"\"]]
"""]]

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,50 @@
### Please describe the problem.
subject
### What steps will reproduce the problem?
repo from http://datasets.datalad.org/workshops/nih-2017/ds000114/derivatives/freesurfer/
### What version of git-annex are you using? On what operating system?
6.20170525+gitge1cf095ae-1~ndall+1
### Please provide any additional information below.
[[!format sh """
[student3@helix freesurfer]$ 'git' '-c' 'receive.autogc=0' '-c' 'gc.auto=0' 'annex' 'get' '--debug' '--json' '--json-progress' 'sub-01/label/lh.aparc.a2009s.annot'
[2017-07-30 09:33:42.479678681] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","ls-files","--cached","-z","--","sub-01/label/lh.aparc.a2009s.annot"]
[2017-07-30 09:33:42.501443872] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2017-07-30 09:33:42.523673251] process done ExitSuccess
[2017-07-30 09:33:42.523765181] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2017-07-30 09:33:42.544559473] process done ExitSuccess
[2017-07-30 09:33:42.553475534] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2017-07-30 09:33:42.554641292] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
[2017-07-30 09:33:42.591306199] call: curl ["-sS","-f","-L","-C","-","-#","-o","/scratch/nih-2017/ds000114/derivatives/freesurfer/.git/annex/tmp/MD5E-s831572--7437ad482af573329af54386a15007fe","https://dl.dropbox.com/s/85yms1fl56pq7io/lh.aparc.a2009s.annot?dl=0","--user-agent","git-annex/6.20170525+gitge1cf095ae-1~ndall+1"]
curl: (77) error setting certificate verify locations:
CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
[2017-07-30 09:33:42.693467186] process done ExitFailure 77
{"command":"get","wanted":[{"here":false,"uuid":"00000000-0000-0000-0000-000000000001","description":"web"},{"here":false,"uuid":"5e47b3f3-f09c-4969-8885-920a49ff8a45","description":"yoh@smaug:/mnt/btrfs/datasets/datalad/crawl/workshops/nih-workshop-2017/ds000114/derivatives/freesurfer"}],"note":"Try making some of these repositories available:\n\t00000000-0000-0000-0000-000000000001 -- web\n \t5e47b3f3-f09c-4969-8885-920a49ff8a45 -- yoh@smaug:/mnt/btrfs/datasets/datalad/crawl/workshops/nih-workshop-2017/ds000114/derivatives/freesurfer\n","skipped":[],"success":false,"key":"MD5E-s831572--7437ad482af573329af54386a15007fe","file":"sub-01/label/lh.aparc.a2009s.annot"}
git-annex: get: 1 failed
[student3@helix freesurfer]$ 'git' '-c' 'receive.autogc=0' '-c' 'gc.auto=0' 'annex' 'get' '--json' '--json-progress' 'sub-01/label/lh.aparc.a2009s.annot'
curl: (77) error setting certificate verify locations:
CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: /etc/ssl/certs
{"command":"get","wanted":[{"here":false,"uuid":"00000000-0000-0000-0000-000000000001","description":"web"},{"here":false,"uuid":"5e47b3f3-f09c-4969-8885-920a49ff8a45","description":"yoh@smaug:/mnt/btrfs/datasets/datalad/crawl/workshops/nih-workshop-2017/ds000114/derivatives/freesurfer"}],"note":"Try making some of these repositories available:\n\t00000000-0000-0000-0000-000000000001 -- web\n \t5e47b3f3-f09c-4969-8885-920a49ff8a45 -- yoh@smaug:/mnt/btrfs/datasets/datalad/crawl/workshops/nih-workshop-2017/ds000114/derivatives/freesurfer\n","skipped":[],"success":false,"key":"MD5E-s831572--7437ad482af573329af54386a15007fe","file":"sub-01/label/lh.aparc.a2009s.annot"}
git-annex: get: 1 failed
while works fine without:
[student3@helix freesurfer]$ 'git' '-c' 'receive.autogc=0' '-c' 'gc.auto=0' 'annex' 'get' 'sub-01/label/lh.aparc.a2009s.annot'
get sub-01/label/lh.aparc.a2009s.annot (from web...)
/scratch/nih-2017/ds000114/ 100%[===========================================>] 812.08K --.-KB/s in 0.08s
2017-07-30 09:34:02 URL:https://dl.dropboxusercontent.com/s/85yms1fl56pq7io/lh.aparc.a2009s.annot?dl=0 [831572/831572] -> "/scratch/nih-2017/ds000114/derivatives/freesurfer/.git/annex/tmp/MD5E-s831572--7437ad482af573329af54386a15007fe" [1]
(checksum...)
(recording state in git...)
"""]]
[[!meta author=yoh]]

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="issue seems to be out there"
date="2017-07-31T15:03:41Z"
content="""
[https://github.com/datalad/datalad/issues/1678](https://github.com/datalad/datalad/issues/1678)
git annex standalone is quite recent so those manpage workarounds should be in place
"""]]