diff --git a/doc/bugs/Resource_exhausted/comment_3_747d16d050fdcf69dd3d2bc5ca469a2e._comment b/doc/bugs/Resource_exhausted/comment_3_747d16d050fdcf69dd3d2bc5ca469a2e._comment new file mode 100644 index 0000000000..07badd83a7 --- /dev/null +++ b/doc/bugs/Resource_exhausted/comment_3_747d16d050fdcf69dd3d2bc5ca469a2e._comment @@ -0,0 +1,39 @@ +[[!comment format=mdwn + username="http://adamspiers.myopenid.com/" + nickname="Adam" + subject="I just reproduced this" + date="2013-07-26T23:37:26Z" + content=""" +On openSUSE 12.3 with this version (which I'm sure is horribly old): + + git-annex version: 4.20130314 + local repository version: 3 + default repository version: 3 + supported repository versions: 3 4 + upgrade supported from repository versions: 0 1 2 + build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP DNS + +I ran 'git annex get' on a large repository, and got this: + + [snipped] + git-annex: /home/adam/music/.git/annex/transfer/failed/download/9c930baf-6b25-4294-b1f9-a90ba9663fb7/SHA1-s7227476--49d203960b062ec00c0d1d7042c4b6aa6720b976: openFile: resource exhausted (Too many open files) + failed + git-annex: runInteractiveProcess: pipe: Too many open files + + git-annex: git: runInteractiveProcess: resource exhausted (Too many open files) + failed + git-annex: get: 1207 failed + +Then I ran it again and saw that after every file retrieved, git-annex leaks another lockfile. lsof shows an ever increasing number of files like this: + + [snipped] + git-annex 32498 adam 86rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 87rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 88rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 89rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 90rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 91rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 92rR REG 253,3 0 787353 /home/adam/music/.git/annex/ssh/adam@atlantic.lock + git-annex 32498 adam 93r FIFO 0,8 0t0 28752703 pipe + git-annex 32498 adam 94r FIFO 0,8 0t0 28765910 pipe +"""]] diff --git a/doc/bugs/__34__Adding_4923_files__34___is_really_slow/comment_4_7e768908ba6983ea13af27635c4a947f._comment b/doc/bugs/__34__Adding_4923_files__34___is_really_slow/comment_4_7e768908ba6983ea13af27635c4a947f._comment new file mode 100644 index 0000000000..0971f1c342 --- /dev/null +++ b/doc/bugs/__34__Adding_4923_files__34___is_really_slow/comment_4_7e768908ba6983ea13af27635c4a947f._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.246.110" + subject="comment 4" + date="2013-07-26T22:28:37Z" + content=""" +Did some playing around, and I am seeing a progressive slowdown when adding lots of identical files in direct mode. +This does not happen when the files have different content, and thinking about it, it's pretty clear what's going on: + +When there are a ton of files with the same content, the map file that has to list all the files using this content +get larger and larger. Each file added has to read and re-write the map file, which is obviously not going to scale well to thousands of duplicate files. +"""]] diff --git a/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent/comment_1_c686df2824d3f588c0bfb339c99168b7._comment b/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent/comment_1_c686df2824d3f588c0bfb339c99168b7._comment new file mode 100644 index 0000000000..64bc18b711 --- /dev/null +++ b/doc/bugs/failure_to_find_file_that___34__should__34___exist_in_remote_is_silent/comment_1_c686df2824d3f588c0bfb339c99168b7._comment @@ -0,0 +1,29 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.152.246.110" + subject="comment 1" + date="2013-07-26T21:08:02Z" + content=""" +Tried to reproduce this, following your instructions. It did not seem to work: + +
+joey@gnu:~/tmp/mydir2>git annex get disk2 +get disk2 + Unable to access these remotes: origin + + Try making some of these repositories available: + 21606444-98f9-4c40-aa60-3ebfba936758 -- origin (joey@gnu:~/tmp/mydir) +failed +git-annex: get: 1 failed +- exit 1 +joey@gnu:~/tmp/mydir2>git annex whereis disk2 +whereis disk2 (1 copy) + 21606444-98f9-4c40-aa60-3ebfba936758 -- origin (joey@gnu:~/tmp/mydir) +ok +joey@gnu:~/tmp/mydir2> ++ +So, not a silent failure. It is a little unclear about why it can't get the file (in this case because I moved .git/annex/objects out of the way), but it's pretty clear it can't. + +AFAIK, the only way `git annex get` can silently not do anything is if you already have the file locally. Perhaps you got confused? OTOH, if you can provide a better way to reproduce this, I'm curious to see it. +"""]] diff --git a/doc/bugs/non-repos_in_repositories_list___40__+_other_weird_output__41___from_git_annex_status/comment_7_073449cc2cb73efd2b2d3d778a5573de._comment b/doc/bugs/non-repos_in_repositories_list___40__+_other_weird_output__41___from_git_annex_status/comment_7_073449cc2cb73efd2b2d3d778a5573de._comment new file mode 100644 index 0000000000..f7046a3496 --- /dev/null +++ b/doc/bugs/non-repos_in_repositories_list___40__+_other_weird_output__41___from_git_annex_status/comment_7_073449cc2cb73efd2b2d3d778a5573de._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://mey.vn/" + ip="46.65.14.106" + subject="comment 7" + date="2013-07-26T22:31:15Z" + content=""" +thanks Joey. i'm sending the output of ```git show $sha``` separately as it contains some non-public data; according to apt/dpkg logs, the rPi was actually using git-annex:armhf 4.20130521 from Debian testing at the time these commits were done. i did in fact install 4.20130709 and then 4.20130723 a couple of days after the issue showed up, but only yesterday i realised that both versions were segfaulting all the time (just invoking ```git-annex``` would segfault); 4.20130521 works reliably, instead (i have just downgraded to test), so my previous question about an interrupted operation possibly causing the metadata corruption is not valid as the git-annex executable in use at the time of the issue was not segfaulting (at least not under every circumstance as the more recent versions *in my setup* - but i'll dig into this separately - probably some library issue or so). + +the only change i can't really trace in terms of timing is when i deleted the remote i had originally set up on my rPi (using encrypted rsync) via the assistant's web interface on my laptop, and set it up again as plain unencrypted ssh remote (bare repo). i'm pretty sure this operation was done the day the commits in the ```git blame``` for ```uuid.log``` were done, but i'm not sure about the time and don't have logs going back to that moment. + +i could disconnect all the remotes from my laptop and redo the setup on the rPi to see if i can reproduce the issue, but i would need some pointers on how to revert my main annex on my laptop to the state it was in before i first started using the rPi as remote: can i just ```git reset --hard