Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2013-04-18 12:34:53 -04:00
commit fc0bc8af48
13 changed files with 196 additions and 2 deletions

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="https://launchpad.net/~maestro-alubia"
nickname="maestro-alubia"
subject="Thanks!"
date="2013-04-17T22:55:24Z"
content="""
Thank you very much for the fast fix! I will try that soon.
I love git-annex, keep up the excellent work :)
"""]]

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
nickname="Laszlo"
subject="comment 4"
date="2013-04-18T05:56:41Z"
content="""
Ok, seems like it is fixed in latest (2013.04.17) release.
Laszlo
"""]]

View file

@ -0,0 +1,43 @@
What steps will reproduce the problem?
My annex dir has 23459 files and uses 749MB disk space.
Just create a repository put this dir inside, and git-annex will crash.
What is the expected output? What do you see instead?
I expect git-annex handles large number of files, and does not watch every single file of it.
What version of git-annex are you using? On what operating system?
I'm using git-annex linux build, version 2013.04.17.
Please provide any additional information below.
[2013-04-17 23:52:35 CEST] Transferrer: Downloaded pappas_hu..di_44.jpg
git-annex: runInteractiveProcess: pipe: Too many open files
Committer crashed: lsof: createProcess: resource exhausted (Too many open files)
[2013-04-17 23:53:52 CEST] Committer: warning Committer crashed: lsof: createProcess: resource exhausted (Too many open files)
git-annex: runInteractiveProcess: pipe: Too many open files
git: createProcess: resource exhausted (Too many open files)
DaemonStatus crashed: /home/user/Desktop/down/annex_test/.git/annex/daemon.status.tmp21215: openFile: resource exhausted (Too many open files)
[2013-04-17 23:57:24 CEST] DaemonStatus: warning DaemonStatus crashed: /home/user/Desktop/down/annex_test/.git/annex/daemon.status.tmp21215: openFile: resource exhausted (Too many open files)
git-annex: runInteractiveProcess: pipe: Too many open files
git: createProcess: resource exhausted (Too many open files)
git-annex: runInteractiveProcess: pipe: Too many open files
NetWatcherFallback crashed: git: createProcess: resource exhausted (Too many open files)
[2013-04-18 00:27:17 CEST] NetWatcherFallback: warning NetWatcherFallback crashed: git: createProcess: resource exhausted (Too many open files)
git-annex: runInteractiveProcess: pipe: Too many open files
git-annex: git: createProcess: resource exhausted (Too many open files)
git-annex: accept: resource exhausted (Too many open files)
Instead of raising system's limit (which is a neverending story), can we make git-annex only watch a directory and not every file of it?
Or could the user specify some directory which he knows it is rarely change, to not be watched only check it once a day?
The best would be if git annex could automatically adapt itself.
Ie. it watches eg. 200 files, and if some of it does not change for three days, then it drops from the watching basket, and those who changed (noticed while sanity checked) it adds to the basket.
I don't really want to raise the ulimit, because my ultimate goal is to have git-annex on multiple raspberry pi with external harddrive (one at my home, one at my mom's home, one at my friends home, etc, etc). And raspberry is fairly low on resource.
I'm interested in your thoughts.
Best,
Laszlo

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
nickname="Laszlo"
subject="comment 1"
date="2013-04-18T06:20:13Z"
content="""
Once resource exhausted git-annex has git [defunct] process too.
"""]]

View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
nickname="Laszlo"
subject="comment 14"
date="2013-04-18T05:54:09Z"
content="""
Hi, these bugs (too detailed logging) and duplicated files issues are both solved in the latest (2013.04.17) release.
Thank you very much, you can close this issue.
Best,
Laszlo
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ"
nickname="Walter"
subject="comment 7"
date="2013-04-17T20:50:34Z"
content="""
So looking in .git/annex/objects, it is obvious that the difference between New1 and New1.txt is the extension.
So, I think that explains all the things I was seeing. I guess the main point of confusion (though I can't think of a better way) is the distinction between data stored in .git/annex/objects, and files in the working folder.
If I thought in terms of hashes, that is all that would need to be shown in git annex whereis is which hashes are where (and then it would be obvious if several files have the same hash, that once one of them is present somewhere then all of them are).
"""]]

View file

@ -0,0 +1,38 @@
**What steps will reproduce the problem?**
Run *git annex assistant --autostart*
git-annex autostart in /home/tobru/annex/repo1
failed
git-annex autostart in /home/tobru/annex/repo2
failed
git-annex autostart in /home/tobru/annex/repo3
failed
git-annex autostart in /home/tobru/annex/repo4
failed
git-annex autostart in /home/tobru/annex/repo5
failed
Running *git annex assistant* in each directory starts the assistant without errors.
What could cause autostart to fail? Is there any log? Or a --debug parameter?
**What is the expected output? What do you see instead?**
The assistant should start on all known repositories
**What version of git-annex are you using? On what operating system?**
4.20130417-g4bb97d5 on Ubuntu
**Please provide any additional information below.**
The ~/.config/git-annex/autostart file looks like this:
/home/tobru/annex/repo1
/home/tobru/annex/repo2
/home/tobru/annex/repo3
/home/tobru/annex/repo4
/home/tobru/annex/repo5