Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2016-06-20 13:26:43 -04:00
commit 13563db2ad
Failed to extract signature
7 changed files with 126 additions and 0 deletions

View file

@ -0,0 +1,42 @@
I'm a user of git-annex for many years now (since the first stable releases) and I've always used it before on debian-like systems. I have now a gentoo OS where I have compiled git-annex.
git-annex version: 6.20160419
build flags: Assistant Webapp Pairing S3(multipartupload) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput TorrentParser MagicMime EKG Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
I don't know if you exactly can solve my problem, as it may be related to haskell stuff, and not directly coming from git-annex code, so I barely hope some hints about the possible source of the problem. When I perform any git annex action (here a git annex sync), I get the following message in the cli:
commit
Error on startup:
bind: resource busy (Address already in use)
git-annex: bind: resource busy (Address already in use)
ok
pull <ANNEX>
<etc...>
Of course, the webapp as also troubles with this. My understanding of the phenomena is that only one git annex action can be performed at the time, so the rest of them crash or fail. Is my understanding correct, do you think? I can consequently, perform git annex actions, but only one at the time, and the whole thing feels unreliable. I suspect that the problem comes from the haskell compiler or platform, but I have no knowledge of haskell and I am consequently not really able to aim at what could be the problem. If you have an idea or some hints, I would be glad to hear from you, as that bug really comes in my way. I use git-annex for every bits of my computer life and that's really a handicap for me :-(.
### Extract of .git/annex/daemon.log
[[!format sh """
[2016-06-19 17:41:57.98439] main: starting assistant version 6.20160419
[2016-06-19 17:42:04.663738] TransferScanner: Syncing with hunbaut
No known network monitor available through dbus; falling back to polling
No known volume monitor available through dbus; falling back to mtab polling
(scanning...) [2016-06-19 17:42:04.833047] Watcher: Performing startup scan
Error on startup:
bind: resource busy (Address already in use)
git-annex: bind: resource busy (Address already in use)
RemoteControl crashed: user error (nice ["ionice","-c3","/usr/bin/git-annex","remotedaemon"] exited 1)
[2016-06-19 17:42:04.945745] RemoteControl: warning RemoteControl crashed: user error (nice ["ionice","-c3","/usr/bin/git-annex","remotedaemon"] exited 1)
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yeah, for years! Thank you so much for it :-)

View file

@ -0,0 +1,23 @@
### Please describe the problem.
After unlocking a repository, the next git status will take time linear to the file size. It seems to be highly inefficient (the I/O on my SSD is not anywhere near the limit).
For a 500Mb flac album it takes ~510s, for my 100GB local music archive it took well over 30min.
### What steps will reproduce the problem?
```
git annex upgrade
git annex unlock
git status / git add -A
```
### What version of git-annex are you using? On what operating system?
6.20160527
NixOS (master branch)
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Not yet. :)
But will use it to keep my music library and sync a smaller (ogg) version to my various devices (with beets handling metadata and conversion)

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="mail@4e627fd997ef5ca9f75e62ffc0aba5b27bd6aea1"
nickname="mail"
subject="comment 1"
date="2016-06-19T16:27:15Z"
content="""
I noticed after filing the bug that the first `git commit` takes as much (or even more!) time as the `git add -A` before it.
So you pay overhead twice somehow.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="m8r-568niv@9cefd6353b1102081f43c2f5bc53afdddc153274"
nickname="m8r-568niv"
subject="comment 2"
date="2016-06-19T22:56:17Z"
content="""
Looks like I missed an obvious link/reference:
[https://git-annex.branchable.com/tips/Bup_repositories_in_git-annex/](https://git-annex.branchable.com/tips/Bup_repositories_in_git-annex/)
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="openmedi"
subject="comment 1"
date="2016-06-18T21:48:42Z"
content="""
So there are no ideas on how to proceed from anyone? Was the way I have written this message in any way too complicated or was it for some reason formulated in a manner, that made it look too demanding of others peoples time? I was not intentionally doing so, but I would appreciate any kind of commend on how to better phrase the question or on how to improve it in a way that this issue can be solved. Thanks in advance!
P.S.: Since people probably don't stumble onto this comment, I'll give it another week or so before I would open up a new post containing the same issue.
"""]]

View file

@ -0,0 +1,5 @@
It would be great to have something to call in post-update-annex which would give a list of all new and lost files given in the update.
This could be `git annex log --all --max-count=1` or somesuch.
This capability could alternatively be provided with a new post-transfer hook, called for every file.

View file

@ -0,0 +1,28 @@
[[!comment format=mdwn
username="baffo32"
subject="Use-case expansion"
date="2016-06-17T23:55:52Z"
content="""
My use-case is to store the absolute disk offsets for files, to aid recovery in case of accidental loss.
I'd like to run a script for every key added to the repo in which the script resides. The script calls filefrag to determine the physical location of the key on the disk and adds it as metadata.
If interested, my pre-commit-annex snippet is:
# requires recent e2fsprogs
if [ -x /usr/sbin/filefrag ]
then
field=\"$(git config annex.uuid)\"-extents
LC_ALL=C /usr/sbin/filefrag -ev \"$f\" | sed -n \
-e 's/.*([0-9]* blocks* of \([0-9]*\) bytes).*/!bs \1/p'\
-e 's/^ *[0-9]*: *\([0-9]*\)\.\. *[0-9]*: *\([0-9]*\)\.\. *[0-9]*: *\([0-9]*\): *.*/\1 \2 \3/p' |
while read value
do
# !bs 4096 means values are in multiples of 4096 bytes (blocksize)
# 0 5287839 5 means 5 blocks starting at block 0 of the file start at block 5287839 of the disk
# extents which are not listed are from sparse files and contain all zeros
equal='+=' addmeta \"$f\" \"$field\" \"$value\"
done
fi
"""]]