mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2015-06-06 12:10:20 -04:00
commit 9eabbfdccf
2 changed files with 23 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
doc/forum/Speed_up_remote_transfers.mdwn Normal file
View file

@ -0,0 +1,5 @@
I am currently trying to transfer git-annex managed files to my server.
I am getting them with `git annex /path`. However, these are very many small files, and git annex seems to take a tiny break in between them, so the process takes considerably longer than uploading a single file of the same size would.
Do you have any suggestions for improving my transfer speed in this case?

18
doc/forum/Speed_up_remote_transfers/comment_1_bd97e1c47338322c5232cea7cdbc26fc._comment Normal file
View file

@ -0,0 +1,18 @@
[[!comment format=mdwn
username="CandyAngel"
subject="comment 1"
date="2015-06-06T14:57:35Z"
content="""
I did this using a couple of tools (off the top of my head):
git annex find path/ --format '${key}\n' > /tmp/keyslist (outputs a list of keys)
find .git/annex/objects -type f | grep -wFf /tmp/keyslist > /tmp/filelist (outputs a list of files)
Then I use 'resync -avhP --files-from=/tmp/filelist . othermachine:some/tmp/dir' to transfer the files to the other machine.
Then I 'git annex import some/tmp/dir' to inject the content, then just delete the additional symlinks (and reset the index).
This speeds things up a bit. Even more speed can be had using tar/netcat.
Hope this points you in the right direction.
"""]]