mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

no attachments w/o admin, copy/pasting instead

Browse source
This commit is contained in:
http://christian.amsuess.com/chrysn 2011-01-19 17:46:08 +00:00 committed by admin
parent 1881b4d7fc
commit 27325f212b
1 changed files with 21 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
doc/forum/migration_to_git-annex_and_rsync.mdwn
View file

@ -1,4 +1,4 @@
When migrating large file repositories to git-annex that are backuped in a way that uses an rsync-style mechanism (e.g. [dirvish](http://www.dirvish.org/)) and thus keeps incremental backups small by using hardlinks, space can be saved by manually reflecting the migration on the backup. So, instead of making a last pre-git-annex backup, migrating, and duplicating all backupped data with the next backup, I used the attached [[migrate.py]], and it saved me roughly a day of backuping.
When migrating large file repositories to git-annex that are backuped in a way that uses an rsync-style mechanism (e.g. [dirvish](http://www.dirvish.org/)) and thus keeps incremental backups small by using hardlinks, space can be saved by manually reflecting the migration on the backup. So, instead of making a last pre-git-annex backup, migrating, and duplicating all backupped data with the next backup, I used the <del>attached</del> migrate.py file below, and it saved me roughly a day of backuping.
A note on terminology: "migrating" here means migrating from not using git-annex at all to using it, not to the ``git annex migrate`` command, for which a similar but different solution may be created.
@ -11,3 +11,23 @@ First, have an up-to-date backup; then, git annex init / add etc as described in
Then copy the resulting migrate.sh to the equivalent location inside your backups and run it there. It will move all files that are now symlinked on the master to their new positions according to the symlinks (inside .git/annex/objects), but not create the symlinks (you will do a backup later anyway).
After that, do a backup as usual. As rsync sees the moved files at their new locations, it will accept them and not duplicate the data.
**migrate.py**:
#!/usr/bin/env python
import os
from pipes import quote
print "#!/bin/sh"
print "set -e"
print ""
for (dirpath, dirnames, filenames) in os.walk("."):
for f in filenames:
fn = os.path.join(dirpath, f)
if os.path.islink(fn):
link = os.path.normpath(os.path.join(dirpath, os.readlink(fn)))
assert link.startswith(".git/annex/objects/")
print "mkdir -p %s"%quote(os.path.dirname(link))
print "mv %s %s"%(quote(fn), quote(link))