mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2020-12-17 16:35:10 -04:00
commit e5ef8aea9a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 78 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

30
doc/forum/rsync_into_annex_without_overwriting_links__63__/comment_3_d765f93511e0ba446f54f3ad30e91e6f._comment Normal file
View file

@ -0,0 +1,30 @@
[[!comment format=mdwn
username="kyle"
avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
subject="comment 3"
date="2020-12-17T17:41:35Z"
content="""
> Is this a case for an rsync remote? (I haven't really figured out
> special remotes yet.) Or is there a typical workflow on the git annex
> side that I could be using to fix this (like `import` rather than
> `add`)?
I think conceptually that's a good fit. You could set
`importtree=yes` with the special remote and ingest changes with `git
annex import` on beta's side. However, the rsync special remote
doesn't support `importtree` yet.
https://git-annex.branchable.com/todo/import_tree_from_rsync_special_remote/
In your followup comment, you mention unlocked files. That would get
you around the link problem. You could call `rsync` with `--checksum`
to limit what is transferred, though that might be expensive depending
on how big your files are.
> Are [the annex.hardlink and annex.thin options] annex wide settings?
> (that seems to be the case). Is it possible to apply them at a
> folder level?
You can set then at the repository level in the repo's .git/config.
"""]]

22
doc/forum/rsync_into_annex_without_overwriting_links__63__/comment_4_62cf96d62228b52bf7dd52eab0b94f67._comment Normal file
View file

@ -0,0 +1,22 @@
[[!comment format=mdwn
username="dscheffy@c203b7661ec8c1ebd53e52627c84536c5f0c9026"
nickname="dscheffy"
avatar="http://cdn.libravatar.org/avatar/62a3a0bf0e203e3746eedbe48fd13f6d"
subject="comment 4"
date="2020-12-17T18:14:16Z"
content="""
Oh boy -- or should I say, oh \"`man`\"... Now I feel like a bit of an idiot for not checking the actual high level man pages... Thanks for that tip.
I did some experimenting and noticed another thing that I hadn't noticed -- although my binary version is v6.20180227 my annexes were all using the v5 index. That came as a bit of a surprise. Once I upgraded my annex to v6 the annex.thin settings started working.
As for rsync, I had tried the `-c` (`--checksum`) option, but it wasn't dereferencing the links on the target side, so the files still registered as different (at least I think I tried this, but I may go back and check again, because I was doing a lot of different things...) Nevermind, I just checked my history and I never actually tried it -- I had been using
```
rsync -n -v -r -c --delete ai2/ beta:/media/winston/library/lectures/coursera2/ai2/
```
to try to confirm that the contents of the two folders matched so that I'd know I could safely delete my local copy, but that didn't work because of the links. I didn't add the `-L` option until I reversed the direction and ran the command from the server side with `alpha` as the target.
Thanks for all the help -- this should work well enough for my stage folder issue, but it also solves a separate problem that I'd been struggling with for making my photos available to a self hosted photo webserver tool that I was trying out (photoprism). It can't currently handle symlinks and my local drive was getting filled up with all the extra copies of my photos directory tree!
"""]]

26
doc/forum/rsync_into_annex_without_overwriting_links__63__/comment_5_40e4e9fe7d301b5e75140136d0132860._comment Normal file
View file

@ -0,0 +1,26 @@
[[!comment format=mdwn
username="kyle"
avatar="http://cdn.libravatar.org/avatar/7d6e85cde1422ad60607c87fa87c63f3"
subject="comment 5"
date="2020-12-17T19:05:12Z"
content="""
> As for rsync, I had tried the -c (--checksum) option, but it wasn't
> dereferencing the links on the target side
Just to clarify: My comment was in the context of unlocked files (in
v6+ repos). In that case, symlinks aren't used: the content is kept
in the working tree (and a pointer file is tracked by git).
Also, since it sounds like you may want all files to be unlocked, you
might want to look into `git annex adjust --unlock` to enter an
adjusted with all files in an unlocked state.
> it also solves a separate problem that I'd been struggling with for
> making my photos available to a self hosted photo webserver tool
> that I was trying out (photoprism). It can't currently handle
> symlinks [...]
FWIW, if you don't need importing for this use case, I think using
`git annex export` with an rsync special remote configured with
`exporttree=yes` would work well.
"""]]