mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2024-09-05 15:25:34 -04:00
commit d717e9aca0
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 204 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

64
doc/bugs/assistant___40__webapp__41___commited_unlocked_link_to_annex/comment_10_ae87af8282fcc4a06c0b703b3c1f8710._comment Normal file
View file

@ -0,0 +1,64 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 10"
date="2024-09-05T14:52:51Z"
content="""
but may be it is actually a separate issue of the unlocked mode since it does drop the file
```
reprostim@reproiner:/data/reprostim$ find .git/annex -iname *377.mkv
.git/annex/objects/Qp/XF/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
.git/annex/objects/Qp/XF/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
reprostim@reproiner:/data/reprostim$ git annex drop Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
drop Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv (locking rolando...) ok
(recording state in git...)
reprostim@reproiner:/data/reprostim$ find .git/annex -iname *377.mkv
reprostim@reproiner:/data/reprostim$ cat Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
/annex/objects/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
```
but then when I get it, it does not actually copy into the tree:
```
reprostim@reproiner:/data/reprostim$ git annex get --json Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
{\"command\":\"get\",\"error-messages\":[],\"file\":\"Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv\",\"input\":[\"Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv\"],\"key\":\"MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv\",\"note\":\"from rolando...\",\"success\":true}
reprostim@reproiner:/data/reprostim$ cat Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
/annex/objects/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
reprostim@reproiner:/data/reprostim$ find .git/annex -iname *377.mkv
.git/annex/objects/Qp/XF/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
.git/annex/objects/Qp/XF/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
```
```
reprostim@reproiner:/data/reprostim$ cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[annex]
uuid = 9806a90e-4cdd-48cb-b03d-7a113663fce7
version = 10
addunlocked = false
[filter \"annex\"]
smudge = git-annex smudge -- %f
clean = git-annex smudge --clean -- %f
process = git-annex filter-process
[remote \"rolando\"]
url = bids@rolando.cns.dartmouth.edu:VIDS/
fetch = +refs/heads/*:refs/remotes/rolando/*
annex-uuid = 285d851e-77a8-4d31-b24c-fa72deb4d3cc
[branch \"master\"]
remote = rolando
merge = refs/heads/master
reprostim@reproiner:/data/reprostim$ git annex version
git-annex version: 10.20240831-1~ndall+1
```
"""]]

23
doc/bugs/assistant___40__webapp__41___commited_unlocked_link_to_annex/comment_9_f3ac5f4aab89893b88fb30cced56827b._comment Normal file
View file

@ -0,0 +1,23 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="ping on this issue : how to recover?"
date="2024-09-05T14:49:06Z"
content="""
I got back to this issue, since even after upgrade of git-annex to `10.20240831-1~ndall+1` and trying on a sample file which I guess was screwed up
```
reprostim@reproiner:/data/reprostim$ git annex get --json Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
reprostim@reproiner:/data/reprostim$ git annex find --in here
Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
reprostim@reproiner:/data/reprostim$ ls -lL Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
-rw-r--r-- 2 reprostim reprostim 72 Sep 5 10:42 Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
reprostim@reproiner:/data/reprostim$ cat Videos/2024/08/2024.08.30-11.31.56.000--2024.08.30-11.48.03.377.mkv
/annex/objects/MD5E-s20610854--4fa8311cf5fc0ea247dca2b0ae556bab.377.mkv
```
so, I need to figure out how to actually get that key/file here.
"""]]

53
doc/bugs/git_annex_sync_borg_fails/comment_5_7929e8cdc6008bb41ee38a7be42cddf6._comment Normal file
View file

@ -0,0 +1,53 @@
[[!comment format=mdwn
username="Rick"
avatar="http://cdn.libravatar.org/avatar/bbc227c89f7136fbb191127764e9d02c"
subject="Similar Borg sync issue"
date="2024-09-03T19:40:57Z"
content="""
I'm also getting `list borg failed` when I run `git annex sync borg`. In my case, syncing succeeds after creating the first borg archive but fails when the borg repo contains a second archive.
I'm running:
- git-annex 10.20240731
- borg 1.4.0
- NixOS 24.11.20240821.c374d94 (Vicuna)
To reproduce this problem:
```
borg init --encryption=keyfile /path/to/borgrepo
git annex initremote borg type=borg borgrepo=/path/to/borgrepo
borg create /path/to/borgrepo::archive1 `pwd`
git annex sync borg
git annex add newfile
borg create /path/to/borgrepo::archive2 `pwd`
git annex sync borg
```
From the debug output the first time running `git-annex sync`, the only ExitFailure line:
```
[2024-08-28 19:13:31.056388087] (Utility.Process) process [79595] done ExitFailure 1
ok
````
And the first appearance of process 79595:
```
[2024-08-28 19:13:31.011783181] (utility.process) process [79595] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit\",\"-a\",\"-m\",\"git-annex in user@nixos:~/sandbox/gr\"]
```
Only once, after running the command a second time, I got the following additional lines:
```
[2024-08-28 19:48:41.942245332] (Utility.Process) process [122585] read: borg [\"list\",\"--format\",\"{size}{NUL}{path}{NUL}{extra}{NUL}\",\"/home/user/sandbox/br::archive2\",\"\"]
...
borg list: error: argument PATH: Empty strings are not accepted as paths.
[2024-08-28 19:48:42.296294751] (Utility.Process) process [122585] done ExitFailure 2
```
I have set `LANG=C` and `git annex enableremote borg subdir=` as suggested in this thread to no avail.
Thanks in advance for your help! I have used and loved git-annex for years and am very thankful for the work Joey and others have put into it. I'm planning to buy a git-annex backpack soon.
"""]]

56
doc/git-annex-fsck/comment_1_80ec8617d99d3f520c22b1e7fd741c16._comment Normal file
View file

@ -0,0 +1,56 @@
[[!comment format=mdwn
username="tapesafer"
avatar="http://cdn.libravatar.org/avatar/8a62b25ea58309a6e15cac10a5c33f1d"
subject="numcopies & force-trusting is ignored by fsck on readonly directory remotes?"
date="2024-09-04T14:50:16Z"
content="""
I have old readonly backup media, say something like
- `tapeA1/apples.txt`
- `tapeA2/apples.txt`
- `tapeB1/earth.svg`
- `tapeB2/earth.svg`
I use git-annex special directory remotes to be able to navigate the directory tree that lives on those media (e.g. to decide if and which media I need to find to copy a file from that I need).
I added the remotes like so (they are too big to import with content):
```
git annex initremote tapeA1 type=directory directory=/tapes/tapeA1 encryption=none importtree=yes
git annex import master:tapeA1 --from tapeA1 --no-content
git annex merge --allow-unrelated-histories tapeA1/main
```
At some point I may buy new hardware and recreate those backup media as proper git-annex remotes, but wouldn't it be great to keep the existing backups as long as they show no sign of bitrot and together hold enough copies?
Though, git-annex fsck behaves unexpected: It seems I cannot force trust these remotes nor does `--numcopies=0 --mincopies=0` have the desired effect.
Concretely, when calling `git annex fsck --from=tapeA1 --numcopies=0 --mincopies=0 --trust=tapeA1 --force`,
for every file that is still intact on tapeA1, git-annex fsck reports a failure as follows
```
fsck tapeA1/apples.txt
Only these untrusted locations may have copies of tapeA1/apples.txt
abc-def-ghi -- [tapeA1]
Back it up to trusted locations with git-annex copy.
failed
```
while I'd be happy to (semi)trust tapeA1 or to accept no copies whatsoever. So fsck ignores `--trust=tapeA1 --force` and/or `--numcopies=0 --mincopies=0` which are common git-annex options that should work for fsck?
Ideally, I would be able to (semi)trust my readonly tape remotes (which likely should be behind a `--force` as it may lead to data loss in classical directory remote settings). Then I can use git-annex to index those tapes, but also to monitor their health via fsck (so I can over the years replace the tapes that are showing signs of corruption).
As for the corruption, I emulated bitrot on a test directory remote, which then leads to a fsck failure as follows:
```
fsck tapeB2/earth.svg
verification of content failed
(checksum...)
tapeB2/earth.svg: Bad file content; failed to drop fromtapeB2: dropping content from this remote is not supported because it is configured with importtree=yes
```
This suffices to detect tapes that should be replaced, and it's kinda expected that files cannot be dropped.
Somehow fsck does not work as I would expect -- am I misunderstanding the numcopies/mincopies arguments here? Is there really no way to force-trust a directory remote, which to me seems appropriate in this case? Is there another way to achieve what I have in mind with git-annex?
Thanks for this great piece of software – also use the assistant in another day-to-day usecase and it's simply great!
"""]]

8
doc/git-annex-fsck/comment_2_44ca99f975e75d0236b8ae6be4d23f72._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="tapesafer"
avatar="http://cdn.libravatar.org/avatar/8a62b25ea58309a6e15cac10a5c33f1d"
subject="PS"
date="2024-09-04T15:48:01Z"
content="""
If I am understanding the documentation of the borg special remote, then having something like `appendonly=yes` for the special directory remote would likely help in my scenario.
"""]]