mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2022-06-09 13:18:10 -04:00
commit 9f1765b32a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 126 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

25
doc/bugs/10.20220525_cannot_probe_for_untrusted_remote_key.mdwn Normal file
View file

@ -0,0 +1,25 @@
### Please describe the problem.
10.20220525 forcefully untrusts `exporttree=yes`, causing `fsck` to unconditionally fail when probing for a key that only exists on such a remote.
The full story is at https://github.com/datalad/datalad-next/issues/72
The summary is that an application needs to know whether a particular key is on such a remote, and it can only be on that remote (there is no other). Because the availability info from `whereis` cannot be trusted (same rational as the one causing the change in git-annex implementation) it uses `annex fsck --fast --key -f ...` which now fails.
Parsing the JSON output of the failing `fsck` call seems to be the only way left to accomplish this goal.
```
% git -C myclone/.git/dl-repoannex/origin/repoannex -c diff.ignoreSubmodules=none annex fsck -f origin --fast --key XDLRA--refs -c annex.dotfiles=true --json
Only these untrusted locations may have copies of XDLRA--refs
d8735795-663e-4702-8f7e-8c684264a9df -- [origin]
Back it up to trusted locations with git-annex copy.
{"dead":[],"command":"fsck","note":"(Avoid this check by running: git annex dead --key )","success":false,"input":[],"untrusted":[{"here":false,"uuid":"d8735795-663e-4702-8f7e-8c684264a9df","description":"[origin]"}],"key":"XDLRA--refs","error-messages":[],"file":null}
fsck: 1 failed
```
That however, is substantially more complex than looking at the exit code -- given that the entire call is about a single key.
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
99.9% are just splendid. Wondering here whether this particular consequence of the change was intentional and avoidable?

88
doc/bugs/error_out_on_file_size_0_from_external_spec_remote.mdwn Normal file
View file

@ -0,0 +1,88 @@
### Please describe the problem.
Here is a reproducer using the place where I encountered it -- using git-annex-remote-rclone (tiny bash script; needs fixing though for recent rclone -- take from [my fork where I am tuning testing etc](https://github.com/yarikoptic/git-annex-remote-rclone/blob/enh-tests/git-annex-remote-rclone)) [https://github.com/DanielDent/git-annex-remote-rclone/](https://github.com/DanielDent/git-annex-remote-rclone/)
<details>
<summary>this is a reproducer which creates repo and commits to annex of size 0</summary>
```bash
#!/bin/bash
cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"
set -eux
# provide versioning information to possibly ease troubleshooting
git annex version
rclone --version
export HOME=$PWD
echo -e '[local]\ntype = local\nnounc =' > ~/.rclone.conf
# to pacify git/git-annex
git config --global user.name Me
git config --global user.email me@example.com
git config --global init.defaultBranch master
# Prepare rclone remote local store
mkdir rclone-local
export RCLONE_PREFIX=$PWD/rclone-local
git-annex version
mkdir testrepo
cd testrepo
git init .
git-annex init
git-annex initremote GA-rclone-CI type=external externaltype=rclone target=local prefix=$RCLONE_PREFIX chunk=100MiB encryption=shared mac=HMACSHA512
# Rudimentary test, spaces in the filename must be ok, 0 length files should be ok
touch "test 0"
#echo 1 > "test 1"
git-annex add *
git-annex copy * --to GA-rclone-CI
git-annex drop *
git-annex get --debug *
```
</details>
where it ends with
```
+ git-annex get --debug 'test 0'
[2022-06-09 13:16:20.668338168] (Utility.Process) process [1022440] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","test 0"]
[2022-06-09 13:16:20.668784836] (Utility.Process) process [1022441] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-06-09 13:16:20.669111286] (Utility.Process) process [1022442] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
[2022-06-09 13:16:20.66965775] (Utility.Process) process [1022443] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
[2022-06-09 13:16:20.670318546] (Utility.Process) process [1022443] done ExitSuccess
[2022-06-09 13:16:20.67074703] (Utility.Process) process [1022444] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
[2022-06-09 13:16:20.671504449] (Utility.Process) process [1022444] done ExitSuccess
[2022-06-09 13:16:20.672125442] (Utility.Process) process [1022445] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..0da4aa78e4397feb9906400d1e08aaebccac1be1","--pretty=%H","-n1"]
[2022-06-09 13:16:20.673019668] (Utility.Process) process [1022445] done ExitSuccess
[2022-06-09 13:16:20.673440353] (Utility.Process) process [1022446] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
get test 0 [2022-06-09 13:16:20.674594171] (Utility.Process) process [1022448] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
(from GA-rclone-CI...)
[2022-06-09 13:16:20.682205613] (Annex.Perms) freezing content directory .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
git-annex: .git/annex/tmp/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: rename: does not exist (No such file or directory)
failed
[2022-06-09 13:16:20.682296739] (Utility.Process) process [1022446] done ExitSuccess
[2022-06-09 13:16:20.682333234] (Utility.Process) process [1022442] done ExitSuccess
[2022-06-09 13:16:20.682384256] (Utility.Process) process [1022441] done ExitSuccess
[2022-06-09 13:16:20.682408227] (Utility.Process) process [1022440] done ExitSuccess
[2022-06-09 13:16:20.682701178] (Utility.Process) process [1022448] done ExitSuccess
get: 1 failed
```
- you can see that there is not even an attempt to talk to the remote. git-annex crashes before then with that " rename: does not exist (No such file or directory)"
so I do not think it is an issue of the remote (which might also have some size 0 issues from looking at some conditions)
### What version of git-annex are you using? On what operating system?
tried both 8.20211123 and 10.20220504 from debian

13
doc/bugs/initremote_type__61__git_is_not_working_for_unkn_reason/comment_2_d51bb87dc5ed0f4ede808c5a9594240d._comment Normal file
View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="jkniiv"
avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d"
subject="comment 2"
date="2022-06-09T02:32:18Z"
content="""
Wouldn't it be possible to support (absolute) file:// urls, eg. something similar to
`file:///home/jkniiv/test-VEfBrTZ/remote2`? In my mind they feel like a reasonable approximation
of ssh:// urls and could be useful for getting a feel for git special remotes before setting
up a bare git-repo/annex on an ssh-server. I know they are not the same thing implementation wise
but I feel that being able to try this feature out on a least-effort basis would be useful
from a pedagogical standpoint.
"""]]