mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2023-01-04 17:34:29 -04:00
commit 946fc20165
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
6 changed files with 166 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

123
doc/bugs/Paths_behind_relative_symlinks_in_repo_don__39__t_work.mdwn Normal file
View file

@ -0,0 +1,123 @@
### Please describe the problem.
Except for `git annex info` many (all?) other git-annex file-based commands seem to have a problem with paths behind relative symlinks to *directories* in the repository. Using the 'full' tracked-by-git or absolute path works. I guess git-annex relies on git for determining if a file is tracked and git itself fails to 'see' files behind relative symlinks to dirs in the repo as well.
### What steps will reproduce the problem?
Run this script:
[[!format sh """
#!/bin/sh
export LC_ALL=C
chmod +w -R git-annex-link
rm -rf git-annex-link
mkdir git-annex-link
cd git-annex-link
(
git init
git config commit.gpgsign false
git annex init
# make folder structure with some files
mkdir -p level1/level2
for f in level1/file level1/level2/file;do echo "$f" > "$f";done
ln -rsf level1 link-to-level1
ln -rsf level1/level2 link-to-level2
git annex add --force-large
git commit -m "Add files"
) >/dev/null 2>/dev/null
echo;(set -x;tree);echo;echo
echo "Test results:"
for cmd in info metadata get lookupkey whereis list fix;do
for file in level1/file level1/level2/file link-to-level1/file link-to-level2/file;do
printf "git annex %10s %20s %s\n" $cmd $file "$(git annex $cmd $file 2>/dev/null >/dev/null && echo -n ok || echo -n fail)"
done
done
echo;echo "The error is: "
(set -x;git annex get link-to-level1/file)
"""]]
Output:
```
+ tree
.
|-- level1
| |-- file -> ../.git/annex/objects/P7/z4/SHA256E-s12--17313e162321955543c4644ea8e1accc8d3039a9e0a9224429f26bafafe0d1b6/SHA256E-s12--17313e162321955543c4644ea8e1accc8d3039a9e0a9224429f26bafafe0d1b6
| `-- level2
| `-- file -> ../../.git/annex/objects/24/K3/SHA256E-s19--633195e34fd1e133557ece8767e268a0ebd9e4caedad4776bab4470c60a84439/SHA256E-s19--633195e34fd1e133557ece8767e268a0ebd9e4caedad4776bab4470c60a84439
|-- link-to-level1 -> level1
`-- link-to-level2 -> level1/level2
4 directories, 2 files
Test results:
git annex info level1/file ok
git annex info level1/level2/file ok
git annex info link-to-level1/file ok
git annex info link-to-level2/file ok
git annex metadata level1/file ok
git annex metadata level1/level2/file ok
git annex metadata link-to-level1/file fail
git annex metadata link-to-level2/file fail
git annex get level1/file ok
git annex get level1/level2/file ok
git annex get link-to-level1/file fail
git annex get link-to-level2/file fail
git annex lookupkey level1/file ok
git annex lookupkey level1/level2/file ok
git annex lookupkey link-to-level1/file fail
git annex lookupkey link-to-level2/file fail
git annex whereis level1/file ok
git annex whereis level1/level2/file ok
git annex whereis link-to-level1/file fail
git annex whereis link-to-level2/file fail
git annex list level1/file ok
git annex list level1/level2/file ok
git annex list link-to-level1/file fail
git annex list link-to-level2/file fail
git annex fix level1/file ok
git annex fix level1/level2/file ok
git annex fix link-to-level1/file fail
git annex fix link-to-level2/file fail
The error is:
+ git annex get link-to-level1/file
error: pathspec 'link-to-level1/file' did not match any file(s) known to git
Did you forget to 'git add'?
get: 1 failed
```
### What version of git-annex are you using? On what operating system?
```
❯ inxi
CPU: 8-core AMD Ryzen 7 5800H with Radeon Graphics (-MT MCP-)
speed/min/max: 1450/1200/4462 MHz Kernel: 6.1.1-1-MANJARO x86_64 Up: 4h 23m
Mem: 4370.6/13832.0 MiB (31.6%) Storage: 953.87 GiB (40.9% used) Procs: 376
Shell: fish inxi: 3.3.24
❯ git annex version
git-annex version: 10.20221212-gab11fd70e
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Benchmark Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22.1 bloomfilter-2.0.1.0 cryptonite-0.30 DAV-1.3.4 feed-1.3.2.1 ghc-9.0.2 http-client-0.7.13.1 persistent-sqlite-2.13.1.0 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.2.1
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
```
### Please provide any additional information below.
I came across this while developing my [`thunar-plugins`](https://pypi.org/project/thunar-plugins/1.0.0/) Python package for git-annex integration in XFCE's Thunar file manager. It is easily [worked around there by resolving all links before handing them to git-annex](https://gitlab.com/nobodyinperson/thunar-plugins/-/blob/de5d273106b79a5f397962c2fb3124943bdd03f7/thunar_plugins/menus/git_annex.py#L133-144). However having git-annex handle this situation natively would be nice, e.g. by also first resolving paths it gets handed.
One motivation for such relative symlinks to directories somewhere in the repository is to make **shortcuts** to some deeply nested folders.
Links to *files* (not directories) get converted to links to the annex, so not a problem there.
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes, `git-annex` is absolutely awesome, 🎉 'finding' it was one of my highlights of 2022 😉!

8
doc/bugs/Paths_behind_relative_symlinks_in_repo_don__39__t_work/comment_1_d7ca8767953e1ec06740727a0eda4697._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="re: Paths behind relative symlinks in repo"
date="2023-01-03T18:28:22Z"
content="""
I'll second this request. See also [[todo/symlinks_to_symlinks_to_the_annex]] and [[todo/import_symlinks_when_importing_from_directory]]. Basic reason for wanting this is uniformity: it would help if operations worked the same way on symlinks as they do on originals.
"""]]

5
doc/bugs/should_create_leading_dirs_for_annex.dbdir.mdwn
View file

@ -66,5 +66,8 @@ Already on 'master'
Your branch is up to date with 'origin/master'.
```
NB IIRC: originally came up within reproin-ification effort on rolando and its NFS mounted partition.
[[!meta author=yoh]]
[[!tag projects/datalad]]
[[!tag projects/repronim]]

9
doc/git-annex-addurl/comment_13_f413cc1febfa70e3ad885c6fa031a209._comment Normal file
View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="matthias.risze@9f2c8f7faed4cac1905d1bf1ee4524d708c13688"
nickname="matthias.risze"
avatar="http://cdn.libravatar.org/avatar/c9f7f022a1d62c39497b72c56a6a535e"
subject="Limitations on allowed characters in and length of URLs"
date="2023-01-04T11:56:35Z"
content="""
Are there any limitations on which characters are allowed in an URL, and on how long an URL can be? I am working on a special remote which kind of abuses this feature by saving information about how to get the file in the URL with a specific scheme. For now it seems like I can use URLs of the form `scheme:<arbitrary json>`, but I am not sure if this might become an issue later. I could also encode the data with base64 or something similar, in which case size limitations would still be relevant; if there are any. Although, the json variant has the added benefit of being much more easily readable in whereis output.
"""]]

14
doc/todo/encrypted_keys_in_git_repository/comment_5_05b0e2e88b233691b403d4c908b50c52._comment Normal file
View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="nobodyinperson"
avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5"
subject="comment 5"
date="2023-01-02T22:58:08Z"
content="""
> I'm talking about a single file that the attacker already knows the content of, and wishes to determine if it's present in the repository.
Ah alright, in that case scrypt doesn't help, I agree. Among the large userbase for such a versatile software as git-annex there surely are people that could make use of hiding the presence of some 'common' files from their public repos. Though it rather seems like quite an edge case to me 😅 Better go full-gcrypt in that case I'd say...
> sha2 is equally good protection as scrypt
In that case, this'll be enough for my application, thanks! 🙏
"""]]

8
doc/todo/reuse_desktop_notifications/comment_1_6193ed5a4ad0a908af82da33d14bd654._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="nobodyinperson"
avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5"
subject="comment 1"
date="2023-01-02T21:58:25Z"
content="""
I'd consider a single 'persistent' notification for git-annex sufficient. When a transfer starts/finishes the notification is updated. Maybe show a configurable amount of last items (`git config annex.notification-max-items` with a default of 5 or so). `-J` Worker threads would just add to that notification, I don't see a need for one notification per worker thread (that can easily go up to another 16 with `--jobs=cpus` on modern CPUs, filling the screen again 🙃).
"""]]