Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
9574e3a8bb
15 changed files with 383 additions and 2 deletions
|
|
@ -0,0 +1,9 @@
|
|||
[[!comment format=mdwn
|
||||
username="sng@353ca358075d9aa328f60a5439a3cee10f8301fe"
|
||||
nickname="sng"
|
||||
avatar="http://cdn.libravatar.org/avatar/d64f4854965b2b1c3ecafee4b2a66fac"
|
||||
subject="comment 1"
|
||||
date="2024-10-06T21:42:13Z"
|
||||
content="""
|
||||
Have the same issue
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Different git-annex operations in a repository initialized with `git init --object-format=sha256` produce an error message of the form `fatal: ambiguous argument '4b825dc642cb6eb9a060e54bf8d69288fbee4904': unknown revision or path not in the working tree.`.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
See transcript below, which produces the error message on `git annex add` and `git annex get`.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
```
|
||||
git-annex version: 10.20240927-g3d7f94ea398b5e84dab3bc89bc5b37746de1d40c
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.24.1 bloomfilter-2.0.1.2 crypton-0.33 DAV-1.3.4 feed-1.3.2.1 ghc-9.4.7 http-client-0.7.14 persistent-sqlite-2.13.2.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 10
|
||||
```
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
$ mkdir test-sha256
|
||||
$ cd test-sha256
|
||||
$ git init --object-format=sha256
|
||||
Initialized empty Git repository in /home/icg149/Playground/test-sha256/.git/
|
||||
$ git annex init
|
||||
init ok
|
||||
(recording state in git...)
|
||||
$ echo test123 > 123.txt
|
||||
$ git annex add 123.txt
|
||||
add 123.txt
|
||||
100% 8 B 27 KiB/s 0sfatal: ambiguous argument '4b825dc642cb6eb9a060e54bf8d69288fbee4904': unknown revision or path not in the working tree.
|
||||
Use '--' to separate paths from revisions, like this:
|
||||
'git <command> [<revision>...] -- [<file>...]'
|
||||
ok
|
||||
(recording state in git...)
|
||||
$ git commit -m "Add file"
|
||||
[main (root-commit) 32fdb51] Add file
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 123.txt
|
||||
$ ls -l
|
||||
total 4
|
||||
lrwxrwxrwx 1 icg149 icg149 186 Okt 8 09:10 123.txt -> .git/annex/objects/05/GP/SHA256E-s8--9572d7f4e812df12cd8c0d26e7308864c33cbb51b004cbe962ad545bc377a4a2.txt/SHA256E-s8--9572d7f4e812df12cd8c0d26e7308864c33cbb51b004cbe962ad545bc377a4a2.txt
|
||||
$ cd ..
|
||||
$ git clone test-sha256/ test-sha256-clone
|
||||
Cloning into 'test-sha256-clone'...
|
||||
done.
|
||||
$ cd test-sha256-clone
|
||||
$ git annex get .
|
||||
get 123.txt (from origin...)
|
||||
fatal: ambiguous argument '4b825dc642cb6eb9a060e54bf8d69288fbee4904': unknown revision or path not in the working tree.
|
||||
Use '--' to separate paths from revisions, like this:
|
||||
'git <command> [<revision>...] -- [<file>...]'
|
||||
ok
|
||||
(recording state in git...)
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
15
doc/bugs/git_remote_annex_-_chunking_unsupported.mdwn
Normal file
15
doc/bugs/git_remote_annex_-_chunking_unsupported.mdwn
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
[[!format text """
|
||||
git-annex version: 10.20240927
|
||||
build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.24.2 bloomfilter-2.0.1.2 crypton-1.0.0 DAV-1.3.4 feed-1.3.2.1 ghc-9.8.2 http-client-0.7.17 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.16 yesod-1.6.2.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external
|
||||
operating system: darwin aarch64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 10
|
||||
"""]]
|
||||
|
||||
Chunking of bundles/the manifest when using `git-remote-annex` does not work. Particularly for a `directory` remote set up with chunking enabled, it appears the manifest is populated with the *unchunked* filename and so the helper reports there being no repository on fetch because it can't find the bundles it needs. Push works, but not fetch/pull.
|
||||
|
||||
Was able to verify that after removing the `SXXX-CX` portion of the chunked file/folder names (thankfully in my case the size of my data hadn't reached the chunking threshold yet) the repo was once again recognized.
|
||||
48
doc/bugs/git_remote_annex_-_rclone.mdwn
Normal file
48
doc/bugs/git_remote_annex_-_rclone.mdwn
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
Related to [git-remote-annex](https://git-annex.branchable.com/tips/storing_a_git_repository_on_any_special_remote/) development.
|
||||
|
||||
Special Remote type: [rclone (dropbox)](https://git-annex.branchable.com/special_remotes/rclone/)
|
||||
|
||||
Implementation (brew, OSX):
|
||||
|
||||
[[!format txt """
|
||||
git-annex version: 10.20240927
|
||||
build flags: Assistant Webapp Pairing FsEvents TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.24.2 bloomfilter-2.0.1.2 crypton-1.0.0 DAV-1.3.4 feed-1.3.2.1 ghc-9.8.2 http-client-0.7.17 persistent-sqlite-2.13.3.0 torrent-10000.1.3 uuid-1.3.16 yesod-1.6.2.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external
|
||||
operating system: darwin aarch64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 10
|
||||
"""]]
|
||||
|
||||
# Protocol Error on Git Remote Annex
|
||||
|
||||
My rclone remote is producing the following error on attempting to treat the special remote as a git remote
|
||||
|
||||
[[!format txt """
|
||||
Full remote url: annex::1eb4e389-9aea-4968-a947-498fd52645e0?chunk=10MiB&encryption=none&type=rclone
|
||||
external special remote protocol error, unexpectedly received "CHECKPRESENT-FAILURE GITMANIFEST--1eb4e389-9aea-4968-a947-498fd52645e0" (command not allowed at this time)
|
||||
git-annex: unable to use special remote due to protocol error
|
||||
"""]]
|
||||
|
||||
This might be a fairly innocuous bug relating to the order of steps the helper takes to perform checks of the remote. However, it does raise some interesting questions about the compatibility of the helper with more complex remotes like rclone:
|
||||
|
||||
- Can the helper support non-`directory` rclonelayouts? (`nodir` or `lower` in particular)?
|
||||
- Are the remaining rclone options not shown in the URL (and neither shown by e.g. `git annex info`) inherited from the `remote.log` or part of the bug here?
|
||||
- Could spaces in the `rcloneprefix` (path) affect the URL specification?
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
1. Init an rclone remote using --with-url (technically I just added annex:: as url to an existing remote)
|
||||
1. Try simple operations like `git remote show <rclone remote>`
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
I'm slowly integrating it into everything I do because it's so cool. Right now, I'm trying to solve the problem of keeping personal info private and thus not uploading git repos to an insecure place like github.com. For now I'm exploring using cloud storages - I use Dropbox almost exclusively - to store the repos. I've tried:
|
||||
|
||||
- [`git-remote-rclone`](https://github.com/datalad/git-remote-rclone) (Datalad's version or Redstreet's `tar.gz` implementation). Syncing is a little fragile (may need to delete `.git/rclone` to refresh corrupt syncs, but also irreparably breaks if you do any forgetting on the annex branch or history rewriting (new tip fails to contain commits the helper expects and there's no force push option)
|
||||
- [`git-remote-dropbox`](https://github.com/anishathalye/git-remote-dropbox) Incredibly slow for having to upload each and every object individually and the upload API for dropbox is slow.
|
||||
- git-remote-annex - Seems incredibly promising despite issues I've had getting it to install (see below) let alone work quite yet
|
||||
|
||||
Also, if anyone is having issue using git-remote-annex, like I did running on OSX installed via homebrew, the solution is to add a symlink manually from (e.g.) `$(realpath /opt/homebrew/bin/git-annex[-shell]) <- /opt/homebrew/bin/git-remote-annex`. In other words, the current homebrew recipe simply doesn't symlink the git-annex command to git-remote-annex on your path, which git annex automagically act like a git remote helper if called like this so luckily it's an easy fix.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
### Please describe the problem.
|
||||
|
||||
The checkpresent endpoint (and also less importantly the gettimestamp endpoint) of the p2phttp protocol is specified to use POST, when what it does is really a GET request for the presence information without any mutation of data warranting a POST. In the dumb http realm this function is served by a HEAD request for the file, which is conceptually also a GET, just without the actual data.
|
||||
|
||||
In forgejo-aneksajo there is a blanket authentication requiring read permissions for all GETs and write/admin permissions for all POSTs (or anything other than GET, really), so right now I have to special case checkpresent while it would just do the right thing if checkpresent was a GET.
|
||||
|
||||
This isn't really a problem, just a bit of an annoyance.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
### Please describe the problem.
|
||||
|
||||
With `p2phttp --wideopen`, a `git annex drop` will lock content on the remote before dropping. With `p2phttp --unauth-readonly` `git annex drop` will instead be satisfied with a "RecentlyVerifiedCopy". This is an issue for forgejo-aneksajo, as it does its own authentication before handing over to `p2phttp --wideopen`, at which point a drop will try to lock the file on the remote but authentication will fail. Instead, it should fallback to the "recently verified is enough" behavior of unauth-readonly (and dumb http).
|
||||
|
||||
Sorry for the rather unuseful title, the character limit makes coming up with a good summary hard.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
- serve a repository with `git annex --debug p2phttp --wideopen`
|
||||
- get and drop a file in a clone
|
||||
- observe file locking
|
||||
- do the same with `git annex --debug p2phttp --unauth-readonly`
|
||||
- do not observe file locking
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
```
|
||||
git-annex version: 10.20240927-g3d7f94ea398b5e84dab3bc89bc5b37746de1d40c
|
||||
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Servant Benchmark Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.24.1 bloomfilter-2.0.1.2 crypton-0.33 DAV-1.3.4 feed-1.3.2.1 ghc-9.4.7 http-client-0.7.14 persistent-sqlite-2.13.2.0 torrent-10000.1.3 uuid-1.3.15 yesod-1.6.2.1
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL GITBUNDLE GITMANIFEST VURL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg rclone hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 10
|
||||
```
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="matrss"
|
||||
avatar="http://cdn.libravatar.org/avatar/59541f50d845e5f81aff06e88a38b9de"
|
||||
subject="comment 1"
|
||||
date="2024-10-07T14:12:23Z"
|
||||
content="""
|
||||
This bug report might have been a bit prematurely, I just noticed that p2phttp returns a 403 for the file locking endpoints in the case of unauth-readonly, while my forgejo-aneksajo implementation returns a 401. This seems to make git-annex prompt for a password instead of falling back to just checking presence.
|
||||
"""]]
|
||||
88
doc/bugs/tries_to_download_a_.mkv_video_without_yt-dlp.mdwn
Normal file
88
doc/bugs/tries_to_download_a_.mkv_video_without_yt-dlp.mdwn
Normal file
|
|
@ -0,0 +1,88 @@
|
|||
### Please describe the problem.
|
||||
|
||||
For some not obvious to me reason, git-annex used `yt-dlp` fine for many files but then does not use for another, very odd:
|
||||
|
||||
here is two files from (pushed all to http://github.com/TrueTube/Andriy_Popik)
|
||||
|
||||
```shell
|
||||
(git)smaug:/tmp/Andriy_Popyk/Andriy_Popyk[master]git
|
||||
$> git annex whereis Чат_рулетка/2024-09-30-Зеки_-_основа_россии._Андрій_Попик.mkv Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
|
||||
whereis "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-09-30-\320\227\320\265\320\272\320\270_-_\320\276\321\201\320\275\320\276\320\262\320\260_\321\200\320\276\321\201\321\201\320\270\320\270._\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (2 copies)
|
||||
00000000-0000-0000-0000-000000000001 -- web
|
||||
05689e10-d659-4239-a614-a6de95b11208 -- yoh@smaug:/mnt/btrfs/datasets/datalad/crawl-misc/TrueTube/popik
|
||||
|
||||
web: https://www.youtube.com/watch?v=Ikn4LnYDcKU
|
||||
ok
|
||||
whereis "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (1 copy)
|
||||
00000000-0000-0000-0000-000000000001 -- web
|
||||
|
||||
web: https://www.youtube.com/watch?v=5rFP3DlC2MU
|
||||
ok
|
||||
|
||||
```
|
||||
|
||||
for one (`$> git annex get --debug Чат_рулетка/2024-09-30-Зеки_-_основа_россии._Андрій_Попик.mkv`) works fine -- starts downloading using yt-dlp. But for the other:
|
||||
|
||||
```
|
||||
$> git annex get --debug Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
|
||||
[2024-10-01 17:00:12.822269342] (Utility.Process) process [2784960] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
|
||||
[2024-10-01 17:00:12.827363166] (Utility.Process) process [2784960] done ExitSuccess
|
||||
[2024-10-01 17:00:12.828029045] (Utility.Process) process [2784961] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
|
||||
[2024-10-01 17:00:12.832174221] (Utility.Process) process [2784961] done ExitSuccess
|
||||
[2024-10-01 17:00:12.832951895] (Utility.Process) process [2784962] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..1c3a3eacf78934c71cdf560061207cdb13422fd0","--pretty=%H","-n1"]
|
||||
[2024-10-01 17:00:12.835645728] (Utility.Process) process [2784962] done ExitSuccess
|
||||
[2024-10-01 17:00:12.840540312] (Utility.Process) process [2784963] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
|
||||
[2024-10-01 17:00:12.855549135] (Utility.Process) process [2784964] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--error-unmatch","--","\1063\1072\1090_\1088\1091\1083\1077\1090\1082\1072/2024-10-01-\1056\1072\1089\1089\1077\1103\1085\1082\1072__\1040\1085\1076\1088\1110\1081_\1055\1086\1087\1080\1082.mkv"]
|
||||
[2024-10-01 17:00:12.856270222] (Utility.Process) process [2784965] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
[2024-10-01 17:00:12.85694905] (Utility.Process) process [2784966] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
[2024-10-01 17:00:12.857415068] (Utility.Process) process [2784967] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
[2024-10-01 17:00:12.864931666] (Utility.Process) process [2784967] done ExitSuccess
|
||||
[2024-10-01 17:00:12.865042169] (Utility.Process) process [2784966] done ExitSuccess
|
||||
[2024-10-01 17:00:12.865124487] (Utility.Process) process [2784965] done ExitSuccess
|
||||
[2024-10-01 17:00:12.865195432] (Utility.Process) process [2784964] done ExitSuccess
|
||||
get "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220
|
||||
\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (from web...)
|
||||
[2024-10-01 17:00:12.898739526] (Utility.Url) Request {
|
||||
host = "www.youtube.com"
|
||||
port = 443
|
||||
secure = True
|
||||
requestHeaders = [("Accept-Encoding","identity"),("User-Agent","git-annex/10.20240927+git1-gf3403e9691-1~ndall+1")]
|
||||
path = "/watch"
|
||||
queryString = "?v=5rFP3DlC2MU"
|
||||
method = "GET"
|
||||
proxy = Nothing
|
||||
rawBody = False
|
||||
redirectCount = 10
|
||||
responseTimeout = ResponseTimeoutDefault
|
||||
requestVersion = HTTP/1.1
|
||||
proxySecureMode = ProxySecureWithConnect
|
||||
get "\320\247\320\260\321\202_\321\200\321\203\320\273\320\265\321\202\320\272\320\260/2024-10-01-\320\240\320\260\321\201\321\201\320\265\321\217\320\275\320\272\320\260__\320\220\320\275\320\264\321\200\321\226\320\271_\320\237\320\276\320\277\320\270\320\272.mkv" (from web...)
|
||||
Verification of content failed
|
||||
|
||||
Unable to access these remotes: web
|
||||
|
||||
No other repository is known to contain the file.
|
||||
|
||||
(Note that these git remotes have annex-ignore set: origin)
|
||||
failed
|
||||
[2024-10-01 17:00:13.516843908] (Utility.Process) process [2784963] done ExitSuccess
|
||||
get: 1 failed
|
||||
|
||||
```
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
```
|
||||
git clone http://github.com/TrueTube/Andriy_Popik
|
||||
cd Andriy*
|
||||
git config annex.security.allowed-ip-addresses all
|
||||
git annex get --debug Чат_рулетка/2024-10-01-Рассеянка__Андрій_Попик.mkv
|
||||
```
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
|
||||
```
|
||||
$> git annex version
|
||||
git-annex version: 10.20240927+git1-gf3403e9691-1~ndall+1
|
||||
```
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
[[!comment format=mdwn
|
||||
username="Spencer"
|
||||
avatar="http://cdn.libravatar.org/avatar/2e0829f36a68480155e09d0883794a55"
|
||||
subject="[FR] Remote Settings for All Clones"
|
||||
date="2024-10-09T23:10:17Z"
|
||||
content="""
|
||||
I'd like to set a few additional configurations so that all clones treat a special remote similarly.
|
||||
|
||||
Particularly I'd like to set the trustlevel and tracking-branch for an exporttree special remote so that any clone that enables this remote also have these configurations enabled.
|
||||
In particular this is justified for a certain remote of mine because it exports to a version controlled environment that I trust, so it would just be nice not to have to run `git config remote.name.annex-tracking-branch` and `git annex trust name semitrusted` for every clone.
|
||||
|
||||
Of course, are `git annex config --set remote.name.annex-trustlevel \"semitrusted\"` and `git annex config --set remote.name.annex-tracking-branch \"main\"` (called once) any easier than the above called multiple times? Maybe not, but it would be slightly less mental overhead to not do the above.
|
||||
|
||||
Off hand can you imagine any caveats that would preclude adding these settings to the list of supported for this command? I agree that only some make sense for all clones to see rather than anything one can set in git config but of course that specification requires manual addition of config cases that do make sense. Maybe this is one of them.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
[[!comment format=mdwn
|
||||
username="annex@9cc004f218c318a28099ff2645959be0fcbc6d94"
|
||||
nickname="annex"
|
||||
avatar="http://cdn.libravatar.org/avatar/18c7af44d49b8f2e318b49d6026c84fb"
|
||||
subject="Support for importtree"
|
||||
date="2024-10-09T06:42:40Z"
|
||||
content="""
|
||||
What's the reason for not supporting importtree via webdav?
|
||||
|
||||
Would be nice to keep a tree in sync on my nextcloud and sync to my phone etc.
|
||||
"""]]
|
||||
|
|
@ -48,8 +48,14 @@ recover any files stored in it.
|
|||
When the git config "remote.name.private" is set, git-annex will avoid
|
||||
recording anything in the git-annex branch about the remote. This is
|
||||
set by `git-annex initremote --private`, and could also be set for
|
||||
git remotes. This could be useful, perhaps. Update this tip if you have a
|
||||
good way to use it.
|
||||
git remotes. This may be useful if, for example, you are trying to deduplicate content,
|
||||
bifurcate repositories, or reinject it using a temporary annex as a staging area.
|
||||
Git annex is excellent for these tasks because it naturally hashes all file content,
|
||||
therefore if a 'copy' appears in one repo that should belong in another, you can drop
|
||||
its content or move it to deduplicate. However, in this case, git-annex logging a relationship
|
||||
between the two repos is undesirable. Especially if the repos are otherwise unrelated or
|
||||
one of them is temporary (to be deleted once emptied), `remote.private` is preferrable to declaring
|
||||
the repo dead and doing a `forget --drop-dead --force` operation.
|
||||
|
||||
## where the data is actually stored
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,20 @@
|
|||
[[!comment format=mdwn
|
||||
username="Spencer"
|
||||
avatar="http://cdn.libravatar.org/avatar/2e0829f36a68480155e09d0883794a55"
|
||||
subject="How to Clone?"
|
||||
date="2024-10-07T20:00:24Z"
|
||||
content="""
|
||||
To access the manifest and bundles, one needs the UUID of the special remote initially configured. Then one can run
|
||||
|
||||
[[!format sh \"\"\"
|
||||
git clone 'annex::<UUID>?type=directory&encryption=none&directory=/path/to/space%20sanitized%20directory'
|
||||
\"\"\"]]
|
||||
|
||||
A bit tedious for both the need to type all settings (even those not shown by the remote helper when doing the push operations from the initial repo, in this case the directory, in other cases all required settings to init the remote in the first place) and for having to HTML sanitize any URL disallowed characters. But doable
|
||||
|
||||
The other option would be to manually clone by initializing the new empty repo, then adding the special remote the normal git annex way. This doesn't work right just yet because `--uuid` is not an allowed option for `initremote`. It would be nice if this were an option simply to avoid the tedium of typing the URL as above (one could copy and paste `git --no-pager show git-annex:remote.log` into `initremote`)
|
||||
|
||||
Despite the URL tedium, an exciting result of the current system is that *any number* of repos and file annexes can share one directory! Like an entire organization (or repo group) in one folder. Datalad has a similar archetype (remote indexed archives) which offer (slightly) improved user friendliness by filing each repo UUID into meaningfully-named folders (unhashed first three/remaining is nice for being actually the UUID but it still doesn't let me easily copy/paste the UUID for cloning). Although I kind of like how git-annex's implementation encourages a single unified \"annex\" (rather than RIA's `UUID/Annex`which gives each UUID a separate annex) and of course bundles over loose git files, especially for cloud special remotes which can be slow to upload each and every loose file.
|
||||
|
||||
Looking forward to seeing how this feature develops!
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
For p2phttp support in forgejo-aneksajo I decided to just spawn a `git annex p2phttp --wideopen` server, do authentication on the Forgejo side, and then proxy requests to p2phttp. Since p2phttp only supports serving one repository at the moment this means that I have to allocate one free port per repository. Actually finding a free port adds complexity and a race condition, as there also seems to be no way to set `--port 0` for p2phttp and then figure out which port it bound to.
|
||||
|
||||
This would be simplified if p2phttp could listen on unix domain sockets instead.
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
From my experimentation it seems to be that git-annex does not discover the `annex.url` config after the initial clone of a repository. There are at least two situations in which this would be useful though:
|
||||
|
||||
1. If the server-side supports p2phttp, but the repository is cloned with an older version of git-annex that doesn't, the annexurl won't be picked up even if the client-side git-annex is later updated to a version that does support p2phttp.
|
||||
2. Likewise, if the server-side initially didn't support p2phttp and didn't set `annex.url` when the repository was cloned, but is later updated to support it, git-annex doesn't automatically pick up this change.
|
||||
|
||||
This automatic discovery would be nice for p2phttp support in forgejo-aneksajo, as existing clones could automatically start making use of it as soon as the instance is updated to support it on the server-side and the git-annex version is updated to be recent enough on the client-side.
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
Plain git tries the same credentials for multiple different repositories on the same host. I.e. with multiple different repositories on <https://atris.fz-juelich.de/>, if I push to one and supply my credentials, these will be saved omitting the actual repository path on the host and will be reused for any other repository that is also located on ATRIS.
|
||||
|
||||
Git-annex with p2phttp behaves differently. It saves the full URL to the p2phttp endpoint including the repository path, which means that two repositories using p2phttp will both ask for credentials and save them separate from each other.
|
||||
|
||||
This difference in behavior seems to stem from a difference in how `git credential` handles schemes: if you ask it for credentials for `http(s)` it will silently omit any supplied path and only match on scheme and host, while asking for `annex+http(s)` matches on the full scheme, host and path.
|
||||
|
||||
There might be some situations in which one would want to associate the credentials with the full path, but in my case for forgejo-aneksajo all authentication is handled by Forgejo and users are global on that instance so per-repository credentials don't make much sense.
|
||||
|
||||
I see some ways to address this:
|
||||
|
||||
1. Remove the path from the request to `git credential` on git-annex' side
|
||||
2. Allow `remote.<name>.annexurl` to be set to `http(s)://` URLs in addition to `annex+http(s)://`, exploiting the difference in the `git credential` behavior
|
||||
3. Perhaps most elegantly: make p2phttp support serving multiple repositories, so that repositories could share the same annexurl and therefore share credentials
|
||||
Loading…
Reference in a new issue