Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
commit
15ef5e62d2
6 changed files with 215 additions and 0 deletions
|
@ -0,0 +1,9 @@
|
|||
[[!comment format=mdwn
|
||||
username="jgsuess@732b8c62c50d8595d7b1d58eea11e5019c2308b1"
|
||||
nickname="jgsuess"
|
||||
avatar="http://cdn.libravatar.org/avatar/35a99d1b5df046b206807941ce9795d3"
|
||||
subject="Automatic watch for the heuristic"
|
||||
date="2021-08-07T12:25:02Z"
|
||||
content="""
|
||||
It seems there is a good heuristic based on the object size. It is given in this thread. Could this check not be used in annex automatic syncing modes to avoid this problem. I have run into it several times now and recovery, while possible, is time-consuming and messy.
|
||||
"""]]
|
|
@ -0,0 +1,43 @@
|
|||
### Please describe the problem.
|
||||
|
||||
it seems that git-annex/linux/current/git-annex-standalone-amd64.tar.gz which is claimed in the last commit to be 8.20210804 is really a 8.20210715-g6e67ea5d7
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
```
|
||||
(git-annex)lena:~datalad/distributions/downloads.kitenet.net[master]git-annex
|
||||
$> git log git-annex/linux/current/git-annex-standalone-amd64.tar.gz | head
|
||||
commit 9670be5007ab9a084e9f7a47c9d315555b477ca0
|
||||
Author: Joey Hess <joeyh@joeyh.name>
|
||||
Date: Tue Aug 3 12:30:46 2021 -0400
|
||||
|
||||
publishing git-annex 8.20210804
|
||||
|
||||
commit 780e402a5f0fd188e2edef61303db023f492459c
|
||||
Author: Joey Hess <joeyh@joeyh.name>
|
||||
Date: Wed Jul 14 12:33:03 2021 -0400
|
||||
|
||||
$> ls -l git-annex/linux/current/git-annex-standalone-amd64.tar.gz
|
||||
lrwxrwxrwx 1 yoh yoh 215 Aug 6 17:35 git-annex/linux/current/git-annex-standalone-amd64.tar.gz -> ../../../.git/annex/objects/9q/67/SHA256E-s51281961--c2419981d633d598b760c89f5a309d1ec7ec66f5b8a1e734a7239b2d4c7a3d21.tar.gz/SHA256E-s51281961--c2419981d633d598b760c89f5a309d1ec7ec66f5b8a1e734a7239b2d4c7a3d21.tar.gz
|
||||
|
||||
$> tar -xzf git-annex/linux/current/git-annex-standalone-amd64.tar.gz
|
||||
|
||||
$> lst
|
||||
total 4
|
||||
drwx------ 1 yoh yoh 472 Aug 6 17:48 git-annex.linux/
|
||||
...
|
||||
|
||||
$> ./git-annex.linux/git-annex version | head
|
||||
git-annex version: 8.20210715-g6e67ea5d7
|
||||
...
|
||||
```
|
||||
|
||||
also note that it is "publishing git-annex 8.20210804" (on 0803) and there is no `8.20210804` version but there is
|
||||
|
||||
```
|
||||
$> git tag | grep 202108
|
||||
8.20210803
|
||||
```
|
||||
|
||||
so it seems odd
|
|
@ -0,0 +1,81 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I have been trying to get to the bottom of it but failed
|
||||
|
||||
Seems to relate to the version and/or build of git-annex on remote end, but happens whenever on remote it is a conda-forge build 8.20210621-g91f9aac
|
||||
|
||||
Whenever I can get/drop without `--debug`, with `--debug` it fails:
|
||||
|
||||
```
|
||||
(git-annex)lena:/tmp/testds-mit[master]git-annex
|
||||
$> git annex get 123
|
||||
get 123 (from origin...)
|
||||
ok
|
||||
(recording state in git...)
|
||||
1 11897.....................................:Fri 06 Aug 2021 06:09:05 PM EDT:.
|
||||
(git-annex)lena:/tmp/testds-mit[master]git-annex
|
||||
$> git annex drop 123
|
||||
drop 123 (locking origin...) ok
|
||||
(recording state in git...)
|
||||
1 11898.....................................:Fri 06 Aug 2021 06:09:11 PM EDT:.
|
||||
(git-annex)lena:/tmp/testds-mit[master]git-annex
|
||||
$> git annex get 123 --debug
|
||||
[2021-08-06 18:09:22.424837137] (Utility.Process) process [1488170] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","symbolic-ref","-q","HEAD"]
|
||||
[2021-08-06 18:09:22.437700889] (Utility.Process) process [1488170] done ExitSuccess
|
||||
[2021-08-06 18:09:22.438748331] (Utility.Process) process [1488171] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","refs/heads/master"]
|
||||
[2021-08-06 18:09:22.449949392] (Utility.Process) process [1488171] done ExitSuccess
|
||||
[2021-08-06 18:09:22.451491273] (Utility.Process) process [1488172] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","ls-files","--stage","-z","--","123"]
|
||||
[2021-08-06 18:09:22.45371749] (Utility.Process) process [1488173] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
[2021-08-06 18:09:22.455744285] (Utility.Process) process [1488174] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
[2021-08-06 18:09:22.458678421] (Utility.Process) process [1488175] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch=%(objectname) %(objecttype) %(objectsize)","--buffer"]
|
||||
get 123 [2021-08-06 18:09:22.471519462] (Utility.Process) process [1488176] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","git-annex"]
|
||||
[2021-08-06 18:09:22.479244675] (Utility.Process) process [1488176] done ExitSuccess
|
||||
[2021-08-06 18:09:22.480965615] (Utility.Process) process [1488177] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","show-ref","--hash","refs/heads/git-annex"]
|
||||
[2021-08-06 18:09:22.494689644] (Utility.Process) process [1488177] done ExitSuccess
|
||||
[2021-08-06 18:09:22.496859969] (Utility.Process) process [1488178] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..b1854b845d5ab7dbae4f50607ae02b76ae526716","--pretty=%H","-n1"]
|
||||
[2021-08-06 18:09:22.505499833] (Utility.Process) process [1488178] done ExitSuccess
|
||||
[2021-08-06 18:09:22.507468513] (Utility.Process) process [1488179] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","log","refs/heads/git-annex..83d0262dcb3d3abf7f64ce7079e5d6e4a27c2293","--pretty=%H","-n1"]
|
||||
[2021-08-06 18:09:22.517489766] (Utility.Process) process [1488179] done ExitSuccess
|
||||
[2021-08-06 18:09:22.519480403] (Utility.Process) process [1488180] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
|
||||
[2021-08-06 18:09:22.519793644] (Utility.Process) process [1488181] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
|
||||
(from origin...)
|
||||
[2021-08-06 18:09:22.524141345] (Utility.Process) process [1488182] read: ssh ["-o","BatchMode=true","-S",".git/annex/ssh/openmind7.mit.edu","-o","ControlMaster=auto","-o","ControlPersist=yes","-n","-T","openmind7.mit.edu","true"]
|
||||
[2021-08-06 18:09:24.650559322] (Utility.Process) process [1488182] done ExitSuccess
|
||||
[2021-08-06 18:09:24.651134161] (Utility.Process) process [1488187] chat: ssh ["openmind7.mit.edu","-S",".git/annex/ssh/openmind7.mit.edu","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","git-annex-shell 'p2pstdio' '/~/tmp/testds' '--debug' '339c2fc7-df7a-44fd-97eb-5a2c41ac8a0a' --uuid f30e2969-4ddf-4edb-ba45-37c455816c9a"]
|
||||
[2021-08-06 18:09:24.80528113] (P2P.IO) [ssh connection Just 1488187] [ThreadId 4] P2P > ERROR auth failed
|
||||
[2021-08-06 18:09:24.84393255] (Utility.Process) process [1488187] done ExitSuccess
|
||||
[2021-08-06 18:09:24.846367147] (Utility.Process) process [1488189] read: rsync ["--progress","--inplace","--perms","-e","'ssh' 'openmind7.mit.edu' '-S' '.git/annex/ssh/openmind7.mit.edu' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'git-annex-shell ''sendkey'' ''/~/tmp/testds'' ''--debug'' ''MD5E-s0--d41d8cd98f00b204e9800998ecf8427e'' --uuid f30e2969-4ddf-4edb-ba45-37c455816c9a ''--'' ''remoteuuid=339c2fc7-df7a-44fd-97eb-5a2c41ac8a0a'' ''unlocked='' ''direct='' ''associatedfile=123'' ''--'''","--","dummy:",".git/annex/tmp/MD5E-s0--d41d8cd98f00b204e9800998ecf8427e"]
|
||||
protocol version mismatch -- is your shell clean?
|
||||
(see the rsync man page for an explanation)
|
||||
rsync error: protocol incompatibility (code 2) at compat.c(178) [Receiver=3.1.3]
|
||||
[2021-08-06 18:09:24.970785079] (Utility.Process) process [1488189] done ExitFailure 2
|
||||
rsync exited 2
|
||||
|
||||
rsync failed -- run git annex again to resume file transfer
|
||||
|
||||
Transfer failed
|
||||
[2021-08-06 18:09:25.014462898] (Utility.Process) process [1488180] done ExitSuccess
|
||||
[2021-08-06 18:09:25.016852848] (Utility.Process) process [1488181] done ExitSuccess
|
||||
|
||||
Unable to access these remotes: origin
|
||||
[2021-08-06 18:09:25.021878961] (Utility.Process) process [1488191] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch"]
|
||||
[2021-08-06 18:09:25.025537927] (Utility.Process) process [1488192] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","annex.debug=true","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
|
||||
|
||||
No other repository is known to contain the file.
|
||||
failed
|
||||
[2021-08-06 18:09:25.038970056] (Utility.Process) process [1488191] done ExitSuccess
|
||||
[2021-08-06 18:09:25.040408598] (Utility.Process) process [1488192] done ExitSuccess
|
||||
[2021-08-06 18:09:25.040589952] (Utility.Process) process [1488175] done ExitSuccess
|
||||
[2021-08-06 18:09:25.040701306] (Utility.Process) process [1488174] done ExitSuccess
|
||||
[2021-08-06 18:09:25.040787814] (Utility.Process) process [1488173] done ExitSuccess
|
||||
[2021-08-06 18:09:25.040861945] (Utility.Process) process [1488172] done ExitSuccess
|
||||
[2021-08-06 18:09:25.041785909] (Utility.Process) process [1488193] read: ssh ["-O","stop","-S","openmind7.mit.edu","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"]
|
||||
[2021-08-06 18:09:25.061503656] (Utility.Process) process [1488193] done ExitSuccess
|
||||
get: 1 failed
|
||||
|
||||
```
|
||||
|
||||
I just merged freshier update of conda-forge to 8.20210803 -- will try that one later
|
||||
|
||||
[[!meta author=yoh]]
|
||||
[[!tag projects/datalad]]
|
|
@ -0,0 +1,33 @@
|
|||
### Please describe the problem.
|
||||
I'm not entirely sure what I did. Maybe add files with weird names and I copied a lot of files with cp -rL
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
No idea.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
git-annex version: 8.20210803-g9cae7c5bb
|
||||
Archlinux, up to date.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
Basically, everything I do is answered with the error mentioned in the title.
|
||||
|
||||
[alex@proliant bannex]$ git annex fix
|
||||
[alex@proliant bannex]$ git annex add .
|
||||
git-annex: readDirStream: inappropriate type (Bad message)
|
||||
[alex@proliant bannex]$ git annex sync
|
||||
git-annex: readDirStream: inappropriate type (Bad message)
|
||||
[alex@proliant bannex]$ git annex fsck
|
||||
git-annex: readDirStream: inappropriate type (Bad message)
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes. This is my third attempt at getting git-annex running. Conceptually, I think it's the best thing out there. I've had some success at moving around data. But I tend to end up in these dead ends...
|
|
@ -0,0 +1,41 @@
|
|||
[[!comment format=mdwn
|
||||
username="Rob"
|
||||
avatar="http://cdn.libravatar.org/avatar/1d12b1e7e4d6ef1dd57cd9f3ec848dc7"
|
||||
subject="creating directory special remote "in-place""
|
||||
date="2021-08-05T18:13:35Z"
|
||||
content="""
|
||||
I'm using git-annex for backing up a variety of data, with several different remotes (including a USB drive for backup, rsync for encrypted cloud backup, etc). I have one particular use case that I am trying to figure out how to implement:
|
||||
|
||||
Pictures, Music, and Video are stored on a WD MyCloud under three corresponding folders. They are accessible via NFS, as well as via AFP; I specifically mount them on my laptop via NFS. I want to be able to access the files in the following ways:
|
||||
|
||||
- Store new photos from other sources, including my laptop, using git-annex.
|
||||
- Have available the list of files on my laptop repository, in git-annex... i.e. like any repository.
|
||||
- Retain all the file contents on the WD MyCloud... i.e. like a special remote.
|
||||
- ONLY pull file contents to the laptop (or other repositories) if specifically requested (i.e. I don't want a sync to cause things to get pulled to the laptop).
|
||||
- When photos are pulled from the special remote, they go into ~/annex/photos (i.e. a different name than the folder on the remote).
|
||||
- I want to be able to access the files on the WD MyCloud \"normally\" from other computers, TVs, etc. I realize there's some 'danger' here. But I wanted to keep normal file trees for the special remote, not the hash-based filenames with symlinks.
|
||||
|
||||
So what I decided I needed was a directory special remote, with both the exporttree and importtree options. So I created the special remote using something like:
|
||||
|
||||
git annex initremote wdmycloud-pics directory=/net/wdmycloud/nfs/Public/Shared\ Pictures encryption=none exportree=yes importtree=yes
|
||||
|
||||
On my laptop's repository, I executed the following:
|
||||
|
||||
git annex import master:photos --from wdmycloud-pics
|
||||
git merge wdmycloud-pics/master
|
||||
|
||||
Problem:
|
||||
- This imports the content from the special remote, into the laptop's repository. I don't want that.
|
||||
|
||||
I do realize that my main git-annex repository on the laptop probably has to actually download all of the files in order to compute their hashes. However, I do not want them to stay on the laptop, as there is nowhere's near enough space for all of them. I did try:
|
||||
|
||||
git-annex wanted wdmycloud-pics \"exclude=photos/*\"
|
||||
|
||||
However, this results in none of the files on the WD MyCloud actually getting processed.
|
||||
|
||||
What am I missing?
|
||||
|
||||
Thanks!
|
||||
-Rob
|
||||
|
||||
"""]]
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="Lukey"
|
||||
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
|
||||
subject="comment 20"
|
||||
date="2021-08-06T06:02:38Z"
|
||||
content="""
|
||||
Note that `git-annex-import` has a `--no-content` switch.
|
||||
"""]]
|
Loading…
Reference in a new issue