This commit is contained in:
leavingchicago@c04c893e78d1c4c76cb3e32b5c227cf42bbf7682
admin
parent
9a772ee987
commit
7ee0e9d816
1 changed files with 215 additions and 0 deletions
|
|
@ -0,0 +1,215 @@
|
|||
### Please describe the problem.
|
||||
Sync / Assistant between a `source` and `transfer` repo results in zero copies for whereis, but data is available.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
See a script below...
|
||||
|
||||
``` sh
|
||||
#!/bin/sh
|
||||
|
||||
TMP=$(mktemp -d)
|
||||
|
||||
|
||||
##########################################
|
||||
# Set up the repos
|
||||
##########################################
|
||||
|
||||
|
||||
###
|
||||
# Create working directories
|
||||
cd ${TMP}
|
||||
mkdir source transfer
|
||||
|
||||
|
||||
###
|
||||
# Set up the source repo
|
||||
cd ${TMP}/source
|
||||
git init
|
||||
git annex init source
|
||||
git-annex group here source
|
||||
git annex wanted here standard
|
||||
|
||||
|
||||
###
|
||||
# Set up the transfer repo
|
||||
# N.B. It can see the source repo
|
||||
cd ${TMP}/transfer
|
||||
git init
|
||||
git remote add source ${TMP}/source
|
||||
git annex init transfer
|
||||
git-annex group here transfer
|
||||
git annex wanted here \
|
||||
'not (inallgroup=client and copies=client:1) and ((include=* and ((exclude=*/archive/* and exclude=archive/*) or (not (copies=archive:1 or copies=smallarchive:1)))) or approxlackingcopies=1)'
|
||||
|
||||
|
||||
##########################################
|
||||
# Add some files and run the assistant
|
||||
##########################################
|
||||
|
||||
###
|
||||
# Make sure it works by adding some files to the source...
|
||||
cd ${TMP}/source
|
||||
date > file.1
|
||||
date > file.2
|
||||
date > file.3
|
||||
|
||||
# Run the assistant in the source...
|
||||
(cd ${TMP}/source && \
|
||||
git annex assistant && \
|
||||
sleep 30 && \
|
||||
git annex assistant --stop && \
|
||||
git annex whereis file.*)
|
||||
# PASS: All files are in source.
|
||||
|
||||
# Now run it in the transfer group
|
||||
(cd ${TMP}/transfer && \
|
||||
git annex assistant && \
|
||||
sleep 30 && \
|
||||
git annex assistant --stop && \
|
||||
git annex whereis file.*)
|
||||
# PASS: All files are in transfer
|
||||
|
||||
# Check the source
|
||||
(cd ${TMP}/source && \
|
||||
git annex whereis file.*)
|
||||
# FAIL: Only file.1 was updated to be in source.
|
||||
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
|
||||
##########################################
|
||||
# Start to scratch head and ask 'Why?'
|
||||
##########################################
|
||||
|
||||
##########
|
||||
# What's happening???
|
||||
|
||||
cd ${TMP}/source && git annex sync
|
||||
# Source thinks it is updated - working tree clean
|
||||
|
||||
cd ${TMP}/transfer && git annex sync
|
||||
# Transfer pulls from source, so run again
|
||||
git annex sync
|
||||
# OK... working tree clean
|
||||
|
||||
|
||||
# Check again
|
||||
cd ${TMP}/source && git annex whereis file.*
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
cd ${TMP}/transfer && git annex whereis file.*
|
||||
# whereis file.1 (1 copy)
|
||||
# 64b193e0-86f0-4349-b073-70af919ce628 -- transfer [here]
|
||||
# ok
|
||||
# whereis file.2 (0 copies) failed
|
||||
# whereis file.3 (0 copies) failed
|
||||
# git-annex: whereis: 2 failed
|
||||
|
||||
|
||||
# HOWEVER... Files are definitely in the transfer repo
|
||||
|
||||
cd ${TMP}/source && cat file.*
|
||||
# cat: file.1: No such file or directory
|
||||
# cat: file.2: No such file or directory
|
||||
# cat: file.3: No such file or directory
|
||||
|
||||
|
||||
cd ${TMP}/transfer && cat file.*
|
||||
# Sun May 14 21:22:02 PDT 2017
|
||||
# Sun May 14 21:22:03 PDT 2017
|
||||
# Sun May 14 21:22:06 PDT 2017
|
||||
```
|
||||
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
#### Version:
|
||||
```
|
||||
git-annex version: 6.20170510
|
||||
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV FsEvents ConcurrentOutput TorrentParser MagicMime Feeds Quvi
|
||||
dependency versions: aws-0.16 bloomfilter-2.0.1.0 cryptonite-0.23 DAV-1.3.1 feed-0.3.12.0 ghc-8.0.2 http-client-0.5.6.1 persistent-sqlite-2.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.4.5
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
||||
local repository version: 5
|
||||
supported repository versions: 3 5 6
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5
|
||||
operating system: darwin x86_64
|
||||
```
|
||||
|
||||
#### OS:
|
||||
MacOS 10.12.4, but seems to apply to Debian Jessie too.
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Annex assistant daemon.log for `source` repo:
|
||||
|
||||
``` sh
|
||||
[2017-05-15 14:11:24.150202] main: starting assistant version 6.20170510
|
||||
[2017-05-15 14:11:24.196597] Cronner: You should enable consistency checking to protect your data.
|
||||
(scanning...) [2017-05-15 14:11:24.257123] Watcher: Performing startup scan
|
||||
(started...)
|
||||
[2017-05-15 14:11:45.505855] Committer: Adding file.1 file.2 file.3
|
||||
add file.1 ok
|
||||
add file.2 ok
|
||||
add file.3 ok
|
||||
[2017-05-15 14:11:45.544277] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
(recording state in git...)
|
||||
```
|
||||
|
||||
Annex assistant daemon.log for `transfer` repo:
|
||||
|
||||
``` sh
|
||||
[2017-05-15 14:12:09.127406] main: starting assistant version 6.20170510
|
||||
[2017-05-15 14:12:09.157611] Cronner: You should enable consistency checking to protect your data.
|
||||
[2017-05-15 14:12:09.177452] TransferScanner: Syncing with source
|
||||
(scanning...) [2017-05-15 14:12:09.210605] Watcher: Performing startup scan
|
||||
(started...)
|
||||
warning: no common commits
|
||||
From /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
* [new branch] git-annex -> source/git-annex
|
||||
* [new branch] master -> source/master
|
||||
* [new branch] synced/master -> source/synced/master
|
||||
(merging source/git-annex into git-annex...)
|
||||
(recording state in git...)
|
||||
|
||||
|
||||
Already up-to-date.
|
||||
[2017-05-15 14:12:10.256195] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
(checksum...) [2017-05-15 14:12:10.425494] Transferrer: Downloaded file.1
|
||||
[2017-05-15 14:12:11.17664] Pusher: Syncing with source
|
||||
(recording state in git...)
|
||||
[2017-05-15 14:12:11.356447] Committer: Committing changes to git
|
||||
(recording state in git...)
|
||||
To /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
* [new branch] git-annex -> synced/git-annex
|
||||
(checksum...) [2017-05-15 14:12:11.428676] Transferrer: Downloaded file.2
|
||||
(checksum...) [2017-05-15 14:12:11.451457] Transferrer: Downloaded file.3
|
||||
drop source file.1 ok
|
||||
drop source file.2 ok
|
||||
drop source file.3 ok
|
||||
[2017-05-15 14:12:13.477636] Pusher: Syncing with source
|
||||
(recording state in git...)
|
||||
To /Users/olaf/tmp/git-annex/play/bug-play/source
|
||||
825d3dd..60c96cb git-annex -> synced/git-annex
|
||||
```
|
||||
|
||||
|
||||
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
||||
|
||||
Yes. Love it. Donated. Have been using it for years. Recommend it and get(/force) my collaborators to use it. ;-)
|
||||
|
||||
Loading…
Add table
Reference in a new issue