Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
022c3910e9
8 changed files with 407 additions and 0 deletions
|
|
@ -0,0 +1,95 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
|
||||
nickname="Richard"
|
||||
subject="comment 3"
|
||||
date="2013-08-02T06:57:12Z"
|
||||
content="""
|
||||
In this particular case
|
||||
|
||||
* It's XFS
|
||||
* It's a normal repo
|
||||
* That the inode can have only one shared mtime is obvious, but the symlinks could have their own:
|
||||
|
||||
Random line to make MarkDown happy
|
||||
|
||||
richih@eudyptes (git)-[master] ~/killme/target % stat * .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
File: ‘foo’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268638577 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:51:35.976674954 +0200
|
||||
Modify: 2013-07-30 01:19:41.000000000 +0200
|
||||
Change: 2013-07-30 01:41:43.700701516 +0200
|
||||
Birth: -
|
||||
File: ‘foobar’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268774374 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:51:35.976674954 +0200
|
||||
Modify: 2013-07-30 01:19:41.000000000 +0200
|
||||
Change: 2013-07-30 01:41:43.700701516 +0200
|
||||
Birth: -
|
||||
File: ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
|
||||
Device: 804h/2052d Inode: 73750 Links: 1
|
||||
Access: (0444/-r--r--r--) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-07-30 01:20:11.915507749 +0200
|
||||
Modify: 2013-07-30 01:19:41.131509226 +0200
|
||||
Change: 2013-07-30 01:20:11.915507749 +0200
|
||||
Birth: -
|
||||
richih@eudyptes (git)-[master] ~/killme/target % touch foo
|
||||
richih@eudyptes (git)-[master] ~/killme/target % stat * .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
File: ‘foo’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268638577 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:51:35.976674954 +0200
|
||||
Modify: 2013-07-30 01:19:41.000000000 +0200
|
||||
Change: 2013-07-30 01:41:43.700701516 +0200
|
||||
Birth: -
|
||||
File: ‘foobar’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268774374 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:51:35.976674954 +0200
|
||||
Modify: 2013-07-30 01:19:41.000000000 +0200
|
||||
Change: 2013-07-30 01:41:43.700701516 +0200
|
||||
Birth: -
|
||||
File: ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
|
||||
Device: 804h/2052d Inode: 73750 Links: 1
|
||||
Access: (0444/-r--r--r--) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:54:34.464668766 +0200
|
||||
Modify: 2013-08-02 08:54:34.464668766 +0200
|
||||
Change: 2013-08-02 08:54:34.464668766 +0200
|
||||
Birth: -
|
||||
richih@eudyptes (git)-[master] ~/killme/target % touch --no-dereference foo
|
||||
richih@eudyptes (git)-[master] ~/killme/target % stat * .git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
|
||||
File: ‘foo’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268638577 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:55:07.796667610 +0200
|
||||
Modify: 2013-08-02 08:55:07.796667610 +0200
|
||||
Change: 2013-08-02 08:55:07.796667610 +0200
|
||||
Birth: -
|
||||
File: ‘foobar’ -> ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 178 Blocks: 8 IO Block: 4096 symbolic link
|
||||
Device: 804h/2052d Inode: 268774374 Links: 1
|
||||
Access: (0777/lrwxrwxrwx) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:51:35.976674954 +0200
|
||||
Modify: 2013-07-30 01:19:41.000000000 +0200
|
||||
Change: 2013-07-30 01:41:43.700701516 +0200
|
||||
Birth: -
|
||||
File: ‘.git/annex/objects/pX/ZJ/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855’
|
||||
Size: 0 Blocks: 0 IO Block: 4096 regular empty file
|
||||
Device: 804h/2052d Inode: 73750 Links: 1
|
||||
Access: (0444/-r--r--r--) Uid: ( 1000/ richih) Gid: ( 1000/ richih)
|
||||
Access: 2013-08-02 08:54:34.464668766 +0200
|
||||
Modify: 2013-08-02 08:54:34.464668766 +0200
|
||||
Change: 2013-08-02 08:54:34.464668766 +0200
|
||||
Birth: -
|
||||
richih@eudyptes (git)-[master] ~/killme/target %
|
||||
|
||||
As to where the initial time of 01:19:41 came from: Yes, those were earlier tests with other imports of empty files.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnWvnTWY6LrcPB4BzYEBn5mRTpNhg5EtEg"
|
||||
nickname="Bence"
|
||||
subject="checkpresent again"
|
||||
date="2013-08-01T23:18:38Z"
|
||||
content="""
|
||||
In the current [HEAD](https://github.com/joeyh/git-annex/commit/bb74db6ef094324062adcf26a677113ee6fd0e58) the \"checkpresent\" method in [Hook.hs](https://github.com/joeyh/git-annex/blob/master/Remote/Hook.hs#L145) is missing a \"return\" while other hooks have a return value eg. [Directory.hs](https://github.com/joeyh/git-annex/blob/master/Remote/Directory.hs#L241), [Rsync.hs](https://github.com/joeyh/git-annex/blob/master/Remote/Rsync.hs#L272), ...
|
||||
|
||||
|
||||
I noticed that if my *checkpresent* hook does not output anything (to be exact: I commented out all the lines in the block) it behaves strangely. I copied the test file to \"copyrepo\", removed it by hand (so git-annex does not know about the change) and executed a fsck.
|
||||
|
||||
~/annex5/test1/123 $ git annex fsck --from copyrepo --fast
|
||||
fsck somefile.1 (checking copyrepo...) (fixing location log)
|
||||
** Based on the location log, somefile.1
|
||||
** was expected to be present, but its content is missing.
|
||||
failed
|
||||
|
||||
running the check again
|
||||
|
||||
~/annex5/test1/123 $ ga fsck --from copyrepo --fast
|
||||
fsck somefile.1 (checking copyrepo...) ok
|
||||
|
||||
and running the check again without --fast
|
||||
|
||||
~/annex5/test1/123 $ ga fsck --from copyrepo
|
||||
fsck somefile.1 (checking copyrepo...) ok
|
||||
|
||||
It thinks, the file is in the repo but it is not.
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.152.108.145"
|
||||
subject="comment 9"
|
||||
date="2013-08-01T23:51:48Z"
|
||||
content="""
|
||||
The behavior you show with `fsck --from` is that the first time it's run against the damaged remote it notices the file is not present using the checkpresent hook. It then updates the location log. The subsequent times it's run, it sees that the location log says the file is not present in the remote. It verifies this is the case by calling the checkpresent hook. Since the two data sources agree, and numcopies is still satisfied, it prints \"ok\". There does not seem to be a bug here.
|
||||
|
||||
(`return` in Haskell does not do what you would expect to happen in a traditional imperative language. It does not alter control flow, and any function using `return` can be mechanically converted to one that does not use `return`.)
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnaH44G3QbxBAYyDwy0PbvL0ls60XoaR3Y"
|
||||
nickname="Nigel"
|
||||
subject="git annex get failed"
|
||||
date="2013-08-02T14:29:30Z"
|
||||
content="""
|
||||
Hi, I am coming back to this and testing Flickr as a repository for moving files about and have run into what may be my very basic misunderstanding with vanilla annex.
|
||||
|
||||
I copied one file to Flickr and dropped it elsewhere (--force). I assumed that the file was on Flickr ok but that the numcopies setting required the force because of the semi-trust level of the Flickr remote.
|
||||
|
||||
Then I find I can't get the file back, even though there is a record of it from whereis.
|
||||
|
||||
Can you help enlighten me as to what am I missing? I assumed whereis would only report files that exist and can be copied back. If not my error, I can raise bug or search for logs. Thanks in advance for any help.
|
||||
|
||||
[[!format perl \"\"\"
|
||||
|
||||
|
||||
nrb@nrb-ThinkPad-T61:~/tmp$ git annex whereis
|
||||
whereis libpeerconnection.log (3 copies)
|
||||
31124688-0792-4214-9e00-7ed115aa6b8e -- flickr (the flickr library)
|
||||
3e3d40d7-de8f-4591-a4ab-747d74a3b278 -- origin (my laptop)
|
||||
ec2d64fc-30d6-48b4-99bf-7b1bc22d420d -- portable USB drive
|
||||
ok
|
||||
whereis test.cgi (1 copy)
|
||||
31124688-0792-4214-9e00-7ed115aa6b8e -- flickr (the flickr library)
|
||||
ok
|
||||
whereis walkthrough.sh (3 copies)
|
||||
31124688-0792-4214-9e00-7ed115aa6b8e -- flickr (the flickr library)
|
||||
3e3d40d7-de8f-4591-a4ab-747d74a3b278 -- origin (my laptop)
|
||||
ec2d64fc-30d6-48b4-99bf-7b1bc22d420d -- portable USB drive
|
||||
ok
|
||||
whereis walkthrough.sh~ (3 copies)
|
||||
31124688-0792-4214-9e00-7ed115aa6b8e -- flickr (the flickr library)
|
||||
3e3d40d7-de8f-4591-a4ab-747d74a3b278 -- origin (my laptop)
|
||||
ec2d64fc-30d6-48b4-99bf-7b1bc22d420d -- portable USB drive
|
||||
ok
|
||||
nrb@nrb-ThinkPad-T61:~/tmp$ git annex get test.cgi
|
||||
get test.cgi (from flickr...)
|
||||
|
||||
git-annex: /home/nrb/tmp/.git/annex/tmp/SHA256E-s48--a01eedbee949120aeda41e566f9ae8faef1c2bacaa6d7bb8e45050fb8df6d09d.cgi: rename: does not exist (No such file or directory)
|
||||
failed
|
||||
git-annex: get: 1 failed
|
||||
nrb@nrb-ThinkPad-T61:~/tmp$
|
||||
|
||||
\"\"\"]]
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnaH44G3QbxBAYyDwy0PbvL0ls60XoaR3Y"
|
||||
nickname="Nigel"
|
||||
subject="re: git annex get failed"
|
||||
date="2013-08-02T15:02:14Z"
|
||||
content="""
|
||||
Another try - this time a slightly simpler setup using my version of the walkthrough commands
|
||||
|
||||
[[!format bash \"\"\"
|
||||
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex drop walkthrough.sh --from usbdrive
|
||||
drop usbdrive walkthrough.sh ok
|
||||
(Recording state in git...)
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex move walkthrough.sh --to flickr
|
||||
move walkthrough.sh (gpg) (checking flickr...) (to flickr...)
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:92: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:100: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
ok
|
||||
(Recording state in git...)
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex whereis
|
||||
whereis walkthrough.sh (1 copy)
|
||||
161b7af0-2075-4314-9767-308a49b86018 -- flickr (the flickr library)
|
||||
ok
|
||||
whereis walkthrough.sh~ (3 copies)
|
||||
161b7af0-2075-4314-9767-308a49b86018 -- flickr (the flickr library)
|
||||
7803d853-d231-4bb4-b696-f12a950fb96b -- here (my laptop)
|
||||
d60d75f9-d878-4214-af20-fa055134ae77 -- usbdrive (portable USB drive)
|
||||
ok
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex get walkthrough.sh
|
||||
get walkthrough.sh (from flickr...) (gpg)
|
||||
git-annex: /home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--02f600d7e8b071d2945270fd5e7fc26dd066ff31: openBinaryFile: does not exist (No such file or directory)
|
||||
gpg: decrypt_message failed: eof
|
||||
|
||||
Unable to access these remotes: flickr
|
||||
|
||||
Try making some of these repositories available:
|
||||
161b7af0-2075-4314-9767-308a49b86018 -- flickr (the flickr library)
|
||||
failed
|
||||
git-annex: get: 1 failed
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex fsck --from flickr
|
||||
fsck walkthrough.sh (gpg) (checking flickr...) (fixing location log)
|
||||
** Based on the location log, walkthrough.sh
|
||||
** was expected to be present, but its content is missing.
|
||||
|
||||
** No known copies exist of walkthrough.sh
|
||||
failed
|
||||
fsck walkthrough.sh~ (checking flickr...) (fixing location log)
|
||||
** Based on the location log, walkthrough.sh~
|
||||
** was expected to be present, but its content is missing.
|
||||
failed
|
||||
(Recording state in git...)
|
||||
git-annex: fsck: 2 failed
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$
|
||||
|
||||
\"\"\" ]]
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,130 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnaH44G3QbxBAYyDwy0PbvL0ls60XoaR3Y"
|
||||
nickname="Nigel"
|
||||
subject="re: git annex get failed -- debug"
|
||||
date="2013-08-02T15:28:41Z"
|
||||
content="""
|
||||
With debug turned on.
|
||||
|
||||
[[!format bash \"\"\"
|
||||
|
||||
initremote flickr (encryption setup) (shared cipher) ok
|
||||
(Recording state in git...)
|
||||
describe flickr ok
|
||||
(Recording state in git...)
|
||||
/home/nrb/repos/annex/laptop-annex
|
||||
fsck walkthrough.sh (checksum...) ok
|
||||
fsck walkthrough.sh~ (checksum...) ok
|
||||
/home/nrb/repos/annex/laptop-annex
|
||||
copy walkthrough.sh (gpg) (checking flickr...) 16:18:52 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:18:52 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=checkpresent ANNEX_KEY=GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 ANNEX_HASH_1=kQ ANNEX_HASH_2=0P /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:18:52 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:18:52 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:18:52 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:18:54 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:18:54 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:18:55 [flickrannex-0.1.5] main : 'Error. found nothing:{'pages': '1', 'cancreate': '1', 'total': '0', 'page': '1', 'perpage': '0'}'
|
||||
16:18:55 [flickrannex-0.1.5] checkFile : 'GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 - u'gitannex' - '8086216@N08''
|
||||
16:18:55 [flickrannex-0.1.5] checkFile : 'No set exists, thus no files exists'
|
||||
16:18:55 [flickrannex-0.1.5] <module> : 'STOP: 2s'
|
||||
(to flickr...)
|
||||
16:18:55 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:18:55 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=store ANNEX_KEY=GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 ANNEX_HASH_1=kQ ANNEX_HASH_2=0P ANNEX_FILE=/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:18:55 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:18:55 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:18:55 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:18:57 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:18:58 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:18:58 [flickrannex-0.1.5] main : 'Error. found nothing:{'pages': '1', 'cancreate': '1', 'total': '0', 'page': '1', 'perpage': '0'}'
|
||||
16:18:58 [flickrannex-0.1.5] postFile : '/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 to u'gitannex' - GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0'
|
||||
16:18:58 [flickrannex-0.1.5] postFile : 'pre /home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 size: 1047 more than 40234050.'
|
||||
16:18:58 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0' - 'rb''
|
||||
16:18:58 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:18:58 [flickrannex-0.1.5] postFile : 'Uploading: /home/nrb/repos/gits/flickrannex/temp/encoded-GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0'
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:92: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:100: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
16:19:01 [flickrannex-0.1.5] postFile : 'Done: <Element 'rsp' at 0x8726b0c>'
|
||||
16:19:01 [flickrannex-0.1.5] <module> : 'STOP: 5s'
|
||||
ok
|
||||
copy walkthrough.sh~ (checking flickr...) 16:19:01 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:19:01 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=checkpresent ANNEX_KEY=GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 ANNEX_HASH_1=m5 ANNEX_HASH_2=kz /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:19:01 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:19:01 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:19:01 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:19:03 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:19:03 [flickrannex-0.1.5] main : 'Photoset gitannex found: <Element 'photoset' at 0x98264ec>'
|
||||
16:19:03 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:19:03 [flickrannex-0.1.5] checkFile : 'GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 - 72157634897264995L - '8086216@N08''
|
||||
16:19:03 [flickrannex-0.1.5] checkFile : 'No set exists, thus no files exists'
|
||||
16:19:03 [flickrannex-0.1.5] <module> : 'STOP: 1s'
|
||||
(to flickr...)
|
||||
16:19:03 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:19:03 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=store ANNEX_KEY=GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 ANNEX_HASH_1=m5 ANNEX_HASH_2=kz ANNEX_FILE=/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:19:03 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:19:03 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:19:03 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:19:05 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:19:05 [flickrannex-0.1.5] main : 'Photoset gitannex found: <Element 'photoset' at 0x913a54c>'
|
||||
16:19:05 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:19:05 [flickrannex-0.1.5] postFile : '/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 to 72157634897264995L - GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9'
|
||||
16:19:05 [flickrannex-0.1.5] postFile : 'pre /home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 size: 1044 more than 40234050.'
|
||||
16:19:05 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/annex/laptop-annex/.git/annex/tmp/GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9' - 'rb''
|
||||
16:19:05 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:19:05 [flickrannex-0.1.5] postFile : 'Uploading: /home/nrb/repos/gits/flickrannex/temp/encoded-GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9'
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:92: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
/home/nrb/repos/gits/flickrannex/flickrannex.py:100: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
|
||||
if res:
|
||||
16:19:08 [flickrannex-0.1.5] postFile : 'Done: <Element 'rsp' at 0x913f0cc>'
|
||||
16:19:08 [flickrannex-0.1.5] <module> : 'STOP: 4s'
|
||||
ok
|
||||
(Recording state in git...)
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex whereis
|
||||
whereis walkthrough.sh (3 copies)
|
||||
86491ded-899c-425d-9470-bf446cb06db1 -- flickr (the flickr library)
|
||||
8e766014-7154-4f4f-a04b-9d1b3d333db1 -- here (my laptop)
|
||||
eed7055b-743b-4ab6-a390-29cfd326005d -- usbdrive (portable USB drive)
|
||||
ok
|
||||
whereis walkthrough.sh~ (3 copies)
|
||||
86491ded-899c-425d-9470-bf446cb06db1 -- flickr (the flickr library)
|
||||
8e766014-7154-4f4f-a04b-9d1b3d333db1 -- here (my laptop)
|
||||
eed7055b-743b-4ab6-a390-29cfd326005d -- usbdrive (portable USB drive)
|
||||
ok
|
||||
nrb@nrb-ThinkPad-T61:~/repos/annex/laptop-annex$ git annex fsck --from flickr
|
||||
fsck walkthrough.sh (gpg) (checking flickr...) 16:22:57 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:22:57 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=checkpresent ANNEX_KEY=GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 ANNEX_HASH_1=kQ ANNEX_HASH_2=0P /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:22:57 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:22:57 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:22:57 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:22:58 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:22:59 [flickrannex-0.1.5] main : 'Photoset gitannex found: <Element 'photoset' at 0x896d92c>'
|
||||
16:22:59 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:22:59 [flickrannex-0.1.5] checkFile : 'GPGHMACSHA1--280dd2d5003ad3962b1ecaa52ba45fdd44381fd0 - 72157634897264995L - '8086216@N08''
|
||||
16:22:59 [flickrannex-0.1.5] checkFile : 'No set exists, thus no files exists'
|
||||
16:22:59 [flickrannex-0.1.5] <module> : 'STOP: 2s'
|
||||
(fixing location log)
|
||||
** Based on the location log, walkthrough.sh
|
||||
** was expected to be present, but its content is missing.
|
||||
failed
|
||||
fsck walkthrough.sh~ (checking flickr...) 16:22:59 [flickrannex-0.1.5] <module> : 'START'
|
||||
16:22:59 [flickrannex-0.1.5] main : 'ARGS: 'ANNEX_ACTION=checkpresent ANNEX_KEY=GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 ANNEX_HASH_1=m5 ANNEX_HASH_2=kz /home/nrb/repos/gits/flickrannex/flickrannex.py --dbglevel 1 --stderr''
|
||||
16:22:59 [flickrannex-0.1.5] readFile : ''/home/nrb/repos/gits/flickrannex/flickrannex.conf' - 'r''
|
||||
16:22:59 [flickrannex-0.1.5] readFile : 'Done'
|
||||
16:22:59 [flickrannex-0.1.5] login : 'nrbray@yahoo.com'
|
||||
16:23:01 [flickrannex-0.1.5] login : 'Done: '72157633920418017-5c0274bd421d7bb1' - None - '8086216@N08''
|
||||
16:23:01 [flickrannex-0.1.5] main : 'Photoset gitannex found: <Element 'photoset' at 0x869992c>'
|
||||
16:23:01 [flickrannex-0.1.5] main : 'Trying page: 1'
|
||||
16:23:01 [flickrannex-0.1.5] checkFile : 'GPGHMACSHA1--131f95d3bc932d23ef6af47cf49db3c04be4f0f9 - 72157634897264995L - '8086216@N08''
|
||||
16:23:01 [flickrannex-0.1.5] checkFile : 'No set exists, thus no files exists'
|
||||
16:23:01 [flickrannex-0.1.5] <module> : 'STOP: 1s'
|
||||
(fixing location log)
|
||||
** Based on the location log, walkthrough.sh~
|
||||
** was expected to be present, but its content is missing.
|
||||
failed
|
||||
(Recording state in git...)
|
||||
git-annex: fsck: 2 failed
|
||||
|
||||
\"\"\" ]]
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
As per IRC
|
||||
|
||||
22:13:10 < RichiH> joeyh: btw, i have been pondering a `git annex import --lazy` or some such which basically goes through a directory and deletes everything i find in the annex it run from
|
||||
22:50:39 < joeyh> not sure of the use case
|
||||
23:41:06 < RichiH> joeyh: the use case is "i have important a ton of data into my annexes. now, i am going through the usual crud of cp -ax'ed, rsync'ed, and other random 'new disk, move stuff around and just put a full dump over there' file dumps and would like to delete everything that's annexed already"
|
||||
23:41:33 < RichiH> joeyh: that would allow me to spend time on dealing with the files which are not yet annexed
|
||||
23:41:54 < RichiH> instead of verifying file after file which has been imported already
|
||||
23:43:19 < joeyh> have you tried just running git annex import in a subdirectory and then deleting the dups?
|
||||
23:45:34 < joeyh> or in a separate branch for that matter, which you could then merge in, etc
|
||||
23:54:08 < joeyh> Thinking anout it some more, it would need to scan the whole work tree to see what keys were there, and populate a lookup table. I prefer to avoid things that need git-annex to do such a large scan and use arbitrary amounts of memory.
|
||||
00:58:11 < RichiH> joeyh: that would force everything into the annex, though
|
||||
00:58:20 < RichiH> a plain import, that is
|
||||
00:58:53 < RichiH> in a usual data dump directory, there's tons of stuff i will never import
|
||||
00:59:00 < RichiH> i want to delete large portions of it
|
||||
00:59:32 < RichiH> but getting rid of duplicates first allows me to spend my time focused on stuff humans are good at: deciding
|
||||
00:59:53 < RichiH> whereas the computer can focus on stuff it's good at: mindless comparision of bits
|
||||
01:00:15 < RichiH> joeyh: as you're saying this is complex, maybe i need to rephrase
|
||||
01:01:40 < RichiH> what i envision is git annex import --foo to 1) decide what hashing algorithm should be used for this file 2) hash that file 3) look into the annex if that hash is annexed 3a) optionally verify numcopies within the annex 4) delete the file in the source directory
|
||||
01:01:47 < RichiH> and then move on to the next file
|
||||
01:02:00 < RichiH> if the hash does not exist in the annex, leave it alone
|
||||
01:02:50 < RichiH> if the hash exists in annex, but numcopies is not fulfilled, just import it as a normal import would
|
||||
01:03:50 < RichiH> that sounds quite easy, to me; in fact i will prolly script it if you decide not to implement it
|
||||
01:04:07 < RichiH> but i think it's useful for a _lot_ of people who migrate tons of data into annexes
|
||||
01:04:31 < RichiH> thus i would rather see this upstream and not hacked locally
|
||||
|
||||
The only failure mode I see in the above is "file has been dropped elsewhere, numcopies not fulfilled, but that info is not synched to the local repo, yet" -- This could be worked around by always importing the data.
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
|
||||
nickname="Richard"
|
||||
subject="comment 2"
|
||||
date="2013-08-02T07:21:50Z"
|
||||
content="""
|
||||
Using symmetric keys is significantly cheaper, computation-wise.
|
||||
|
||||
The scheme of encrypting symmetric keys with asymmetric ones is ancient, well-proven, and generally accepted as a good approach.
|
||||
|
||||
Using per-key files makes access control more fine-grained and is only a real performance issue once while creating the private key and a little bit every time more than one file needs to be decrypted as more than one symmetric key needs to be taken care of.
|
||||
"""]]
|
||||
Loading…
Add table
Add a link
Reference in a new issue