mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Added a comment

Browse source
This commit is contained in:
https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ 2015-04-25 08:36:25 +00:00 • committed by admin
parent 4a8e25fb8d
commit c7d105c594
1 changed files with 88 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

88
doc/todo/enable_fsck_--fast_for_S3_remotes/comment_3_b7402508dfc7bbbd09382692aa740c39._comment Normal file
View file

@ -0,0 +1,88 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawnSenxKyE_2Z6Wb-EBMO8FciyRywjx1ZiQ"
nickname="Walter"
subject="comment 3"
date="2015-04-25T08:36:25Z"
content="""
I also obtain the expected result if a file is thought to be present, but isn't.
[[!format sh \"\"\"
> git annex setpresentkey `git annex lookupkey notpresent` be992080-b1db-11e1-8f79-1b10bb4092ef 1
setpresentkey SHA256E-s37--2f9b7d77d43f49b59fb00148bc1b3d31a887ba717c988be55b9377d403a91f53 ok
> git annex fsck --debug -f cloud --fast notpresent
[2015-04-25 09:24:25 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"]
[2015-04-25 09:24:25 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2015-04-25 09:24:25 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..0547dfd2d61ff9a24a08ff97cf4984bebbd4f0f1\",\"-n1\",\"--pretty=%H\"]
[2015-04-25 09:24:25 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..b43028d651236ce59a3e47240bead91cdbfc37ea\",\"-n1\",\"--pretty=%H\"]
[2015-04-25 09:24:25 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"]
[2015-04-25 09:24:25 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"notpresent\"]
fsck notpresent [2015-04-25 09:24:25 BST] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--decrypt\"]
(checking cloud...) [2015-04-25 09:24:25 BST] String to sign: \"HEAD\n\n\nSat, 25 Apr 2015 08:24:25 GMT\n/BUCKET/GPGHMACSHA1--e46ce4a11bc47622fb40affac818d6128bcd94bd\"
[2015-04-25 09:24:25 BST] Host: \"BUCKET.s3-ap-southeast-2.amazonaws.com\"
[2015-04-25 09:24:25 BST] Path: \"/GPGHMACSHA1--e46ce4a11bc47622fb40affac818d6128bcd94bd\"
[2015-04-25 09:24:25 BST] Query string: \"\"
[2015-04-25 09:24:25 BST] Response status: Status {statusCode = 404, statusMessage = \"Not Found\"}
[2015-04-25 09:24:25 BST] Response header 'x-amz-request-id': 'D562150974717AB1'
[2015-04-25 09:24:25 BST] Response header 'x-amz-id-2': 'Geq6BKC3Sg1rUuhgOHE7fOa5fq+L5ecShidW0ktI/ri3zNXKudhK5O5qT2qmUraJP6BCzDFuj1Q='
[2015-04-25 09:24:25 BST] Response header 'Content-Type': 'application/xml'
[2015-04-25 09:24:25 BST] Response header 'Transfer-Encoding': 'chunked'
[2015-04-25 09:24:25 BST] Response header 'Date': 'Sat, 25 Apr 2015 08:24:24 GMT'
[2015-04-25 09:24:25 BST] Response header 'Server': 'AmazonS3'
[2015-04-25 09:24:25 BST] Response metadata: S3: request ID=<none>, x-amz-id-2=<none>
(fixing location log)
** Based on the location log, notpresent
** was expected to be present, but its content is missing.
failed
\"\"\"]]
That leaves only one case: when the file isn't thought to be in cloud, but is. For completeness
[[!format sh \"\"\"
> git annex copy --to cloud notpresent
copy notpresent (checking cloud...) (to cloud...)
ok
(recording state in git...)
> git annex setpresentkey `git annex lookupkey notpresent` be992080-b1db-11e1-8f79-1b10bb4092ef 0
setpresentkey SHA256E-s37--2f9b7d77d43f49b59fb00148bc1b3d31a887ba717c988be55b9377d403a91f53 ok
> git annex fsck --debug -f cloud --fast notpresent
[2015-04-25 09:26:33 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"git-annex\"]
[2015-04-25 09:26:33 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2015-04-25 09:26:33 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..dca379d2631cd39bd205ccb7d6c192faea7c05c5\",\"-n1\",\"--pretty=%H\"]
[2015-04-25 09:26:33 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"log\",\"refs/heads/git-annex..b43028d651236ce59a3e47240bead91cdbfc37ea\",\"-n1\",\"--pretty=%H\"]
[2015-04-25 09:26:33 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"cat-file\",\"--batch\"]
[2015-04-25 09:26:33 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"notpresent\"]
fsck notpresent [2015-04-25 09:26:33 BST] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--decrypt\"]
(checking cloud...) [2015-04-25 09:26:33 BST] String to sign: \"HEAD\n\n\nSat, 25 Apr 2015 08:26:33 GMT\n/BUCKET/GPGHMACSHA1--e46ce4a11bc47622fb40affac818d6128bcd94bd\"
[2015-04-25 09:26:33 BST] Host: \"BUCKET.s3-ap-southeast-2.amazonaws.com\"
[2015-04-25 09:26:33 BST] Path: \"/GPGHMACSHA1--e46ce4a11bc47622fb40affac818d6128bcd94bd\"
[2015-04-25 09:26:33 BST] Query string: \"\"
[2015-04-25 09:26:34 BST] Response status: Status {statusCode = 200, statusMessage = \"OK\"}
[2015-04-25 09:26:34 BST] Response header 'x-amz-id-2': '4Ti/62fBMzjW0woyrX5C++tQUw4uV97bbowjSiCkUNI6X2bAt+JCKbRYvZf/Is1QSY6SI2Aqgv4='
[2015-04-25 09:26:34 BST] Response header 'x-amz-request-id': '9311809D4C8485FD'
[2015-04-25 09:26:34 BST] Response header 'Date': 'Sat, 25 Apr 2015 08:26:35 GMT'
[2015-04-25 09:26:34 BST] Response header 'Last-Modified': 'Sat, 25 Apr 2015 08:26:22 GMT'
[2015-04-25 09:26:34 BST] Response header 'ETag': '\"c5c3c0f720110210e73c7bf962d76390\"'
[2015-04-25 09:26:34 BST] Response header 'Accept-Ranges': 'bytes'
[2015-04-25 09:26:34 BST] Response header 'Content-Type': 'binary/octet-stream'
[2015-04-25 09:26:34 BST] Response header 'Content-Length': '99'
[2015-04-25 09:26:34 BST] Response header 'Server': 'AmazonS3'
[2015-04-25 09:26:34 BST] Response metadata: S3: request ID=<none>, x-amz-id-2=<none>
failed to download file from remote
(fixing location log) failed
[2015-04-25 09:26:34 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"]
[2015-04-25 09:26:34 BST] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-index\",\"-z\",\"--index-info\"]
[2015-04-25 09:26:34 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
(recording state in git...)
[2015-04-25 09:26:34 BST] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"write-tree\"]
[2015-04-25 09:26:34 BST] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"commit-tree\",\"83cec04d148757f98565eacda236d6e9dbd48678\",\"--no-gpg-sign\",\"-p\",\"refs/heads/git-annex\"]
[2015-04-25 09:26:34 BST] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"update-ref\",\"refs/heads/git-annex\",\"31d4a714f6977197029faf23b099ea32a298be59\"]
git-annex: fsck: 1 failed
\"\"\"]]
This correctly determines that the file is present, and updates the location log. But I don't understand why the message ``failed to download file from remote`` is used (which is also used when a file is present, and thought to be present). For a fast fsck it shouldn't be trying to download the file. Also, I don't think this is specific to S3, I expect any remote will have the same behaviour.
"""]]