tagged the past 2 years of open todos and followed up to a few of them

also moved some that were really bug reports to bugs/ and closed a
couple
This commit is contained in:
Joey Hess 2020-01-30 15:22:05 -04:00
parent c08d5612ee
commit cffa2446e8
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
107 changed files with 406 additions and 4 deletions

View file

@ -0,0 +1,29 @@
I tried to use the S3 special remote to access DigitalOcean's Spaces API. [Their docs](https://developers.digitalocean.com/documentation/spaces/) suggest that this should be possible. However, it doesn't work.
The command I ran, with key removed:
git annex --debug initremote xamsi-everything type=S3 protocol=https host=sfo2.digitaloceanspaces.com datacenter=sfo2 chunk=64MiB encryption=hybrid keyid=XXX
The non-debug output, in full, with key removed:
initremote xamsi-everything (encryption setup) (to gpg keys: XXX) (checking bucket...) (creating bucket in sfo2...)
git-annex: XmlException {xmlErrorMessage = "Missing error Message"}
failed
git-annex: initremote: 1 failed
The debug output of the part that breaks, again with key material removed:
(creating bucket in sfo2...) [2019-10-15 08:40:41.119524792] String to sign: "PUT\n\n\nTue, 15 Oct 2019 15:40:41 GMT\n/xamsi-everything-a36e2044-07ac-4d85-8450-e5760c897a9b/"
[2019-10-15 08:40:41.119586065] Host: "xamsi-everything-a36e2044-07ac-4d85-8450-e5760c897a9b.sfo2.digitaloceanspaces.com"
[2019-10-15 08:40:41.119639648] Path: "/"
[2019-10-15 08:40:41.119683721] Query string: ""
[2019-10-15 08:40:41.11972899] Header: [("Date","Tue, 15 Oct 2019 15:40:41 GMT"),("Authorization","AWS XXX")]
[2019-10-15 08:40:41.119846915] Body: "<?xml version=\"1.0\" encoding=\"UTF-8\"?><CreateBucketConfiguration xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><LocationConstraint>sfo2</LocationConstraint></CreateBucketConfiguration>"
[2019-10-15 08:40:41.174450718] Response status: Status {statusCode = 403, statusMessage = "Forbidden"}
[2019-10-15 08:40:41.174566002] Response header 'Content-Length': '190'
[2019-10-15 08:40:41.174627301] Response header 'x-amz-request-id': 'tx0000000000001c5b175eb-005da5e879-23e283-sfo2a'
[2019-10-15 08:40:41.174685597] Response header 'Accept-Ranges': 'bytes'
[2019-10-15 08:40:41.174730858] Response header 'Content-Type': 'application/xml'
[2019-10-15 08:40:41.174776256] Response header 'Date': 'Tue, 15 Oct 2019 15:40:41 GMT'
[2019-10-15 08:40:41.174821726] Response header 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains; preload'
[2019-10-15 08:40:41.174984394] Response metadata: S3: request ID=tx0000000000001c5b175eb-005da5e879-23e283-sfo2a, x-amz-id-2=<none>

View file

@ -0,0 +1,20 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2020-01-06T17:41:21Z"
content="""
This is either a bug in <http://hackage.haskell.org/package/aws> or
Digital Ocean's implementation of the S3 protocol. I don't know which,
but I do know it has to be fixed in one of those two place, and not
in git-annex.
The fact that the aws library can't even parse an error message out of
their response kind of says something.
But the real problem seems to
While they claim to support V2 signatures,
my guess is there's a problem with their support for V2, since they're
mostly dealing with V4. The aws library's support for V4 is experimental
and [apparently buggy](https://github.com/aristidb/aws/issues/262), but
as git-annex uses it, it will only use V2.
"""]]

View file

@ -0,0 +1,50 @@
Hello.
What does this log mean? It seems to tell" "success", then "openDirStream" fails, then "1 failed". What failed?
Context is in [todo/git annex repair: performance can be abysmal, huge improvements possible](https://git-annex.branchable.com/ikiwiki.cgi?do=goto&page=todo%2Fgit_annex_repair__58___performance_can_be_abysmal__44___huge_improvements_possible)
fatal: bad object refs/heads/git-annex
fatal: bad object refs/heads/git-annex
fatal: bad object refs/heads/git-annex
error: Could not read somehashA
fatal: Failed to traverse parents of commit somehashB
error: Could not read somehashA
fatal: Failed to traverse parents of commit somehashB
error: Could not read somehashA
fatal: Failed to traverse parents of commit somehashB
error: Could not read somehashA
fatal: Failed to traverse parents of commit somehashB
Deleted these local branches, which could not be recovered due to missing objects:
refs/heads/master
refs/heads/git-annex
You currently have refs/heads/master checked out. You may have staged changes in the index that can be committed to recover the lost state of this branch!
Successfully recovered repository!
Please carefully check that the changes mentioned above are ok..
git-annex: .git/annex/journal/: openDirStream: does not exist (No such file or directory)
failed
git-annex: repair: 1 failed
The fact is: this repo is a plain git clone of a git annex repository.
There is no `.git/annex` directory there before `git-annex-repair` is run.
After it ran, there is a `.git/annex` directory with that content:
total 24
drwxrwxr-x 3 4096 Jul 22 15:41 .
drwxrwxr-x 9 4096 Jul 23 07:24 ..
-rw-rw-r-- 1 65 Jul 20 11:59 index
-rw-rw-r-- 1 41 Jul 20 11:59 index.lck
-rw-rw-r-- 1 0 Jul 22 15:41 journal.lck
-rw-rw-r-- 1 211 Jul 20 11:59 mergedrefs
drwxrwxr-x 2 4096 Jul 22 15:41 misctmp
Perhaps git-annex-repair gets confused when recovering a repository that is a plain git clone of a git annex repository?
I did that because annexed objects are 1.7TB big here, so I wanted a local copy of pure git part only to perform repair of the repo, and propagate things somehow the objects at a later stage.
I'll keep the repo lying around for a few days, maybe weeks, if some experiment or further feedback is needed.
Thank you for your attention.