From bc6e6060147767250800e09b65a41305c798961b Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 31 Jan 2023 15:58:21 -0400 Subject: [PATCH] analysis --- ..._fb7a448a2a4231b08277741805c12c76._comment | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment diff --git a/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment new file mode 100644 index 0000000000..49eb41ff72 --- /dev/null +++ b/doc/bugs/S3_remote_errors_with_garage_endpoint/comment_5_fb7a448a2a4231b08277741805c12c76._comment @@ -0,0 +1,42 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 5""" + date="2023-01-31T18:45:08Z" + content=""" +Firstly, this code is working, as far as I know, when accessing AWS. +And I want to be very careful to avoid breaking that. So patching all +that out would have to be very carefully examined and/or tested. + +I've verified that the s3Endpoint is supposed to be a hostname. It's +used as such in s3SignQuery: + + (host, path) = case s3RequestStyle of + PathStyle -> ([Just s3Endpoint], [Just "/", fmap (`B8.snoc` '/') s3QBucket, urlEncodedS3QObject]) + +If s3Endpoint were just "garage" that would break. + +s3SignQuery also has `region = s3ExtractRegion s3Endpoint`. +s3ExtractRegion parses a hostname like +"s3-foo.amazonaws.com" to "foo", and that is used as the region +(or "scope" as you've called it). + +git-annex makes sure to set s3Endpoint to a hostname. When the +default AWS hostname is used, it converts the datacenter=foo value +to a hostname like s3-foo.amazonaws.com and sets s3Endpoint to that. +When some other hostname= is provided, it set s3Endpoints to +that hostname, ignoring the datacenter= value. + +This seems like the only thing git-annex *can* do. Your proposed +patch would make a configuration of "datacenter=us-west-1" set +s3Endpoint to "us-west-1" and s3SignQuery would use that as the +host. + +So I think this is probably a bug or shortcoming of the aws library. +It seems that to fix this, the aws library would need to have +a way to specify a "scope" or "region" separate from the s3Endpoint. +And git-annex would then need to expose that as something other +than datacenter= + +I've filed an issue on aws about this: + +"""]]