mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

deal with Amazon S3 breaking change for public=yes

Browse source 
* S3: Amazon S3 buckets created after April 2023 do not support ACLs,
  so public=yes cannot be used with them. Existing buckets configured
  with public=yes will keep working.
* S3: Allow setting publicurl=yes without public=yes, to support
  buckets that are configured with a Bucket Policy that allows public
  access.

Sponsored-by: Joshua Antonishen on Patreon
This commit is contained in:
Joey Hess 2023-07-21 13:48:49 -04:00
parent ddc7f36d53
commit 33ba537728
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 105 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

29
doc/bugs/S3_ACL_deprecation/comment_2_63b541a937cb34cc4c68504e00dc3fb8._comment Normal file
View file

@ -0,0 +1,29 @@
[[!comment format=mdwn
username="joey"
subject="""comment 2"""
date="2023-07-21T17:53:25Z"
content="""
This only affects new S3 buckets. Existing S3 buckets that were
created before April 2023 and were set up to allow public access should
keep working, including ACL settings when storing new files in them.
Per [Amazon's announcement](https://aws.amazon.com/about-aws/whats-new/2022/12/amazon-s3-automatically-enable-block-public-access-disable-access-control-lists-buckets-april-2023/),
"There is no change for existing buckets."
I've made `publicurl` orthogonal to `public`.
As for the idea of `HTTP HEAD` before trying to set the ACL,
the ACL is currently sent at past of the PutObject request. And
either there is not a way to change the ACL later, or the aws haskell library
is missing support for the API to do that.
While git-annex could HEAD without creds when publicyes=yes to verify that the
user has configured the bucket correctly, and at least warn about a
misconfiguration, that would add some overhead, and I guess if the user has not
configured the bucket correctly, they will notice in some other way eventually
and can fix its bucket policy after the fact. So I'm inclined not to do
that.
Instead I've simply depredated `public`, noting that it should not be set
on new buckets. The user will have to deal with setting up the Bucket
Policy themselves.
"""]]