Fix failure to build with aws-0.13.0 and finish nearline support.
* Fix failure to build with aws-0.13.0. * When built with aws-0.13.0, the S3 special remote can be used to create google nearline buckets, by setting storageclass=NEARLINE.
This commit is contained in:
parent
e9c7cc316e
commit
4153507864
6 changed files with 50 additions and 2 deletions
10
Remote/S3.hs
10
Remote/S3.hs
|
@ -330,10 +330,20 @@ genBucket c u = do
|
||||||
(bucket info)
|
(bucket info)
|
||||||
(acl info)
|
(acl info)
|
||||||
locconstraint
|
locconstraint
|
||||||
|
#if MIN_VERSION_aws(0,13,0)
|
||||||
|
storageclass
|
||||||
|
#endif
|
||||||
writeUUIDFile c u info h
|
writeUUIDFile c u info h
|
||||||
|
|
||||||
locconstraint = mkLocationConstraint $ T.pack datacenter
|
locconstraint = mkLocationConstraint $ T.pack datacenter
|
||||||
datacenter = fromJust $ M.lookup "datacenter" c
|
datacenter = fromJust $ M.lookup "datacenter" c
|
||||||
|
#if MIN_VERSION_aws(0,13,0)
|
||||||
|
-- "NEARLINE" as a storage class when creating a bucket is a
|
||||||
|
-- nonstandard extension of Google Cloud Storage.
|
||||||
|
storageclass = case getStorageClass c of
|
||||||
|
sc@(S3.OtherStorageClass "NEARLINE") -> Just sc
|
||||||
|
_ -> Nothing
|
||||||
|
#endif
|
||||||
|
|
||||||
{- Writes the UUID to an annex-uuid file within the bucket.
|
{- Writes the UUID to an annex-uuid file within the bucket.
|
||||||
-
|
-
|
||||||
|
|
3
debian/changelog
vendored
3
debian/changelog
vendored
|
@ -9,6 +9,9 @@ git-annex (5.20151020) UNRELEASED; urgency=medium
|
||||||
* Catch up with current git behavior when both repo and repo.git exist;
|
* Catch up with current git behavior when both repo and repo.git exist;
|
||||||
it seems it now prefers repo in this case, although historically it may
|
it seems it now prefers repo in this case, although historically it may
|
||||||
have preferred repo.git.
|
have preferred repo.git.
|
||||||
|
* Fix failure to build with aws-0.13.0.
|
||||||
|
* When built with aws-0.13.0, the S3 special remote can be used to create
|
||||||
|
google nearline buckets, by setting storageclass=NEARLINE.
|
||||||
|
|
||||||
-- Joey Hess <id@joeyh.name> Mon, 19 Oct 2015 17:00:21 -0400
|
-- Joey Hess <id@joeyh.name> Mon, 19 Oct 2015 17:00:21 -0400
|
||||||
|
|
||||||
|
|
|
@ -90,3 +90,5 @@ initremote cloud (checking bucket...) [2015-05-31 17:38:21 EDT] String to sign:
|
||||||
[2015-05-31 17:38:22 EDT] Response metadata: S3: request ID=<none>, x-amz-id-2=<none>
|
[2015-05-31 17:38:22 EDT] Response metadata: S3: request ID=<none>, x-amz-id-2=<none>
|
||||||
git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "InvalidArgument", s3ErrorMessage = "Invalid argument.", s3ErrorResource = Nothing, s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing}
|
git-annex: S3Error {s3StatusCode = Status {statusCode = 400, statusMessage = "Bad Request"}, s3ErrorCode = "InvalidArgument", s3ErrorMessage = "Invalid argument.", s3ErrorResource = Nothing, s3ErrorHostId = Nothing, s3ErrorAccessKeyId = Nothing, s3ErrorStringToSign = Nothing}
|
||||||
"""]]
|
"""]]
|
||||||
|
|
||||||
|
> [[done]], see comments --[[Joey]]
|
||||||
|
|
|
@ -0,0 +1,11 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 5"""
|
||||||
|
date="2015-11-02T14:55:54Z"
|
||||||
|
content="""
|
||||||
|
aws-0.13.0 was released with the ncessary support. git-annex will now pass
|
||||||
|
the NEARLINE storage class when creating a bucket.
|
||||||
|
|
||||||
|
Note that most builds of git-annex are not yet made with that version of
|
||||||
|
aws, but it will trickle out to the build systems with time.
|
||||||
|
"""]]
|
|
@ -43,9 +43,14 @@ the S3 remote.
|
||||||
|
|
||||||
When using Amazon S3, if you have configured git-annex to preserve
|
When using Amazon S3, if you have configured git-annex to preserve
|
||||||
multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY"
|
multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY"
|
||||||
to save money. Or, if the remote will be used for backup or archival,
|
to save money.
|
||||||
|
|
||||||
|
Or, if the remote will be used for backup or archival,
|
||||||
and so its files are Infrequently Accessed, "STANDARD_IA" is also a
|
and so its files are Infrequently Accessed, "STANDARD_IA" is also a
|
||||||
good choice to save money.
|
good choice to save money. (Requires a git-annex built with aws-0.13.0)
|
||||||
|
|
||||||
|
When using Google Cloud Storage, to make a nearline bucket, set this to
|
||||||
|
"NEARLINE". (Requires a git-annex built with aws-0.13.0)
|
||||||
|
|
||||||
Note that changing the storage class of an existing S3 remote will
|
Note that changing the storage class of an existing S3 remote will
|
||||||
affect new objects sent to the remote, but not objects already
|
affect new objects sent to the remote, but not objects already
|
||||||
|
|
|
@ -0,0 +1,17 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 5"""
|
||||||
|
date="2015-11-02T14:59:08Z"
|
||||||
|
content="""
|
||||||
|
Nice job bgilbert on the external special remote.
|
||||||
|
|
||||||
|
git-annex's S3 special remote also supports nearline now:
|
||||||
|
|
||||||
|
git annex initremote cloud type=S3 encryption=none host=storage.googleapis.com port=80 storageclass=NEARLINE
|
||||||
|
|
||||||
|
For the nearline support to work, git-annex has to be built with version 0.13.0 of
|
||||||
|
the haskell aws library.
|
||||||
|
|
||||||
|
But, it seems like bgilbert's external special remote is easier to set up,
|
||||||
|
due to not needing to configure Google Cloud Storage for interoperable access.
|
||||||
|
"""]]
|
Loading…
Add table
Add a link
Reference in a new issue