Commit graph

15 commits

Author SHA1 Message Date
Joey Hess
57872b457b pass metadata headers and storage class to S3 when putting objects 2014-08-09 14:44:53 -04:00
Joey Hess
6fcca2f13e WIP converting S3 special remote from hS3 to aws library
Currently, initremote works, but not the other operations. They should be
fairly easy to add from this base.

Also, https://github.com/aristidb/aws/issues/119 blocks internet archive
support.

Note that since http-conduit is used, this also adds https support to S3.
Although git-annex encrypts everything anyway, so that may not be extremely
useful. It is not enabled by default, because existing S3 special remotes
have port=80 in their config. Setting port=443 will enable it.

This commit was sponsored by Daniel Brockman.
2014-08-08 19:00:53 -04:00
Joey Hess
32e4368377 S3: support chunking
The assistant defaults to 1MiB chunk size for new S3 special remotes.
Which will work around a couple of bugs:
  http://git-annex.branchable.com/bugs/S3_memory_leaks/
  http://git-annex.branchable.com/bugs/S3_upload_not_using_multipart/
2014-08-02 15:51:58 -04:00
Joey Hess
47d0219ac8 update docs for changed initremote encryption syntax 2013-09-04 23:46:50 -04:00
Joey Hess
85d83e7756 To enable an existing special remote, the new enableremote command must be used. The initremote command now is used only to create new special remotes. 2013-04-26 18:22:52 -04:00
Joey Hess
9221e62d87 Allow controlling whether login credentials for S3 and webdav are committed to the repository, by setting embedcreds=yes|no when running initremote. 2012-11-19 17:32:58 -04:00
Joey Hess
e4bf74a965 store S3 creds in a 600 mode file inside the local git repo 2012-09-26 14:42:32 -04:00
Joey Hess
ad4e152fd6 S3: Add fileprefix setting. 2012-08-09 13:54:54 -04:00
https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY
8cca7616ae 2012-05-30 00:25:22 +00:00
Joey Hess
f97009fffe correct S3 environment variable names 2012-05-29 15:01:36 -04:00
Joey Hess
67c9f84a1f fix broken links 2011-11-08 12:23:03 -04:00
Joey Hess
27285adc20 link 2011-05-16 11:55:07 -04:00
Joey Hess
1d2984441c add a few tweaks to make it easy to use the Internet Archive's variant of S3
In particular, munge key filenames to comply with the IA's filename limits,
disable encryption, support their nonstandard way of creating buckets, and
allow x-amz-* headers to be specified in initremote to set item metadata.

Still TODO: initremote does not handle multiword metadata headers right.
2011-05-16 11:20:35 -04:00
Joey Hess
e259c86975 cleanup 2011-05-16 02:21:40 -04:00
Joey Hess
647f7cf47c added documentation for using the Internet Archive as a remote via S3
Renamed Amazon_S3 page to just S3.
2011-05-16 02:07:59 -04:00
Renamed from doc/special_remotes/Amazon_S3.mdwn (Browse further)