more s3 docs

This commit is contained in:
Joey Hess 2011-03-28 02:12:05 -04:00
parent a7bd63eb01
commit 1878745a46
6 changed files with 63 additions and 37 deletions

2
debian/changelog vendored
View file

@ -1,6 +1,6 @@
git-annex (0.20110326) UNRELEASED; urgency=low
* Amazon is S3 now supported as a special type of remote.
* Amazon S3 is now supported as a special type of remote.
* Generalized remotes handling, laying groundwork for remotes that are
not regular git remotes.
* Provide a less expensive version of `git annex copy --to`, enabled

View file

@ -132,6 +132,16 @@ Many git-annex commands will stage changes for later `git commit` by you.
by uuid. To change the description of the current repository, use
"."
* s3bucket name description [datacenter host port]
Creates a bucket in Amazon S3. The bucket's name can be used
to configure git remote using the bucket.
The datacenter defaults to "US". Other values include "EU",
"us-west-1", and "ap-southeast-1".
To use a different, S3-compatable service, specify a host and port.
* fsck [path ...]
With no parameters, this command checks the whole annex for consistency,
@ -387,6 +397,25 @@ Here are all the supported configuration settings.
Default ssh and rsync options to use if a remote does not have
specific options.
* `remote.<name>.annex-s3-access-key-id`
Your S3 Access Key ID. Does not need to be kept private.
If not set, the environment variable `AWS_ACCESS_KEY_ID`
will be used.
* `remote.<name>.annex-s3-secret-access-key`
Your S3 Secret Access Key. This is a password.
If not set, the environment variable `AWS_SECRET_ACCESS_KEY`
will be used.
* `remote.<name>.annex-s3-storageclass`
Storage class to use when adding new content to S3. The default
is "STANDARD". If you have configured git-annex to preserve
multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY"
to save money.
* `annex.diskreserve`
Amount of disk space to reserve. Disk space is checked when transferring
@ -401,6 +430,8 @@ Here are all the supported configuration settings.
Automatically maintained, and used to automate upgrades between versions.
# CONFIGURATION VIA .gitattributes
The backend used when adding a new file to the annex can be configured

View file

@ -30,6 +30,17 @@ space and then the description through to the end of the line. Example:
e605dca6-446a-11e0-8b2a-002170d25c55 laptop
26339d22-446b-11e0-9101-002170d25c55 usb disk
## `git-annex/s3.log`
Associates the UUIDs of Amazon S3 buckets with a bucket nickname and connection
information. Example:
be72acb8-5901-11e0-b600-002170d25c55 mybucket s3.amazonaws.com 80
Note that the actual bucket name used on S3 in the above example
is "mybucket-be72acb8-5901-11e0-b600-002170d25c55". The UUID is included
in the bucket name to ensure it is globally unique.
## `.git-annex/trust.log`
Records the [[trust]] information for repositories. Does not exist unless

View file

@ -9,34 +9,4 @@ They cannot be used by other git commands though.
## Amazon S3
Stores file contents in a bucket in Amazon S3 or a similar service.
Example of configuring such a remote:
git config remote.mys3.annex-s3-bucket myannex
export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t"
That creates a remote named "mys3" using the bucket named "myannex",
which will be created if it doesn't already exist.
Here is the full set of configurable settings for Amazon S3.
Each setting can be configured on a per-remote basis in git-config,
or globally in an environment variable.
* `remote.$name.annex-s3-access-key-id` `ANNEX_S3_ACCESS_KEY_ID`
Your S3 Access Key ID. Does not need to be kept private.
* `remote.$name.annex-s3-secret-access-key` `ANNEX_S3_SECRET_ACCESS_KEY`
Your S3 Secret Access Key. This is a password. Usually stored in the
environment variable to avoid it being exposed.
* `remote.$name.annex-s3-host` `ANNEX_S3_HOST`
Host to connect to. Default is s3.amazonaws.com.
* `remote.$name.annex-s3-port` `ANNEX_S3_PORT`
Port to connect to. Default is 80.
* `remote.$name.annex-s3-datacenter` `ANNEX_S3_DATACENTER`
Name of the datacenter to use when creating a new bucket. Default is "US";
other valid values include "EU", "us-west-1", and "ap-southeast-1".
* `remote.$name.annex-s3-storageclass` `ANNEX_S3_STORAGECLASS`
Name of storage class to use when adding new content to the bucket.
Default is "STANDARD". If you have configured git-annex to preserve
multiple [[copies]], consider setting this to "REDUCED_REDUNDANCY" to
save money.
See [[walkthrough/using_Amazon_S3]] for examples.

View file

@ -1,3 +1,5 @@
[[done]]
Support Amazon S3 as a file storage backend.
There's a haskell library that looks good. Not yet in Debian.

View file

@ -2,14 +2,26 @@ git-annex extends git's usual remotes with some [[special_remotes]], that
are not git repositories. This way you can set up a remote using say,
Amazon S3, and use git-annex to transfer files into the cloud.
# git config remote.mys3.annex-s3-bucket myannex
# export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
# export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t"
First, export your S3 credentials:
export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t"
Next, create a bucket, giving it a name and a description:
git annex s3bucket mybucket "my Amazon S3 bucket"
s3bucket (creating mybucket...) ok
Finally, configure a git remote to use the bucket you created:
git config remote.mys3.annex-s3-bucket mybucket
Now the remote can be used like any other remote.
# git annex copy my_cool_big_file --to mys3
copy my_cool_big_file (to mys3...) ok
# git annex move video/hackity_hack_and_kaxxt.mov --to mys3
move video/hackity_hack_and_kaxxt.mov (to mys3...) ok
An Amazon S3 remote works just like a ssh remote, except it does not have
a git repository at the other end, and it costs you money. :) For full
details about setting them up, see [[special_remotes]].
a git repository at the other end, and it costs you money. :)