2011-05-16 06:07:59 +00:00
|
|
|
[The Internet Archive](http://www.archive.org/) allows members to upload
|
|
|
|
|
collections using an Amazon S3
|
|
|
|
|
[compatible API](http://www.archive.org/help/abouts3.txt), and this can
|
|
|
|
|
be used with git-annex's [[special_remotes/S3]] support.
|
|
|
|
|
|
2011-05-16 15:52:33 +00:00
|
|
|
So, you can locally archive things with git-annex, define remotes that
|
|
|
|
|
correspond to "items" at the Internet Archive, and use git-annex to upload
|
|
|
|
|
your files to there. Of course, your use of the Internet Archive must
|
|
|
|
|
comply with their [terms of service](http://www.archive.org/about/terms.php).
|
2011-05-16 06:07:59 +00:00
|
|
|
|
2013-04-26 05:00:58 +00:00
|
|
|
A nice added feature is that whenever git-annex sends a file to the
|
|
|
|
|
Internet Archive, it records its url, the same as if you'd run `git annex
|
|
|
|
|
addurl`. So any users who can clone your repository can download the files
|
|
|
|
|
from archive.org, without needing any login or password info. This makes
|
|
|
|
|
the Internet Archive a nice way to publish the large files associated with
|
|
|
|
|
a public git repository.
|
|
|
|
|
|
|
|
|
|
----
|
|
|
|
|
|
2011-05-16 06:07:59 +00:00
|
|
|
Sign up for an account, and get your access keys here:
|
|
|
|
|
<http://www.archive.org/account/s3.php>
|
|
|
|
|
|
|
|
|
|
# export AWS_ACCESS_KEY_ID=blahblah
|
|
|
|
|
# export AWS_SECRET_ACCESS_KEY=xxxxxxx
|
|
|
|
|
|
2011-05-16 15:20:30 +00:00
|
|
|
Specify `host=s3.us.archive.org` when doing `initremote` to set up
|
|
|
|
|
a remote at the Archive. This will enable a special Internet Archive mode:
|
|
|
|
|
Encryption is not allowed; you are required to specify a bucket name
|
2011-05-16 16:12:03 +00:00
|
|
|
rather than having git-annex pick a random one; and you can optionally
|
2011-05-16 15:20:30 +00:00
|
|
|
specify `x-archive-meta*` headers to add metadata as explained in their
|
|
|
|
|
[documentation](http://www.archive.org/help/abouts3.txt).
|
|
|
|
|
|
2011-05-16 15:52:33 +00:00
|
|
|
[[!template id=note text="""
|
2013-10-09 22:48:18 +00:00
|
|
|
/!\ There seems to be a [[bug|bugs/S3 buckets with capital letters breaks authentication]] in either hS3 or the archive that breaks
|
2011-05-16 15:52:33 +00:00
|
|
|
authentication when the bucket name contains spaces or upper-case letters..
|
|
|
|
|
use all lowercase and no spaces when making the bucket with `initremote`.
|
|
|
|
|
"""]]
|
|
|
|
|
|
|
|
|
|
# git annex initremote archive-panama type=S3 \
|
|
|
|
|
host=s3.us.archive.org bucket=panama-canal-lock-blueprints \
|
2011-05-16 15:20:30 +00:00
|
|
|
x-archive-meta-mediatype=texts x-archive-meta-language=eng \
|
|
|
|
|
x-archive-meta-title="original Panama Canal lock design blueprints"
|
2011-05-16 17:10:26 +00:00
|
|
|
initremote archive-panama (Internet Archive mode) ok
|
2011-05-16 17:33:33 +00:00
|
|
|
# git annex describe archive-panama "a man, a plan, a canal: panama"
|
2011-05-16 06:07:59 +00:00
|
|
|
describe archive-panama ok
|
|
|
|
|
|
|
|
|
|
Then you can annex files and copy them to the remote as usual:
|
|
|
|
|
|
2011-05-16 15:46:34 +00:00
|
|
|
# git annex add photo1.jpeg --backend=SHA1E
|
|
|
|
|
add photo1.jpeg (checksum...) ok
|
2011-05-16 15:20:30 +00:00
|
|
|
# git annex copy photo1.jpeg --fast --to archive-panama
|
|
|
|
|
copy (to archive-panama...) ok
|
|
|
|
|
|
2011-05-16 15:46:34 +00:00
|
|
|
Note the use of the SHA1E [[backend|backends]]. It makes most sense
|
|
|
|
|
to use the WORM or SHA1E backend for files that will be stored in
|
|
|
|
|
the Internet Archive, since the key name will be exposed as the filename
|
|
|
|
|
there, and since the Archive does special processing of files based on
|
|
|
|
|
their extension.
|
