125 lines
4.9 KiB
Markdown
125 lines
4.9 KiB
Markdown
git-annex can transfer data to and from configured git remotes.
|
|
Normally those remotes are normal git repositories (bare and non-bare;
|
|
local and remote), that store the file contents in their own git-annex
|
|
directory.
|
|
|
|
But, git-annex also extends git's concept of remotes, with these special
|
|
types of remotes. These can be used by git-annex to store and retrieve
|
|
the content of files. They cannot be used by other git commands, and
|
|
the git history is not stored in them.
|
|
|
|
* [[adb]] (for Android devices)
|
|
* [[Amazon_Glacier|glacier]]
|
|
* [[bittorrent]]
|
|
* [[bup]]
|
|
* [[ddar]]
|
|
* [[directory]]
|
|
* [[gcrypt]] (encrypted git repositories!)
|
|
* [[git-lfs]]
|
|
* [[hook]]
|
|
* [[rsync]]
|
|
* [[S3]] (Amazon S3, and other compatible services)
|
|
* [[tahoe]]
|
|
* [[tor]]
|
|
* [[web]]
|
|
* [[webdav]]
|
|
* [[git]]
|
|
* [[xmpp]]
|
|
|
|
The above special remotes are built into git-annex, and can be used
|
|
to tie git-annex into many cloud services.
|
|
|
|
Here are specific instructions for using git-annex with various services:
|
|
|
|
* [[Amazon_Glacier|tips/using_Amazon_Glacier]]
|
|
* [[Amazon_S3|tips/using_Amazon_S3]]
|
|
* [[Backblaze B2|tips/using_Backblaze_B2]]
|
|
* [[Box.com|tips/using_box.com_as_a_special_remote]]
|
|
* [Ceph](https://github.com/mhameed/git-annex-remote-ceph)
|
|
* [chef-vault](https://github.com/3ofcoins/knife-annex/)
|
|
* [[Dropbox|tips/dropboxannex]]
|
|
* [[FTP|rclone]]
|
|
* [[Flickr|tips/flickrannex]]
|
|
* [Freenet](https://github.com/xloem/gitlakepy)
|
|
* [[Google Cloud Storage|tips/using_Google_Cloud_Storage]]
|
|
* [[Google Drive|tips/using_Google_Drive]]
|
|
* [[hubiC|tips/using_Hubic]]
|
|
* [[IMAP|forum/special_remote_for_IMAP]]
|
|
* [[tips/Internet_Archive_via_S3]]
|
|
* [[ipfs]]
|
|
* [[Jottacloud|rclone]]
|
|
* [[Mega|tips/megaannex]]
|
|
* [[Microsoft Azure Blob Storage|rclone]]
|
|
* [[Microsoft OneDrive|rclone]]
|
|
* [[OpenDrive|rclone]]
|
|
* [[Openstack Swift / Rackspace cloud files / Memset Memstore|rclone]]
|
|
* [[OwnCloud|tips/owncloudannex]]
|
|
* [[pCloud|rclone]]
|
|
* [[QingStor|rclone]]
|
|
* [[SFTP|rclone]]
|
|
* [[SkyDrive|tips/skydriveannex]]
|
|
* [smb / sftp](https://github.com/grawity/code/blob/master/net/git-annex-remote-gvfs)
|
|
* [[Usenet|forum/nntp__47__usenet special remote]]
|
|
* [[Yandex Disk|rclone]]
|
|
|
|
If a service is not mentioned above, it's worth checking if
|
|
[rclone](https://rclone.org/) supports it, then you can use the
|
|
[[rclone_special_remote|rclone]].
|
|
|
|
Want to add support for something else? [[Write your own!|external]]
|
|
|
|
## Ways to use special remotes
|
|
|
|
There are many use cases for a special remote. You could use it as a
|
|
backup. You could use it to archive files offline in a drive with
|
|
encryption enabled so if the drive is stolen your data is not. You could
|
|
git annex move --to specialremote large files when your local drive is
|
|
getting full, and then git annex move the files back when free space is
|
|
again available. You could have one repository copy files to a special
|
|
remote, and then git annex get them on another repository, to transfer the
|
|
files between computers that do not communicate directly.
|
|
|
|
The git-annex assistant makes it easy to set up rsync remotes using this
|
|
last scenario, which is referred to as a transfer repository, and arranges
|
|
to drop files from the transfer repository once they have been transferred
|
|
to all known clients.
|
|
|
|
None of these use cases are particular to particular special remote types.
|
|
Most special remotes can all be used in these and other ways. It largely
|
|
doesn't matter for your use what underlying transport the special remote
|
|
uses.
|
|
|
|
## Unused content on special remotes
|
|
|
|
Over time, special remotes can accumulate file content that is no longer
|
|
referred to by files in git. Normally, unused content in the current
|
|
repository is found by running `git annex unused`. To detect unused content
|
|
on special remotes, instead use `git annex unused --from`. Example:
|
|
|
|
$ git annex unused --from mys3
|
|
unused mys3 (checking for unused data...)
|
|
Some annexed data on mys3 is not used by any files in this repository.
|
|
NUMBER KEY
|
|
1 WORM-s3-m1301674316--foo
|
|
(To see where data was previously used, try: git log --stat -S'KEY')
|
|
(To remove unwanted data: git-annex dropunused --from mys3 NUMBER)
|
|
$ git annex dropunused --from mys3 1
|
|
dropunused 12948 (from mys3...) ok
|
|
|
|
## Testing special remotes
|
|
|
|
To make sure that a special remote is working correctly, you can use the
|
|
`git annex testremote` command. This expects you to have set up the remote
|
|
as usual, and it then runs a lot of tests, using random data. It's
|
|
particularly useful to test new implementations of special remotes.
|
|
|
|
By default it will upload and download files of around 1MiB to the remote
|
|
it tests; the `--size` parameter can adjust it to test using smaller files.
|
|
|
|
It's safe to use this command even when you're already storing data in a
|
|
remote; it won't touch your existing files stored on the remote.
|
|
|
|
For most remotes, it also won't bloat the remote with any data, since
|
|
it cleans up the stuff it uploads. However, the bup, ddar, and tahoe
|
|
special remotes don't support removal of uploaded files, so be careful
|
|
with those.
|