git-annex/doc/special_remotes.mdwn

git-annex can transfer data to and from configured git remotes.
Normally those remotes are normal git repositories (bare and non-bare;
local and remote), that store the file contents in their own git-annex
directory.

But, git-annex also extends git's concept of remotes, with these special
types of remotes. These can be used by git-annex to store and retrieve
the content of files.

* [[adb]] (for Android devices)
* [[Amazon_Glacier|glacier]]
* [[bittorrent]]
* [[bup]]
* [[ddar]]
* [[directory]]
* [[gcrypt]] (encrypted git repositories!)
* [[git-lfs]]
* [[hook]]
* [[rsync]]
* [[S3]] (Amazon S3, and other compatible services)
* [[tahoe]]
* [[tor]]
* [[web]]
* [[webdav]]
* [[git]]
* [[httpalso]]
* [[borg]]
* [[rclone]]

The above special remotes are built into git-annex, and can be used
to tie git-annex into many cloud services.

Here are specific instructions for using git-annex with various services:

* [[Amazon_Glacier|tips/using_Amazon_Glacier]]
* [[Amazon_S3|tips/using_Amazon_S3]]
* [[Backblaze B2|tips/using_Backblaze_B2]]
* [[Box.com|tips/using_box.com_as_a_special_remote]]
* [Ceph](https://github.com/mhameed/git-annex-remote-ceph)
* [chef-vault](https://github.com/3ofcoins/knife-annex/)
* [[Dropbox|tips/dropboxannex]]
* [[FTP|rclone]]
* [[Flickr|tips/flickrannex]]
* [Freenet and Siacoin Skynet](https://github.com/xloem/gitlakepy)
* [[Google Cloud Storage|tips/using_Google_Cloud_Storage]]
* [[Google Drive|tips/using_Google_Drive]]
* [[hubiC|tips/using_Hubic]]
* [[IMAP|forum/special_remote_for_IMAP]]
* [[tips/Internet_Archive_via_S3]]
* [[ipfs]]
* [[Jottacloud|rclone]]
* [[Mega|tips/megaannex]]
* [[Microsoft Azure Blob Storage|rclone]]
* [[Microsoft OneDrive|rclone]]
* [NNCP](https://git.sr.ht/~ehmry/git-annex-remote-nncp)
* [[OpenDrive|rclone]]
* [[Openstack Swift / Rackspace cloud files / Memset Memstore|rclone]]
* [[OwnCloud|tips/owncloudannex]]
* [[pCloud|rclone]]
* [[QingStor|rclone]]
* [[SFTP|rclone]]
* [[SkyDrive|tips/skydriveannex]]
* [smb / sftp](https://github.com/grawity/code/blob/master/net/git-annex-remote-gvfs)
* [[Usenet|forum/nntp__47__usenet special remote]]
* [[Yandex Disk|rclone]]

If a service is not mentioned above, it's worth checking if 
[rclone](https://rclone.org/) supports it, then you can use the
[[rclone_special_remote|rclone]].

Want to add support for something else? [[Write your own!|external]]

## Ways to use special remotes

There are many use cases for a special remote. You could use it as a
backup. You could use it to archive files offline in a drive with
encryption enabled so if the drive is stolen your data is not. You could
git annex move --to specialremote large files when your local drive is
getting full, and then git annex move the files back when free space is
again available. You could have one repository copy files to a special
remote, and then git annex get them on another repository, to transfer the
files between computers that do not communicate directly.

None of these use cases are particular to particular special remote types.
Most special remotes can all be used in these and other ways. It largely
doesn't matter for your use what underlying transport the special remote
uses.

## Setting up a special remote

To initialize a new special remote, use 
[[git-annex initremote|git-annex-initremote]]. See the documentation for
the special remote you want to use for details about configuration and
examples of how to initremote it.

Once a special remote has been initialized, other clones of the repository can
also enable it, by using [[git-annex enableremote|git-annex-enableremote]]
with the same name that was used to initialize it. (Run the command without
any name to get a list of available special remotes.)

Initializing or enabling a special remote adds it as a remote of your git
repository.

## Storing a git repository in a special remote

Most special remotes do not include a clone of the git repository
by default, so you can't use commands like `git push` and `git pull`
with them. (There are some exceptions like [[git-lfs]].)

But it is possible to store a git repository in many special remotes,
using the [[git-remote-annex]] command. This involves configuring
the remote with an "annex::" url. It's even possible to `git clone`
from a special remote using such an url. See the documentation of
[[git-remote-annex]] for details.

## Unused content on special remotes

Over time, special remotes can accumulate file content that is no longer
referred to by files in git. Normally, unused content in the current
repository is found by running `git annex unused`. To detect unused content
on special remotes, instead use `git annex unused --from`. Example:

	$ git annex unused --from mys3
	unused mys3 (checking for unused data...) 
	  Some annexed data on mys3 is not used by any files in this repository.
	    NUMBER  KEY
	    1       WORM-s3-m1301674316--foo
	  (To see where data was previously used, try: git log --stat -S'KEY')
	  (To remove unwanted data: git-annex dropunused --from mys3 NUMBER)
	$ git annex dropunused --from mys3 1
	dropunused 12948 (from mys3...) ok

## Removing special remotes

Like git remotes, a special remote can be removed from your repository
by using `git remote remove`. Note that does not delete the special remote,
or prevent other repositories from enabling or using it.

## Testing special remotes

To make sure that a special remote is working correctly, you can use the
[[git annex testremote|git-annex-testremote]] command. This expects you to
have set up the remote as usual, and it then runs a lot of tests, using
random data. It's particularly useful to test new implementations of
special remotes.