2012-04-11 18:22:59 +00:00
|
|
|
git-annex can transfer data to and from configured git remotes.
|
2011-08-17 01:12:48 +00:00
|
|
|
Normally those remotes are normal git repositories (bare and non-bare;
|
2012-04-11 18:22:59 +00:00
|
|
|
local and remote), that store the file contents in their own git-annex
|
2011-08-17 01:12:48 +00:00
|
|
|
directory.
|
2011-03-28 02:52:13 +00:00
|
|
|
|
|
|
|
|
But, git-annex also extends git's concept of remotes, with these special
|
2019-03-22 13:59:56 +00:00
|
|
|
types of remotes. These can be used by git-annex to store and retrieve
|
|
|
|
|
the content of files. They cannot be used by other git commands, and
|
|
|
|
|
the git history is not stored in them.
|
2011-03-28 02:52:13 +00:00
|
|
|
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[adb]] (for Android devices)
|
2012-11-20 21:13:35 +00:00
|
|
|
* [[Amazon_Glacier|glacier]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[bittorrent]]
|
2011-04-08 18:56:57 +00:00
|
|
|
* [[bup]]
|
2014-05-15 18:44:00 +00:00
|
|
|
* [[ddar]]
|
2011-03-30 17:18:46 +00:00
|
|
|
* [[directory]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[gcrypt]] (encrypted git repositories!)
|
2019-08-04 17:15:33 +00:00
|
|
|
* [[git-lfs]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[hook]]
|
2018-09-06 00:54:40 +00:00
|
|
|
* [[rclone]]
|
2011-04-28 00:06:07 +00:00
|
|
|
* [[rsync]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[S3]] (Amazon S3, and other compatible services)
|
2014-01-08 20:14:37 +00:00
|
|
|
* [[tahoe]]
|
2016-12-07 19:44:18 +00:00
|
|
|
* [[tor]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[web]]
|
|
|
|
|
* [[webdav]]
|
2019-10-04 15:22:46 +00:00
|
|
|
* [[git]]
|
2013-03-24 13:09:41 +00:00
|
|
|
* [[xmpp]]
|
2012-03-04 00:06:27 +00:00
|
|
|
|
2013-12-26 22:18:13 +00:00
|
|
|
The above special remotes are built into git-annex, and can be used
|
|
|
|
|
to tie git-annex into many cloud services.
|
2013-08-22 20:21:47 +00:00
|
|
|
|
|
|
|
|
Here are specific instructions
|
2013-09-28 20:19:57 +00:00
|
|
|
for using git-annex with various services:
|
2012-03-04 00:06:27 +00:00
|
|
|
|
2013-05-23 23:28:45 +00:00
|
|
|
* [[Amazon_Glacier|tips/using_Amazon_Glacier]]
|
2016-04-23 19:22:16 +00:00
|
|
|
* [Amazon Cloud drive](https://github.com/DanielDent/git-annex-remote-rclone)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Amazon_S3|tips/using_Amazon_S3]]
|
|
|
|
|
* [Backblaze's B2](https://github.com/encryptio/git-annex-remote-b2)
|
2013-05-23 23:28:45 +00:00
|
|
|
* [[Box.com|tips/using_box.com_as_a_special_remote]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [Ceph](https://github.com/mhameed/git-annex-remote-ceph)
|
2013-09-28 20:19:57 +00:00
|
|
|
* [chef-vault](https://github.com/3ofcoins/knife-annex/)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [Dropbox](https://github.com/DanielDent/git-annex-remote-rclone)
|
|
|
|
|
* [[Flickr|tips/flickrannex]]
|
2018-10-03 16:54:34 +00:00
|
|
|
* [Freenet](https://github.com/xloem/gitlakepy)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Google Cloud Storage|tips/using_Google_Cloud_Storage]]
|
|
|
|
|
* [Google Drive](https://github.com/Lykos153/git-annex-remote-googledrive)
|
2014-02-11 13:35:30 +00:00
|
|
|
* [hubiC](https://github.com/Schnouki/git-annex-remote-hubic)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[IMAP|forum/special_remote_for_IMAP]]
|
|
|
|
|
* [[tips/Internet_Archive_via_S3]]
|
2015-03-05 16:08:50 +00:00
|
|
|
* [[ipfs]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Mega.co.nz|tips/megaannex]]
|
2016-04-23 19:22:16 +00:00
|
|
|
* [Microsoft One Drive](https://github.com/DanielDent/git-annex-remote-rclone)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [Openstack Swift / Rackspace cloud files / Memset Memstore](https://github.com/DanielDent/git-annex-remote-rclone)
|
|
|
|
|
* [[OwnCloud|tips/owncloudannex]]
|
|
|
|
|
* [pCloud](https://github.com/tochev/git-annex-remote-pcloud)
|
|
|
|
|
* [[SkyDrive|tips/skydriveannex]]
|
2017-09-16 17:57:37 +00:00
|
|
|
* [smb / sftp](https://github.com/grawity/code/blob/master/net/git-annex-remote-gvfs)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Usenet|forum/nntp__47__usenet special remote]]
|
|
|
|
|
* [Yandex Disk](https://github.com/DanielDent/git-annex-remote-rclone)
|
2011-04-03 00:59:41 +00:00
|
|
|
|
2013-12-26 22:18:13 +00:00
|
|
|
Want to add support for something else? [[Write your own!|external]]
|
|
|
|
|
|
|
|
|
|
## Ways to use special remotes
|
|
|
|
|
|
|
|
|
|
There are many use cases for a special remote. You could use it as a
|
|
|
|
|
backup. You could use it to archive files offline in a drive with
|
|
|
|
|
encryption enabled so if the drive is stolen your data is not. You could
|
|
|
|
|
git annex move --to specialremote large files when your local drive is
|
|
|
|
|
getting full, and then git annex move the files back when free space is
|
|
|
|
|
again available. You could have one repository copy files to a special
|
|
|
|
|
remote, and then git annex get them on another repository, to transfer the
|
|
|
|
|
files between computers that do not communicate directly.
|
|
|
|
|
|
|
|
|
|
The git-annex assistant makes it easy to set up rsync remotes using this
|
|
|
|
|
last scenario, which is referred to as a transfer repository, and arranges
|
|
|
|
|
to drop files from the transfer repository once they have been transferred
|
|
|
|
|
to all known clients.
|
|
|
|
|
|
|
|
|
|
None of these use cases are particular to particular special remote types.
|
|
|
|
|
Most special remotes can all be used in these and other ways. It largely
|
|
|
|
|
doesn't matter for your use what underlying transport the special remote
|
|
|
|
|
uses.
|
|
|
|
|
|
2011-04-03 00:59:41 +00:00
|
|
|
## Unused content on special remotes
|
|
|
|
|
|
|
|
|
|
Over time, special remotes can accumulate file content that is no longer
|
|
|
|
|
referred to by files in git. Normally, unused content in the current
|
|
|
|
|
repository is found by running `git annex unused`. To detect unused content
|
|
|
|
|
on special remotes, instead use `git annex unused --from`. Example:
|
|
|
|
|
|
|
|
|
|
$ git annex unused --from mys3
|
2011-05-29 02:20:22 +00:00
|
|
|
unused mys3 (checking for unused data...)
|
2011-04-03 00:59:41 +00:00
|
|
|
Some annexed data on mys3 is not used by any files in this repository.
|
|
|
|
|
NUMBER KEY
|
|
|
|
|
1 WORM-s3-m1301674316--foo
|
|
|
|
|
(To see where data was previously used, try: git log --stat -S'KEY')
|
|
|
|
|
(To remove unwanted data: git-annex dropunused --from mys3 NUMBER)
|
|
|
|
|
$ git annex dropunused --from mys3 1
|
|
|
|
|
dropunused 12948 (from mys3...) ok
|
2014-08-01 22:10:02 +00:00
|
|
|
|
|
|
|
|
## Testing special remotes
|
|
|
|
|
|
|
|
|
|
To make sure that a special remote is working correctly, you can use the
|
|
|
|
|
`git annex testremote` command. This expects you to have set up the remote
|
|
|
|
|
as usual, and it then runs a lot of tests, using random data. It's
|
|
|
|
|
particularly useful to test new implementations of special remotes.
|
|
|
|
|
|
|
|
|
|
By default it will upload and download files of around 1MiB to the remote
|
2017-11-02 22:25:47 +00:00
|
|
|
it tests; the `--size` parameter can adjust it to test using smaller files.
|
2014-08-01 22:10:02 +00:00
|
|
|
|
|
|
|
|
It's safe to use this command even when you're already storing data in a
|
|
|
|
|
remote; it won't touch your existing files stored on the remote.
|
|
|
|
|
|
|
|
|
|
For most remotes, it also won't bloat the remote with any data, since
|
|
|
|
|
it cleans up the stuff it uploads. However, the bup, ddar, and tahoe
|
|
|
|
|
special remotes don't support removal of uploaded files, so be careful
|
|
|
|
|
with those.
|
