2012-04-11 18:22:59 +00:00
|
|
|
git-annex can transfer data to and from configured git remotes.
|
2011-08-17 01:12:48 +00:00
|
|
|
Normally those remotes are normal git repositories (bare and non-bare;
|
2012-04-11 18:22:59 +00:00
|
|
|
local and remote), that store the file contents in their own git-annex
|
2011-08-17 01:12:48 +00:00
|
|
|
directory.
|
2011-03-28 02:52:13 +00:00
|
|
|
|
|
|
|
But, git-annex also extends git's concept of remotes, with these special
|
2019-03-22 13:59:56 +00:00
|
|
|
types of remotes. These can be used by git-annex to store and retrieve
|
|
|
|
the content of files. They cannot be used by other git commands, and
|
|
|
|
the git history is not stored in them.
|
2011-03-28 02:52:13 +00:00
|
|
|
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[adb]] (for Android devices)
|
2012-11-20 21:13:35 +00:00
|
|
|
* [[Amazon_Glacier|glacier]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[bittorrent]]
|
2011-04-08 18:56:57 +00:00
|
|
|
* [[bup]]
|
2014-05-15 18:44:00 +00:00
|
|
|
* [[ddar]]
|
2011-03-30 17:18:46 +00:00
|
|
|
* [[directory]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[gcrypt]] (encrypted git repositories!)
|
2019-08-04 17:15:33 +00:00
|
|
|
* [[git-lfs]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[hook]]
|
2011-04-28 00:06:07 +00:00
|
|
|
* [[rsync]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[S3]] (Amazon S3, and other compatible services)
|
2014-01-08 20:14:37 +00:00
|
|
|
* [[tahoe]]
|
2016-12-07 19:44:18 +00:00
|
|
|
* [[tor]]
|
2020-09-02 14:41:27 +00:00
|
|
|
* [[web]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[webdav]]
|
2019-10-04 15:22:46 +00:00
|
|
|
* [[git]]
|
2020-09-02 14:41:27 +00:00
|
|
|
* [[httpalso]]
|
2020-12-17 17:12:35 +00:00
|
|
|
* [[borg]]
|
2024-04-17 19:19:42 +00:00
|
|
|
* [[rclone]]
|
2012-03-04 00:06:27 +00:00
|
|
|
|
2013-12-26 22:18:13 +00:00
|
|
|
The above special remotes are built into git-annex, and can be used
|
|
|
|
to tie git-annex into many cloud services.
|
2013-08-22 20:21:47 +00:00
|
|
|
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
Here are specific instructions for using git-annex with various services:
|
2012-03-04 00:06:27 +00:00
|
|
|
|
2013-05-23 23:28:45 +00:00
|
|
|
* [[Amazon_Glacier|tips/using_Amazon_Glacier]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Amazon_S3|tips/using_Amazon_S3]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[Backblaze B2|tips/using_Backblaze_B2]]
|
2013-05-23 23:28:45 +00:00
|
|
|
* [[Box.com|tips/using_box.com_as_a_special_remote]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [Ceph](https://github.com/mhameed/git-annex-remote-ceph)
|
2013-09-28 20:19:57 +00:00
|
|
|
* [chef-vault](https://github.com/3ofcoins/knife-annex/)
|
2020-03-05 02:42:33 +00:00
|
|
|
* [[Dropbox|tips/dropboxannex]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[FTP|rclone]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Flickr|tips/flickrannex]]
|
2020-03-31 23:40:12 +00:00
|
|
|
* [Freenet and Siacoin Skynet](https://github.com/xloem/gitlakepy)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Google Cloud Storage|tips/using_Google_Cloud_Storage]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[Google Drive|tips/using_Google_Drive]]
|
|
|
|
* [[hubiC|tips/using_Hubic]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[IMAP|forum/special_remote_for_IMAP]]
|
|
|
|
* [[tips/Internet_Archive_via_S3]]
|
2015-03-05 16:08:50 +00:00
|
|
|
* [[ipfs]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[Jottacloud|rclone]]
|
|
|
|
* [[Mega|tips/megaannex]]
|
|
|
|
* [[Microsoft Azure Blob Storage|rclone]]
|
|
|
|
* [[Microsoft OneDrive|rclone]]
|
2022-09-02 20:55:36 +00:00
|
|
|
* [NNCP](https://git.sr.ht/~ehmry/git-annex-remote-nncp)
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[OpenDrive|rclone]]
|
|
|
|
* [[Openstack Swift / Rackspace cloud files / Memset Memstore|rclone]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[OwnCloud|tips/owncloudannex]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[pCloud|rclone]]
|
|
|
|
* [[QingStor|rclone]]
|
|
|
|
* [[SFTP|rclone]]
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[SkyDrive|tips/skydriveannex]]
|
2017-09-16 17:57:37 +00:00
|
|
|
* [smb / sftp](https://github.com/grawity/code/blob/master/net/git-annex-remote-gvfs)
|
2018-09-06 00:52:58 +00:00
|
|
|
* [[Usenet|forum/nntp__47__usenet special remote]]
|
improve special remotes list
In several cases, there's an old, maybe unmaintained special remote,
and rclone also supports the service. I don't know which is better,
so I added tips pages for these, listing both possibilies, so users can
fill in the details.
Added some things listed in special_remotes/rclone that were not
in the cloud provider list, so users will see it supports those.
But, omitted S3 and webdav, since it's probably better for users to use
what's built into git-annex than going via rclone.
The rclone list is out of date apparently, as rclone.org has more,
so I added a note at the bottom for users to check.
This commit was sponsored by Ethan Aubin.
2020-03-05 02:27:56 +00:00
|
|
|
* [[Yandex Disk|rclone]]
|
|
|
|
|
|
|
|
If a service is not mentioned above, it's worth checking if
|
|
|
|
[rclone](https://rclone.org/) supports it, then you can use the
|
|
|
|
[[rclone_special_remote|rclone]].
|
2011-04-03 00:59:41 +00:00
|
|
|
|
2013-12-26 22:18:13 +00:00
|
|
|
Want to add support for something else? [[Write your own!|external]]
|
|
|
|
|
|
|
|
## Ways to use special remotes
|
|
|
|
|
|
|
|
There are many use cases for a special remote. You could use it as a
|
|
|
|
backup. You could use it to archive files offline in a drive with
|
|
|
|
encryption enabled so if the drive is stolen your data is not. You could
|
|
|
|
git annex move --to specialremote large files when your local drive is
|
|
|
|
getting full, and then git annex move the files back when free space is
|
|
|
|
again available. You could have one repository copy files to a special
|
|
|
|
remote, and then git annex get them on another repository, to transfer the
|
2024-02-05 19:48:15 +00:00
|
|
|
files between computers that do not communicate directly.
|
2013-12-26 22:18:13 +00:00
|
|
|
|
|
|
|
None of these use cases are particular to particular special remote types.
|
|
|
|
Most special remotes can all be used in these and other ways. It largely
|
|
|
|
doesn't matter for your use what underlying transport the special remote
|
|
|
|
uses.
|
|
|
|
|
2024-02-05 19:48:15 +00:00
|
|
|
## Setting up a special remote
|
|
|
|
|
2024-02-05 19:50:41 +00:00
|
|
|
To initialize a new special remote, use
|
2024-02-05 19:48:15 +00:00
|
|
|
[[git-annex initremote|git-annex-initremote]]. See the documentation for
|
|
|
|
the special remote you want to use for details about configuration and
|
|
|
|
examples of how to initremote it.
|
|
|
|
|
2024-02-05 19:50:41 +00:00
|
|
|
Once a special remote has been initialize, other clones of the repository can
|
2024-02-05 19:48:15 +00:00
|
|
|
also enable it, by using [[git-annex enableremote|git-annex-enableremote]]
|
|
|
|
with the same name that was used to initialize it. (Run the command without
|
|
|
|
any name to get a list of available special remotes.)
|
|
|
|
|
|
|
|
Initializing or enabling a special remote adds it as a remote of your git
|
|
|
|
repository. You can't use git commands like `git pull` with the remote
|
|
|
|
(usually, there are exceptions like [[git-lfs]]), but you can use git-annex
|
|
|
|
commands.
|
|
|
|
|
2011-04-03 00:59:41 +00:00
|
|
|
## Unused content on special remotes
|
|
|
|
|
|
|
|
Over time, special remotes can accumulate file content that is no longer
|
|
|
|
referred to by files in git. Normally, unused content in the current
|
|
|
|
repository is found by running `git annex unused`. To detect unused content
|
|
|
|
on special remotes, instead use `git annex unused --from`. Example:
|
|
|
|
|
|
|
|
$ git annex unused --from mys3
|
2011-05-29 02:20:22 +00:00
|
|
|
unused mys3 (checking for unused data...)
|
2011-04-03 00:59:41 +00:00
|
|
|
Some annexed data on mys3 is not used by any files in this repository.
|
|
|
|
NUMBER KEY
|
|
|
|
1 WORM-s3-m1301674316--foo
|
|
|
|
(To see where data was previously used, try: git log --stat -S'KEY')
|
|
|
|
(To remove unwanted data: git-annex dropunused --from mys3 NUMBER)
|
|
|
|
$ git annex dropunused --from mys3 1
|
|
|
|
dropunused 12948 (from mys3...) ok
|
2014-08-01 22:10:02 +00:00
|
|
|
|
2024-02-05 19:48:15 +00:00
|
|
|
## Removing special remotes
|
2014-08-01 22:10:02 +00:00
|
|
|
|
2024-02-05 19:48:15 +00:00
|
|
|
Like git remotes, a special remote can be removed from your repository
|
|
|
|
by using `git remote remove`. Note that does not delete the special remote,
|
|
|
|
or prevent other repositories from enabling or using it.
|
2014-08-01 22:10:02 +00:00
|
|
|
|
2024-02-05 19:48:15 +00:00
|
|
|
## Testing special remotes
|
2014-08-01 22:10:02 +00:00
|
|
|
|
2024-02-05 19:48:15 +00:00
|
|
|
To make sure that a special remote is working correctly, you can use the
|
|
|
|
[[git annex testremote|git-annex-testremote]] command. This expects you to
|
|
|
|
have set up the remote as usual, and it then runs a lot of tests, using
|
|
|
|
random data. It's particularly useful to test new implementations of
|
|
|
|
special remotes.
|