43 lines
1.8 KiB
Markdown
43 lines
1.8 KiB
Markdown
Some [[special_remotes]] have support for breaking large files up into
|
|
chunks that are stored on the remote.
|
|
|
|
This can be useful to work around limitations on the size of files
|
|
on the remote.
|
|
|
|
Chunking also allows for resuming interrupted downloads and uploads.
|
|
|
|
Note that git-annex has to buffer chunks in memory before they are sent to
|
|
a remote. So, using a large chunk size will make it use more memory.
|
|
|
|
To enable chunking, pass a `chunk=nnMiB` parameter to `git annex
|
|
initremote`, specifying the chunk size.
|
|
|
|
Good chunk sizes will depend on the remote, but a good starting place
|
|
is probably `1MiB`. Very large chunks are problimatic, both because
|
|
git-annex needs to buffer one chunk in memory when uploading, and because
|
|
a larger chunk will make resuming interrupted transfers less efficient.
|
|
On the other hand, when a file is split into a great many chunks,
|
|
there can be increased overhead of making many requests to the remote.
|
|
|
|
To disable chunking of a remote that was using chunking,
|
|
pass `chunk=0` to `git annex enableremote`. Any content already stored on
|
|
the remote using chunks will continue to be accessed via chunks, this
|
|
just prevents using chunks when storing new content.
|
|
|
|
To change the chunk size, pass a `chunk=nnMiB` parameter to
|
|
`git annex enableremote`. This only affects the chunk sized used when
|
|
storing new content.
|
|
|
|
# old-style chunking
|
|
|
|
Note that older versions of git-annex used a different chunk method, which
|
|
was configured by passing `chunksize=nnMib` when initializing a remote.
|
|
|
|
The old-style chunking had a number of problems, including being less
|
|
efficient, and not allowing resumes of encrypted uploads.
|
|
|
|
It's not possible to change a remote using that old chunking method to the
|
|
new one, but git-annex continues to support the old-style chunking to
|
|
support such remotes.
|
|
|
|
See also: [[design document|design/assistant/chunks]]
|