S3: support chunking

The assistant defaults to 1MiB chunk size for new S3 special remotes.
Which will work around a couple of bugs:
  http://git-annex.branchable.com/bugs/S3_memory_leaks/
  http://git-annex.branchable.com/bugs/S3_upload_not_using_multipart/
This commit is contained in:
Joey Hess 2014-08-02 15:51:58 -04:00
parent c3750901d8
commit 32e4368377
6 changed files with 38 additions and 60 deletions

View file

@ -15,6 +15,7 @@ module Types.Key (
file2key,
nonChunkKey,
chunkKeyOffset,
isChunkKey,
prop_idempotent_key_encode,
prop_idempotent_key_decode
@ -62,6 +63,9 @@ chunkKeyOffset k = (*)
<$> keyChunkSize k
<*> (pred <$> keyChunkNum k)
isChunkKey :: Key -> Bool
isChunkKey k = isJust (keyChunkSize k) && isJust (keyChunkNum k)
fieldSep :: Char
fieldSep = '-'