git-annex/doc/bugs/S3_upload_not_using_multipart.mdwn
https://www.google.com/accounts/o8/id?id=AItOawnWvnTWY6LrcPB4BzYEBn5mRTpNhg5EtEg 2a1a21cb08 IMHO: there is no multipart parameter in the source but partsize (I don't speek Haskell, sorry) as of now.
2014-12-08 17:27:06 +00:00

62 lines
2 KiB
Markdown

What steps will reproduce the problem?
> Try to copy/move a file greater than 5G to S3.
git annex copy large_file.tgz --to cloud
What is the expected output? What do you see instead?
> Looks like git-annex may not be using the Multipart Upload API: http://docs.aws.amazon.com/AmazonS3/latest/dev/uploadobjusingmpu.html
> Expected transfer to succeed, instead this error is output:
copy large-file.tgz (gpg) (checking cloud...) (to cloud...) Reading passphrase from file descriptor 12
Your proposed upload exceeds the maximum allowed size
failed
git-annex: copy: 1 failed
What version of git-annex are you using? On what operating system?
> OSX 10.8.2
Please provide any additional information below.
annex [master●] % git annex status
supported backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
supported remote types: git S3 bup directory rsync web webdav glacier hook
repository mode: indirect
trusted repositories: 0
semitrusted repositories: 3
00000000-0000-0000-0000-000000000001 -- web
BE1D8EC7-C64B-47DE-AD4E-2A50437532B4 -- cloud
E84568BA-6A4B-4AA1-B622-605B9248EDB1 -- here (eric laptop)
untrusted repositories: 0
dead repositories: 0
transfers in progress: none
available local disk space: 169 gigabytes (+1 megabyte reserved)
temporary directory size: 218 megabytes (clean up with git-annex unused)
local annex keys: 24
local annex size: 8 gigabytes
known annex keys: 25
known annex size: 8 gigabytes
bloom filter size: 16 mebibytes (0% full)
backend usage:
SHA256E: 49
annex [master●] % git annex version
git-annex version: 3.20130114
local repository version: 3
default repository version: 3
supported repository versions: 3
upgrade supported from repository versions: 0 1 2
[[!tag confirmed]]
> [[fixed|done]] This is now supported, when git-annex is built with a new
> enough version of the aws library. You need to configure the remote to
> use an appropriate value for multipart, eg:
>
> git annex enableremote cloud partsize=1GiB
>
> --[[Joey]]