Commit graph

20677 commits

Author SHA1 Message Date
https://www.google.com/accounts/o8/id?id=AItOawlVUq_c3-lrQBculOEUu3yjvdavE7JbvEI
b43a6bc7e5 Added a comment 2014-07-25 10:02:53 +00:00
https://www.google.com/accounts/o8/id?id=AItOawnG_weCiH6IleQfVmRPmFJflHQMHdAcDkU
54f3ff878c add a note about direct mode 2014-07-25 07:57:21 +00:00
https://www.google.com/accounts/o8/id?id=AItOawnG_weCiH6IleQfVmRPmFJflHQMHdAcDkU
5820a8d3cd file a bug 2014-07-25 07:42:52 +00:00
Joey Hess
014794f4ed improve a bit 2014-07-24 17:18:14 -04:00
Joey Hess
d52c2b37ad Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-24 16:49:45 -04:00
Joey Hess
3de535cdac devblog 2014-07-24 16:48:07 -04:00
Joey Hess
ceea04e77f move meteredWriteFileChunks out of legacy 2014-07-24 16:42:35 -04:00
Joey Hess
e2c44bf656 implement chunk logs
Slightly tricky as they are not normal UUIDBased logs, but are instead maps
from (uuid, chunksize) to chunkcount.

This commit was sponsored by Frank Thomas.
2014-07-24 16:23:36 -04:00
Joey Hess
bbdb2c04d5 improve chunk data types 2014-07-24 15:08:07 -04:00
Joey Hess
9e2d49d441 prepare for new style chunking
Moved old legacy chunking code, and cleaned up the directory and webdav
remotes use of it, so when no chunking is configured, that code is not
used.

The config for new style chunking will be chunk=1M instead of chunksize=1M.

There should be no behavior changes from this commit.

This commit was sponsored by Andreas Laas.
2014-07-24 14:49:22 -04:00
Joey Hess
d751591ac8 add chunk metadata to Key
Added new fields for chunk number, and chunk size. These will not appear
in normal keys ever, but will be used for chunked data stored on special
remotes.

This commit was sponsored by Jouni K Seppanen.
2014-07-24 13:36:23 -04:00
Joey Hess
4bbc629cb0 document new chunk logfiles 2014-07-24 13:28:54 -04:00
Joey Hess
9e472c9c59 clarify field order reqirement 2014-07-24 12:54:29 -04:00
Joey Hess
937197842e update 2014-07-24 12:41:34 -04:00
Xyem
d0ea0a0479 2014-07-24 12:31:31 +00:00
Joey Hess
ca1d80d708 chunk then encrypt 2014-07-23 22:38:14 -04:00
https://www.google.com/accounts/o8/id?id=AItOawlASRUX9jg2piT1lge4KkBU_ZGty6tvM5c
a42d2a6e4b added gpg instructions 2014-07-24 00:48:33 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmp_51DcIT2sqrOhtg7LgCj2Pyaa5ujxTI
46fec9c1ce 2014-07-23 23:38:55 +00:00
markusk
6df94fb489 Added a comment 2014-07-23 23:18:37 +00:00
http://joeyh.name/
7eb483156d Added a comment 2014-07-23 22:10:37 +00:00
Joey Hess
d82d9b4f30 Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-23 17:56:31 -04:00
Joey Hess
a354bf488b link 2014-07-23 17:56:26 -04:00
Joey Hess
09d807100b devblog 2014-07-23 17:55:44 -04:00
Joey Hess
f15c1fdc8f minor 2014-07-23 17:55:28 -04:00
markusk
0ae8bef4dd 2014-07-23 21:45:18 +00:00
Joey Hess
20627e9fab 4 designs for better chunking
Having a hard time finding a way to totally obscure file sizes, but
otherwise happy with design #4.

This commit was sponsored by Michael Alan Dorman.
2014-07-23 17:41:48 -04:00
markusk
ce7a8d895f 2014-07-23 21:01:22 +00:00
Joey Hess
5262e6a372 recommend users not change the chunksize
Suppose A is 10 mb, and B is 20 mb, and the upload speed is the same. If B
starts first, when A will overwrite the file it is uploading for the 1st chunk.
Then A uploads the second chunk, and once A is done, B finishes the 1st chunk
and uploads its second. We now have 1(from A), 2(from B), so data loss.
2014-07-23 16:46:15 -04:00
markusk
19a0c16232 New bug using webapp with --listen=IP:Port 2014-07-23 20:29:40 +00:00
http://joeyh.name/
c8b2e701ef Added a comment 2014-07-23 15:14:22 +00:00
Joey Hess
95590f878d Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-23 11:06:05 -04:00
http://joeyh.name/
48bb63b034 Added a comment 2014-07-23 15:05:36 +00:00
Joey Hess
28b355e98c close; not a git-annex bug 2014-07-23 11:04:29 -04:00
http://joeyh.name/
b427f72260 Added a comment 2014-07-23 15:02:04 +00:00
Joey Hess
c675e83f86 Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-23 10:43:45 -04:00
Joey Hess
70d932a6d7 update per request 2014-07-23 10:43:09 -04:00
https://www.google.com/accounts/o8/id?id=AItOawkt8DjG40gowj_ETchFLDS_Z216tr7p1xw
d2894d7e48 Added a comment: ok some progress 2014-07-23 14:27:51 +00:00
https://www.google.com/accounts/o8/id?id=AItOawkt8DjG40gowj_ETchFLDS_Z216tr7p1xw
dcc43ef57a 2014-07-23 10:36:34 +00:00
divB
e47a651ce9 Added a comment 2014-07-23 05:20:47 +00:00
https://www.google.com/accounts/o8/id?id=AItOawnD-eBOHeC-94xRxJ10VZHT8t_4FbOLiPM
b040fd1420 2014-07-23 00:19:10 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmMLeU-zCzx2mc5pL2XT8a1UNkQwHAHjg8
98fda2fc5f Added a comment 2014-07-22 02:10:14 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmMLeU-zCzx2mc5pL2XT8a1UNkQwHAHjg8
37797784f3 Added a comment 2014-07-22 01:47:29 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmMLeU-zCzx2mc5pL2XT8a1UNkQwHAHjg8
005325b924 Added a comment 2014-07-22 00:29:20 +00:00
divB
1c32e11369 Added a comment 2014-07-22 00:24:47 +00:00
divB
9e2100df4c Added a comment 2014-07-22 00:11:34 +00:00
Joey Hess
0b6dc31330 Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-21 19:20:34 -04:00
Joey Hess
7844dfb23d devblog 2014-07-21 19:16:05 -04:00
http://joeyh.name/
76778d5878 Added a comment 2014-07-21 22:57:32 +00:00
Joey Hess
9dc3ceb145 Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-21 18:43:32 -04:00
Joey Hess
000dd42ac4 improve repair of bad branches
The repair code assumed that if fsck found no broken objects, after
removing bad objects and possibly pulling replacements from remote, all was
well.. but this is not really true. Removing bad objects could leave some
branches broken. fsck doesn't report any missing objects in this case,
and its messages about broken branches are ignored by the fsck output
parser.

To deal with this, added a separate scan of all refs to find broken ones
and remove them when --forced. This will also let anyone who ran into this
bug run repair again to fix up the incomplete repair done before.

This commit was sponsored by Aaron Whitehouse.
2014-07-21 18:42:58 -04:00