Commit graph

20934 commits

Author SHA1 Message Date
Joey Hess
b978adb503 document testremote 2014-08-01 18:10:02 -04:00
Joey Hess
5aa2286e7b Merge branch 'newchunks'
I am happy enough with this to make it live!
2014-08-01 18:00:47 -04:00
Joey Hess
370c01811f Merge branch 'master' of ssh://git-annex.branchable.com 2014-08-01 17:56:57 -04:00
Joey Hess
85ec4130bc devblog 2014-08-01 17:56:31 -04:00
Joey Hess
1ee24a0366 testremote now tests with and without encryption 2014-08-01 17:52:40 -04:00
Joey Hess
50a1cac24f fix example external remote script to write files atomically 2014-08-01 17:52:09 -04:00
Joey Hess
de0da0aece minor optimisation 2014-08-01 17:18:39 -04:00
Joey Hess
3991327d09 testremote: Test retrieveKeyFile resume
And fixed a bug found by these tests; retrieveKeyFile would fail
when the dest file was already complete.

This commit was sponsored by Bradley Unterrheiner.
2014-08-01 17:16:20 -04:00
Joey Hess
20d7295386 improve testremote command, adding chunk size testing
And also a --size parameter to configure the basic object size.
2014-08-01 16:50:24 -04:00
Joey Hess
f4f82e2741 deriving Show 2014-08-01 16:30:33 -04:00
Joey Hess
9636cfd9e1 fix a fenchpost bug when resuming chunked store at end
Discovered thanks to testremote command!
2014-08-01 16:29:39 -04:00
Joey Hess
8fce4e4bd7 fix chunk=0
Found by testremote
2014-08-01 15:36:11 -04:00
Joey Hess
9720ee9e56 testremote: New command to test uploads/downloads to a remote.
This only performs some basic tests so far; no testing of chunking or
resuming. Also, the existing encryption type of the remote is used; it
would be good later to derive an encrypted and a non-encrypted version of
the remote and test them both.

This commit was sponsored by Joseph Liu.
2014-08-01 15:10:01 -04:00
gernot
0b078b2d37 2014-08-01 17:57:44 +00:00
Joey Hess
c03e1c5648 add new section for testing commands 2014-08-01 12:49:26 -04:00
https://www.google.com/accounts/o8/id?id=AItOawmAINLSovhWM_4_KrbngOcxduIbBuKv8ZA
0d5fe783e4 Added a comment 2014-08-01 09:28:21 +00:00
Xyem
4e9e700548 Added a comment 2014-08-01 09:05:45 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmAINLSovhWM_4_KrbngOcxduIbBuKv8ZA
532231a748 2014-08-01 08:56:23 +00:00
zardoz
b100938c37 Added a comment 2014-07-31 11:43:16 +00:00
zardoz
044dc0252d Add tip to make Firefox behave as expected when opening annexed HTML documents. 2014-07-31 10:48:11 +00:00
https://www.google.com/accounts/o8/id?id=AItOawm8BAEUyzYhORZmMuocRTk4M-3IumDm5VU
e1713220bc Added a comment: Bug still valid 2014-07-31 08:35:30 +00:00
http://olivier.mehani.name/
b9ddecac7a typo 2014-07-31 05:09:33 +00:00
https://www.google.com/accounts/o8/id?id=AItOawn-TDneVW-8kwb1fyTRAJfH3l1xs2VSEmk
a5eeefedd2 Added a comment 2014-07-30 20:37:27 +00:00
https://www.google.com/accounts/o8/id?id=AItOawmwjQzWgiD7_I3zw-_91rMRf_6qoThupis
76d546adac Added a comment 2014-07-30 20:33:44 +00:00
Joey Hess
76d894f2e5 Display exception message when a transfer fails due to an exception.
For example, I had a copy to a remote that was failing for an unknown
reason. This let me see the exception was createDirectory: permission
denied; the underlying problem being a permissions issue.
2014-07-30 15:57:19 -04:00
Joey Hess
5848793d6b chunk size advice 2014-07-30 12:06:59 -04:00
Joey Hess
97f60ac140 Merge branch 'master' of ssh://git-annex.branchable.com 2014-07-30 11:58:17 -04:00
Joey Hess
f3e457a195 add missing Ord constraint (fixes android build)
Probably the new ghc used on android is the root cause of needing this
constraint.
2014-07-30 11:57:40 -04:00
Joey Hess
b5ac627fee WebDAV: Dropped support for DAV before 0.6.1.
0.6.1 is in testing, and stable does not have DAV at all, so I can dispense
with this compatability code
2014-07-30 11:20:35 -04:00
http://joeyh.name/
64db37a974 Added a comment: interesting idea 2014-07-30 15:03:46 +00:00
http://joeyh.name/
1defbdb10c Added a comment 2014-07-30 14:55:16 +00:00
Joey Hess
83c58f17e0 mention possibility of parallel chunk upload/download 2014-07-30 10:43:21 -04:00
Joey Hess
89416ba2d9 only chunk stable keys
The content of unstable keys can potentially be different in different
repos, so eg, resuming a chunked upload started by another repo would
corrupt data.
2014-07-30 10:34:39 -04:00
Joey Hess
58f8b8f250 devblog 2014-07-29 20:44:26 -04:00
Joey Hess
a963d790d3 update progress after each chunk, at least
This way, when the remote implementation neglects to update progress,
there will still be a somewhat useful progress display, as long as chunks
are used.
2014-07-29 20:31:16 -04:00
Joey Hess
444944c7a9 fix cleanup of FileContents once done when them when retrieving 2014-07-29 20:27:13 -04:00
Joey Hess
53b87a859e optimise case of remote that retrieves FileContent, when chunks and encryption are not being used
No need to read whole FileContent only to write it back out to a file in
this case. Can just rename! Yay.

Also indidentially, fixed an attempt to open a file for write that was
already opened for write, which caused a crash and deadlock.
2014-07-29 20:10:14 -04:00
Joey Hess
c0dc134cde support chunking for all external special remotes!
Removing code and at the same time adding great features, including
upload/download resuming.

This commit was sponsored by Romain Lenglet.
2014-07-29 18:50:20 -04:00
Joey Hess
bc9e4697b9 better type for Retriever
Putting a callback in the Retriever type allows for the callback to
remove the retrieved file when it's done with it.

I did not really want to make Retriever be fixed to Annex Bool,
but when I tried to use Annex a, I got into some type of type mess.
2014-07-29 18:41:41 -04:00
Joey Hess
47e522979c allow Retriever action to update the progress meter
Needed for eg, Remote.External.

Generally, any Retriever that stores content in a file is responsible for
updating the meter, while ones that procude a lazy bytestring cannot update
the meter, so are not asked to.
2014-07-29 17:18:49 -04:00
Joey Hess
1d263e1e7e lift types from IO to Annex
Some remotes like External need to run store and retrieve actions in Annex,
not IO. In order to do that lift, I had to dive pretty deep into the
utilities, making Utility.Gpg and Utility.Tmp be partly converted to using
MonadIO, and Control.Monad.Catch for exception handling.

There should be no behavior changes in this commit.

This commit was sponsored by Michael Barabanov.
2014-07-29 16:28:44 -04:00
https://www.google.com/accounts/o8/id?id=AItOawlYGpCG2u-0fVbpQhIDMX_ZfE64E-K2EMI
585d7a84bf 2014-07-29 20:11:30 +00:00
Joey Hess
f5af470875 add ContentSource type, for remotes that act on files rather than ByteStrings
Note that currently nothing cleans up a ContentSource's file, when eg,
retrieving chunks.
2014-07-29 15:16:12 -04:00
Joey Hess
216fdbd6bd fix non-checked hasKeyChunks 2014-07-29 15:07:32 -04:00
Xyem
bb0ddec05d 2014-07-29 15:38:40 +00:00
http://svario.it/gioele
ff9131c93a new wish: add repository name to commit messages 2014-07-29 15:29:30 +00:00
http://svario.it/gioele
53518beb1c new bug: whereis does not work in direct mode 2014-07-29 15:17:09 +00:00
http://svario.it/gioele
2695dfbab5 Added a comment 2014-07-29 14:25:19 +00:00
https://id.koumbit.net/anarcat
bc64ec4880 i hope this is not too silly 2014-07-28 23:02:39 +00:00
https://id.koumbit.net/anarcat
61c31be21e found some files in misctmp 2014-07-28 22:56:14 +00:00