Commit graph

91 commits

Author SHA1 Message Date
guilhem
8293ed619f Allow public-key encryption of file content.
With the initremote parameters "encryption=pubkey keyid=788A3F4C".

/!\ Adding or removing a key has NO effect on files that have already
been copied to the remote. Hence using keyid+= and keyid-= with such
remotes should be used with care, and make little sense unless the point
is to replace a (sub-)key by another. /!\

Also, a test case has been added to ensure that the cipher and file
contents are encrypted as specified by the chosen encryption scheme.
2013-09-03 14:34:16 -04:00
Joey Hess
38022f4f49 Windows: Fixed permissions problem that prevented removing files from directory special remote.
Directory special remotes now fully usable.
2013-08-04 13:43:48 -04:00
Joey Hess
93f2371e09 get rid of __WINDOWS__, use mingw32_HOST_OS
The latter is harder for me to remember, but avoids build failures in code
used by the configure program.
2013-08-02 12:27:32 -04:00
Joey Hess
ca9ac8770f directory special remote: Fix checking that there is enough disk space to hold an object, was broken when using encryption. 2013-07-20 16:30:49 -04:00
Joey Hess
abe8d549df fix permission damage (thanks, Windows) 2013-05-11 23:54:25 -04:00
Joey Hess
3c7e30a295 git-annex now builds on Windows (doesn't work) 2013-05-11 15:03:00 -05:00
Joey Hess
a0f6dab8de When initializing a directory special remote with a relative path, the path is made absolute.
Using a relative path would work, until the user changed to some other
directory in the repo and tried to access the remote from there..
2013-05-06 17:15:36 -04:00
Joey Hess
9e11699c76 connect existing meters to the transfer log for downloads
Most remotes have meters in their implementations of retrieveKeyFile
already. Simply hooking these up to the transfer log makes that information
available. Easy peasy.

This is particularly valuable information for encrypted remotes, which
otherwise bypass the assistant's polling of temp files, and so don't have
good progress bars yet.

Still some work to do here (see progressbars.mdwn changes), but this
is entirely an improvement from the lack of progress bars for encrypted
downloads.
2013-04-11 17:32:31 -04:00
Joey Hess
cf07a2c412 webapp: Progess bar fixes for many types of special remotes.
There was confusion in different parts of the progress bar code about
whether an update contained the total number of bytes transferred, or the
number of bytes transferred since the last update. One way this bug
showed up was progress bars that seemed to stick at zero for a long time.
In order to fix it comprehensively, I add a new BytesProcessed data type,
that is explicitly a total quantity of bytes, not a delta.

Note that this doesn't necessarily fix every problem with progress bars.
Particularly, buffering can now cause progress bars to seem to run ahead
of transfers, reaching 100% when data is still being uploaded.
2013-03-28 17:04:37 -04:00
Joey Hess
449520a573 add globallyAvailable to remotes 2013-03-15 19:16:13 -04:00
Joey Hess
19c0a0d5b1 split cost out into its own module
Added a function to insert a new cost into a list, which could be used to
asjust costs after a drag and drop.
2013-03-13 16:30:34 -04:00
guilhem
d2bc0e9f3e GnuPG options for symmetric encryption. 2013-03-11 09:48:38 -04:00
Joey Hess
47477b2807 crippled filesystem support, probing and initial support
git annex init probes for crippled filesystems, and sets direct mode, as
well as `annex.crippledfilesystem`.

Avoid manipulating permissions of files on crippled filesystems.
That would likely cause an exception to be thrown.

Very basic support in Command.Add for cripped filesystems; avoids the lock
down entirely since doing it needs both permissions and hard links.
Will make this better soon.
2013-02-14 14:15:26 -04:00
Joey Hess
1bc49b7158 Special remotes now all rollback storage of keys that get modified during the transfer, which can happen in direct mode. 2013-01-09 18:42:29 -04:00
Joey Hess
909f67443f Fix transferring files to special remotes in direct mode. 2013-01-06 14:29:01 -04:00
Joey Hess
4008590c68 type based git config handling for remotes
Still a couple of places that use git config ad-hoc, but this is most of it
done.
2013-01-01 13:58:14 -04:00
Joey Hess
0d50a6105b whitespace fixes 2012-12-13 00:45:27 -04:00
Joey Hess
020a25abe1 avoid unnecessary Maybe 2012-11-30 00:55:59 -04:00
Joey Hess
7b71685a93 Bugfix: directory special remote could loop forever storing a key when a too small chunksize was configured.
Ensure that each file has something written to it, even if the bytestring
chunk size is greater than the configured chunksize.

This means we may write a bit larger than the configured value, but only
when the configured value is very small; ie, < 8 kb.
2012-11-19 13:30:58 -04:00
Joey Hess
5f977cc725 directory special remote: Made more efficient and robust.
Files are now written to a tmp directory in the remote, and once all
chunks are written, etc, it's moved into the final place atomically.

For now, checkpresent still checks every single chunk of a file, because
the old method could leave partially transferred files with some chunks
present and others not.
2012-11-19 13:18:23 -04:00
Joey Hess
7df1e71fe3 S3: Added progress display for uploading and downloading. 2012-11-18 22:49:07 -04:00
Joey Hess
c8751be151 simplify 2012-11-18 18:27:53 -04:00
Joey Hess
81379bb29c better streaming while encrypting/decrypting
Both the directory and webdav special remotes used to have to buffer
the whole file contents before it could be decrypted, as they read
from chunks. Now the chunks are streamed through gpg with no buffering.
2012-11-18 15:27:44 -04:00
Joey Hess
1fe76b57d6 webdav now checks presence of and receives chunked content
Note that receiving encrypted chunked content currently involves buffering.
(So does doing so with the directory special remote.)
2012-11-16 23:16:18 -04:00
Joey Hess
92d5d81c2c generic chunked content helper
However, directory still uses its optimzed chunked file writer, as it uses
less memory than the generic one in the helper.
2012-11-16 17:58:08 -04:00
Joey Hess
2172cc586e where indenting 2012-11-11 00:51:07 -04:00
Joey Hess
ff32ee5152 upload progress tracking for the directory special remote 2012-09-21 14:54:24 -04:00
Joey Hess
226781c047 unify types 2012-09-21 14:50:14 -04:00
Joey Hess
aff09a1f33 add a progress callback to storeKey, and threaded it all the way through
Transfer info files are updated when the callback is called, updating
the number of bytes transferred.

Left unused p variables at every place the callback should be used.
Which is rather a lot..
2012-09-19 16:08:37 -04:00
Joey Hess
271ea49978 add support for readonly remotes
Currently only the web special remote is readonly, but it'd be possible to
also have readonly drives, or other remotes. These are handled in the
assistant by only downloading from them, and never trying to upload to
them.
2012-08-26 15:39:02 -04:00
Joey Hess
78d3add86b tweak field name 2012-08-26 14:26:43 -04:00
Joey Hess
4ec9244f1a add a path field to remotes
Also broke out some helper functions around constructing remotes,
to be used later.
2012-07-22 14:30:43 -04:00
Joey Hess
7225c2bfc0 record transfer information on local git remotes
In order to record a semi-useful filename associated with the key,
this required plumbing the filename all the way through to the remotes'
storeKey and retrieveKeyFile.

Note that there is potential for deadlock here, narrowly avoided.
Suppose the repos are A and B. A sends file foo to B, and at the same
time, B gets file foo from A. So, A locks its upload transfer info file,
and then locks B's download transfer info file. At the same time,
B is taking the two locks in the opposite order. This is only not a
deadlock because the lock code does not wait, and aborts. So one of A or
B's transfers will be aborted and the other transfer will continue.
Whew!
2012-07-01 17:15:11 -04:00
Joey Hess
29335bf326 pointlessness 2012-06-29 10:00:05 -04:00
Joey Hess
75b6ee81f9 avoid ByteString.Char8 where not needed
Its truncation behavior is a red flag, so avoid using it in these places
where only raw ByteStrings are used, without looking at the data inside.
2012-06-20 13:13:40 -04:00
Joey Hess
ed79596b75 noop 2012-04-21 23:32:33 -04:00
Joey Hess
5cc76098ca Directory special remotes now check annex.diskreserve. 2012-04-20 16:24:44 -04:00
Joey Hess
4eb5112681 rationalize getConfig
getConfig got a remote-specific config, and this confusing name caused it
to be used a couple of places that only were interested in global configs.
Rename to getRemoteConfig and make getConfig only get global configs.

There are no behavior changes here, but remote.<name>.annex-web-options
never actually worked (and per-remote web options is a very unlikely to be
useful case so I didn't make it work), so fix the documentation for it.
2012-03-22 17:32:47 -04:00
Joey Hess
c0c9991c9f nukes another 15 lines thanks to ifM 2012-03-15 20:39:25 -04:00
Joey Hess
3960825cef better chunked file retrieval
Avoids opening every chunk at once, instead streaming them in.

Not done for encrypted file retrieval yet.
2012-03-04 11:48:23 -04:00
Joey Hess
7ba79cfb8c thread through original key to retrieveEnctypted
Allows showing progress bar for this last case of the directory special
remote.
2012-03-04 03:36:39 -04:00
Joey Hess
4638314001 add progress display when receiving files
That was actually really easy. But, when getting a file from an encrypted
directory special remote, no meter can be shown, because the total file
size is not known.
2012-03-04 03:25:41 -04:00
Joey Hess
9856c24a59 Add progress bar display to the directory special remote.
So far I've only written progress bars for sending files, not yet
receiving.

No longer uses external cp at all. ByteString IO is fast enough.
2012-03-04 03:17:25 -04:00
Joey Hess
50c897c082 tweak 2012-03-03 20:02:48 -04:00
Joey Hess
3436aba6de Directory special remotes now support chunking files written to them
Avoiding writing files larger than a specified size is useful on certian
things. For example, box.com has a file size limit of 100 mb. Could also
be useful on really crappy removable media.
2012-03-03 18:05:55 -04:00
Joey Hess
cb631ce518 whereis: Prints the urls of files that the web special remote knows about. 2012-02-14 03:49:48 -04:00
Joey Hess
ce5637498f remove Utility.Conditional and use IfElse
This drops the >>! and >>? with the nice low fixity. IfElse does have
undocumented >>=>>! and >>=>>? operators, but I deem that too fishy.
Anyway, using whenM and unlessM is easier; I sometimes mixed the operators
up.
2012-01-24 16:22:07 -04:00
Joey Hess
61dbad505d fsck --from remote --fast
Avoids expensive file transfers, at the expense of checking file size
and/or contents.

Required some reworking of the remote code.
2012-01-20 13:23:11 -04:00
Joey Hess
71cb04bb6d optimize fsck --from directory special remote
No need to copy anything, just symlink to the file.
2012-01-19 16:14:40 -04:00
Joey Hess
06b0cb6224 add tmp flag parameter to retrieveKeyFile 2012-01-19 16:07:36 -04:00