Commit graph

325 commits

Author SHA1 Message Date
Joey Hess
449520a573 add globallyAvailable to remotes 2013-03-15 19:16:13 -04:00
Joey Hess
19c0a0d5b1 split cost out into its own module
Added a function to insert a new cost into a list, which could be used to
asjust costs after a drag and drop.
2013-03-13 16:30:34 -04:00
Joey Hess
f7de51e8b6 Bugfix: Fix bug in inode cache sentinal check, which broke copying to local repos if the repo being copied from had moved to a different filesystem or otherwise changed all its inodes' 2013-03-12 16:41:54 -04:00
guilhem
d2bc0e9f3e GnuPG options for symmetric encryption. 2013-03-11 09:48:38 -04:00
Joey Hess
69ab9701eb copyToRemote should return True when the remote already has the key
This got broken in commit e9238e9588.
I observed a key that had been copied to a remote, but the location
log was out of date, and due to this bug, git annex transferkey failed
and so the file could not be dropped when it was moved to an archive
directory.
2013-03-10 17:54:27 -04:00
Joey Hess
56830af8d8 simpler use of MIN_VERSION checks 2013-03-10 15:43:17 -04:00
Joey Hess
ff6ce2bc15 print a warning message when garbage is received from configlist 2013-03-04 23:27:18 -04:00
Joey Hess
0c13d3065e git subcommand cleanup
Pass subcommand as a regular param, which allows passing git parameters
like -c before it. This was already done in the pipeing set of functions,
but not the command running set.
2013-03-03 13:39:07 -04:00
Joey Hess
b117efc19b deal with http-conduit changing a data type
Pity that the library does not provide a function to extract the status
code from the StatusCodeException, so when they had to add a new field, it
breaks every single place that does it.
2013-02-27 00:07:28 -04:00
Joey Hess
a7a1bcd1d6 Avoid passing -p to rsync, to interoperate with crippled filesystems.
In general, git-annex does not try to preserve file permissions. For
example, they don't round trip through special remotes. So it's ok to not
preserve them for git remotes either.

On crippled filesystems, rsync has been observed failing after the file
was transferred because it couldn't set some permission or other.
2013-02-22 15:23:29 -04:00
Joey Hess
96613e85a9 build fix 2013-02-15 13:48:25 -04:00
Joey Hess
9e69fca5bb optimise sending to encrypted rsync
With an encrypted rsync remote, the encrpyted file can be renamed, rather
than being copied, in crippled filesystem mode. This gets back to just as
fast as non-crippled mode for this very common case.
2013-02-15 13:42:41 -04:00
Joey Hess
92b4a63a06 rsync special remote support for crippled filesystem mode
Cannot make a hard link, have to copy.

I did find a way to make it work without setting up a tree, just using
--include and --exclude. But it needs the same hash directories to be used
on both sides, which is normally not the case. Still, I hope one day I will
convert non-bare repos to use the same hash dirs as everything else, and
then this will get more efficient.
2013-02-15 13:33:36 -04:00
Joey Hess
47477b2807 crippled filesystem support, probing and initial support
git annex init probes for crippled filesystems, and sets direct mode, as
well as `annex.crippledfilesystem`.

Avoid manipulating permissions of files on crippled filesystems.
That would likely cause an exception to be thrown.

Very basic support in Command.Add for cripped filesystems; avoids the lock
down entirely since doing it needs both permissions and hard links.
Will make this better soon.
2013-02-14 14:15:26 -04:00
Joey Hess
18a6935e42 safe recv-key in direct mode
Checks the key's size and checksum. This is sorta expensive, but it avoids
needing to add another round-trip to the protocol.
2013-01-11 16:03:45 -04:00
Joey Hess
fec55e742f check for direct mode file change when copying from a local git remote
Only missing direct mode transfer check now is git-annex shell recvkey.
2013-01-10 11:56:06 -04:00
Joey Hess
a6a5ed8121 check for direct mode file change when copying to a local git remote 2013-01-10 11:45:44 -04:00
Joey Hess
1bc49b7158 Special remotes now all rollback storage of keys that get modified during the transfer, which can happen in direct mode. 2013-01-09 18:42:29 -04:00
Joey Hess
909f67443f Fix transferring files to special remotes in direct mode. 2013-01-06 14:29:01 -04:00
Joey Hess
24c6eae1b5 show errors 2013-01-02 13:50:16 -04:00
Joey Hess
4008590c68 type based git config handling for remotes
Still a couple of places that use git config ad-hoc, but this is most of it
done.
2013-01-01 13:58:14 -04:00
Joey Hess
ffdd08fd2e Merge branch 'master' into desymlink 2012-12-13 00:46:10 -04:00
Joey Hess
0d50a6105b whitespace fixes 2012-12-13 00:45:27 -04:00
Joey Hess
e7b8cb0063 direct mode committing 2012-12-12 19:20:38 -04:00
Joey Hess
b4c6da9cbd Got object sending working in direct mode.
However, I don't yet have a reliable way to deal with files being modified
while they're being transferred. I have code that detects it on the sending
side, but the receiver is still free to move the wrong content into its
annex, and record that it has the content. So that's not acceptable, and
I'll need to work on it some more.

However, at this point I can use a direct mode repository as a remote and
transfer files from and to it.
2012-12-08 17:03:39 -04:00
Joey Hess
5460414486 webdav: Avoid trying to set props, avoiding incompatability with livedrive.com. Needs DAV version 0.3. 2012-12-01 17:12:41 -04:00
Joey Hess
757f041cd8 instrument webdav test 2012-12-01 14:32:50 -04:00
Joey Hess
0b6c889012 webapp: S3 and Glacier forms now have a select list of all currently-supported AWS regions. 2012-12-01 14:11:37 -04:00
Joey Hess
020a25abe1 avoid unnecessary Maybe 2012-11-30 00:55:59 -04:00
Joey Hess
ea5d7292e6 dropping from web 2012-11-29 17:01:07 -04:00
Joey Hess
3b35cde0e8 assistant: Retrival from glacier now handled. 2012-11-29 15:23:33 -04:00
Joey Hess
8dd1d9aaf9 webapp: Defaults to sharing box.com account info with friends, allowing one-click enabling of the repository. 2012-11-28 13:31:49 -04:00
Joey Hess
5ff666ec99 rsync: Fix bug introduced in last release that broke encrypted rsync special remotes. 2012-11-27 16:29:31 -04:00
Joey Hess
356120652f remove redundant showOutput calls.
The meter code does that too.
2012-11-25 16:13:06 -04:00
Joey Hess
fb19d56476 progress bars for glacier downloads 2012-11-25 13:49:22 -04:00
Joey Hess
606c210378 progress bars for glacier uploads 2012-11-25 13:27:20 -04:00
Joey Hess
da6c738dad adjust glacier remote cost to 1000
Higher than any other remote, this is mostly due to the long retrieval
time, so it'd make sense to get a file from nearly any other remote.
(Unless it's behind a very slow connection.)
2012-11-22 16:59:10 -04:00
Joey Hess
f53496830a pass --quiet to checkpresent 2012-11-21 19:35:28 -04:00
Joey Hess
a5111a6d85 Amazon Glacier special remote; 100% working 2012-11-20 16:43:58 -04:00
Joey Hess
9221e62d87 Allow controlling whether login credentials for S3 and webdav are committed to the repository, by setting embedcreds=yes|no when running initremote. 2012-11-19 17:32:58 -04:00
Joey Hess
f7a7ec4ebf new storage regime implemented for webdav 2012-11-19 14:08:39 -04:00
Joey Hess
7b71685a93 Bugfix: directory special remote could loop forever storing a key when a too small chunksize was configured.
Ensure that each file has something written to it, even if the bytestring
chunk size is greater than the configured chunksize.

This means we may write a bit larger than the configured value, but only
when the configured value is very small; ie, < 8 kb.
2012-11-19 13:30:58 -04:00
Joey Hess
5f977cc725 directory special remote: Made more efficient and robust.
Files are now written to a tmp directory in the remote, and once all
chunks are written, etc, it's moved into the final place atomically.

For now, checkpresent still checks every single chunk of a file, because
the old method could leave partially transferred files with some chunks
present and others not.
2012-11-19 13:18:23 -04:00
Joey Hess
d3dfeeb3d9 remove annex/ from key locations used for webdav 2012-11-18 23:59:39 -04:00
Joey Hess
7df1e71fe3 S3: Added progress display for uploading and downloading. 2012-11-18 22:49:07 -04:00
Joey Hess
b0e08ae457 S3: upload progress display 2012-11-18 22:20:43 -04:00
Joey Hess
e2b7fc1ebd refactor 2012-11-18 21:50:16 -04:00
Joey Hess
afa2f9c967 upload progress bars for webdav! 2012-11-18 20:30:05 -04:00
Joey Hess
c8751be151 simplify 2012-11-18 18:27:53 -04:00
Joey Hess
81379bb29c better streaming while encrypting/decrypting
Both the directory and webdav special remotes used to have to buffer
the whole file contents before it could be decrypted, as they read
from chunks. Now the chunks are streamed through gpg with no buffering.
2012-11-18 15:27:44 -04:00