Commit graph

344 commits

Author SHA1 Message Date
Joey Hess
543a78bae0 Support building with DAV 0.4. 2013-04-30 14:10:55 -04:00
Joey Hess
883b17af01 Store an annex-uuid file in the bucket when setting up a new S3 remote. 2013-04-27 17:01:24 -04:00
Joey Hess
c3498042fd webapp: Now automatically fills in any creds used by an existing remote when creating a new remote of the same type. Done for Internet Archive, S3, Glacier, and Box.com remotes. 2013-04-27 15:16:06 -04:00
Joey Hess
3c7f4d2bd1 Automatically register public urls for files uploaded to the Internet Archive. 2013-04-25 17:28:25 -04:00
Joey Hess
e3ea36174b webapp: Display some additional information about a repository on its edit page. 2013-04-25 16:42:17 -04:00
Joey Hess
3e396a3b89 S3: Dropping content from the Internet Archive doesn't work, but their API indicates it does. Always refuse to drop from there. 2013-04-25 15:20:31 -04:00
Joey Hess
8284b310a7 support enabling IA repositories 2013-04-25 13:14:49 -04:00
Joey Hess
4b1cf3d731 Detect when the remote is broken like bitbucket is, and exits 0 when it fails to run git-annex-shell. 2013-04-23 20:06:02 -04:00
Joey Hess
8a2d1988d3 expose Control.Monad.join
I think I've been looking for that function for some time.
Ie, I remember wanting to collapse Just Nothing to Nothing.
2013-04-22 20:24:53 -04:00
Joey Hess
8861e270be sync, assistant: Sync with remotes that have annex-ignore set
This is so git remotes on servers without git-annex installed can be used
to keep clients' git repos in sync.

This is a behavior change, but since annex-sync can be set to disable
syncing with a remote, I think it's acceptable.
2013-04-22 14:57:09 -04:00
Joey Hess
b9904b0c42 fix tab damage 2013-04-13 19:26:59 -04:00
guilhem
a1eded8641 Allow rsync to use other remote shells.
Introduced a new per-remote option 'annex-rsync-transport' to specify
the remote shell that it to be used with rsync. In case the value is
'ssh', connections are cached unless 'sshcaching' is unset.
2013-04-13 19:26:24 -04:00
Joey Hess
9e11699c76 connect existing meters to the transfer log for downloads
Most remotes have meters in their implementations of retrieveKeyFile
already. Simply hooking these up to the transfer log makes that information
available. Easy peasy.

This is particularly valuable information for encrypted remotes, which
otherwise bypass the assistant's polling of temp files, and so don't have
good progress bars yet.

Still some work to do here (see progressbars.mdwn changes), but this
is entirely an improvement from the lack of progress bars for encrypted
downloads.
2013-04-11 17:32:31 -04:00
Joey Hess
c511eb048f changelog & minor style fixes 2013-04-06 16:14:57 -04:00
guilhem
00fc21bfec Generate ciphers with a better entropy.
Unless highRandomQuality=false (or --fast) is set, use Libgcypt's
'GCRY_VERY_STRONG_RANDOM' level by default for cipher generation, like
it's done for OpenPGP key generation.

On the assistant side, the random quality is left to the old (lower)
level, in order not to scare the user with an enless page load due to
the blocking PRNG waiting for IO actions.
2013-04-06 16:09:51 -04:00
Joey Hess
f1b0a4b404 Use lower case hash directories for storing files on crippled filesystems, same as is already done for bare repositories.
* since this is a crippled filesystem anyway, git-annex doesn't use
  symlinks on it
* so there's no reason to use the mixed case hash directories that we're
  stuck using to avoid breaking everyone's symlinks to the content
* so we can do what is already done for all bare repos, and make non-bare
  repos on crippled filesystems use the all-lower case hash directories
* which are, happily, all 3 letters long, so they cannot conflict with
  mixed case hash directories
* so I was able to 100% fix this and even resuming `git annex add` in the
  test case will recover and it will all just work.
2013-04-04 15:46:33 -04:00
Joey Hess
8a5b397ac4 hlint 2013-04-03 03:52:41 -04:00
guilhem
55f0f858ee Allow other MAC algorithms in the Remote Config. 2013-03-29 18:04:52 -04:00
Joey Hess
cf07a2c412 webapp: Progess bar fixes for many types of special remotes.
There was confusion in different parts of the progress bar code about
whether an update contained the total number of bytes transferred, or the
number of bytes transferred since the last update. One way this bug
showed up was progress bars that seemed to stick at zero for a long time.
In order to fix it comprehensively, I add a new BytesProcessed data type,
that is explicitly a total quantity of bytes, not a delta.

Note that this doesn't necessarily fix every problem with progress bars.
Particularly, buffering can now cause progress bars to seem to run ahead
of transfers, reaching 100% when data is still being uploaded.
2013-03-28 17:04:37 -04:00
Joey Hess
449520a573 add globallyAvailable to remotes 2013-03-15 19:16:13 -04:00
Joey Hess
19c0a0d5b1 split cost out into its own module
Added a function to insert a new cost into a list, which could be used to
asjust costs after a drag and drop.
2013-03-13 16:30:34 -04:00
Joey Hess
f7de51e8b6 Bugfix: Fix bug in inode cache sentinal check, which broke copying to local repos if the repo being copied from had moved to a different filesystem or otherwise changed all its inodes' 2013-03-12 16:41:54 -04:00
guilhem
d2bc0e9f3e GnuPG options for symmetric encryption. 2013-03-11 09:48:38 -04:00
Joey Hess
69ab9701eb copyToRemote should return True when the remote already has the key
This got broken in commit e9238e9588.
I observed a key that had been copied to a remote, but the location
log was out of date, and due to this bug, git annex transferkey failed
and so the file could not be dropped when it was moved to an archive
directory.
2013-03-10 17:54:27 -04:00
Joey Hess
56830af8d8 simpler use of MIN_VERSION checks 2013-03-10 15:43:17 -04:00
Joey Hess
ff6ce2bc15 print a warning message when garbage is received from configlist 2013-03-04 23:27:18 -04:00
Joey Hess
0c13d3065e git subcommand cleanup
Pass subcommand as a regular param, which allows passing git parameters
like -c before it. This was already done in the pipeing set of functions,
but not the command running set.
2013-03-03 13:39:07 -04:00
Joey Hess
b117efc19b deal with http-conduit changing a data type
Pity that the library does not provide a function to extract the status
code from the StatusCodeException, so when they had to add a new field, it
breaks every single place that does it.
2013-02-27 00:07:28 -04:00
Joey Hess
a7a1bcd1d6 Avoid passing -p to rsync, to interoperate with crippled filesystems.
In general, git-annex does not try to preserve file permissions. For
example, they don't round trip through special remotes. So it's ok to not
preserve them for git remotes either.

On crippled filesystems, rsync has been observed failing after the file
was transferred because it couldn't set some permission or other.
2013-02-22 15:23:29 -04:00
Joey Hess
96613e85a9 build fix 2013-02-15 13:48:25 -04:00
Joey Hess
9e69fca5bb optimise sending to encrypted rsync
With an encrypted rsync remote, the encrpyted file can be renamed, rather
than being copied, in crippled filesystem mode. This gets back to just as
fast as non-crippled mode for this very common case.
2013-02-15 13:42:41 -04:00
Joey Hess
92b4a63a06 rsync special remote support for crippled filesystem mode
Cannot make a hard link, have to copy.

I did find a way to make it work without setting up a tree, just using
--include and --exclude. But it needs the same hash directories to be used
on both sides, which is normally not the case. Still, I hope one day I will
convert non-bare repos to use the same hash dirs as everything else, and
then this will get more efficient.
2013-02-15 13:33:36 -04:00
Joey Hess
47477b2807 crippled filesystem support, probing and initial support
git annex init probes for crippled filesystems, and sets direct mode, as
well as `annex.crippledfilesystem`.

Avoid manipulating permissions of files on crippled filesystems.
That would likely cause an exception to be thrown.

Very basic support in Command.Add for cripped filesystems; avoids the lock
down entirely since doing it needs both permissions and hard links.
Will make this better soon.
2013-02-14 14:15:26 -04:00
Joey Hess
18a6935e42 safe recv-key in direct mode
Checks the key's size and checksum. This is sorta expensive, but it avoids
needing to add another round-trip to the protocol.
2013-01-11 16:03:45 -04:00
Joey Hess
fec55e742f check for direct mode file change when copying from a local git remote
Only missing direct mode transfer check now is git-annex shell recvkey.
2013-01-10 11:56:06 -04:00
Joey Hess
a6a5ed8121 check for direct mode file change when copying to a local git remote 2013-01-10 11:45:44 -04:00
Joey Hess
1bc49b7158 Special remotes now all rollback storage of keys that get modified during the transfer, which can happen in direct mode. 2013-01-09 18:42:29 -04:00
Joey Hess
909f67443f Fix transferring files to special remotes in direct mode. 2013-01-06 14:29:01 -04:00
Joey Hess
24c6eae1b5 show errors 2013-01-02 13:50:16 -04:00
Joey Hess
4008590c68 type based git config handling for remotes
Still a couple of places that use git config ad-hoc, but this is most of it
done.
2013-01-01 13:58:14 -04:00
Joey Hess
ffdd08fd2e Merge branch 'master' into desymlink 2012-12-13 00:46:10 -04:00
Joey Hess
0d50a6105b whitespace fixes 2012-12-13 00:45:27 -04:00
Joey Hess
e7b8cb0063 direct mode committing 2012-12-12 19:20:38 -04:00
Joey Hess
b4c6da9cbd Got object sending working in direct mode.
However, I don't yet have a reliable way to deal with files being modified
while they're being transferred. I have code that detects it on the sending
side, but the receiver is still free to move the wrong content into its
annex, and record that it has the content. So that's not acceptable, and
I'll need to work on it some more.

However, at this point I can use a direct mode repository as a remote and
transfer files from and to it.
2012-12-08 17:03:39 -04:00
Joey Hess
5460414486 webdav: Avoid trying to set props, avoiding incompatability with livedrive.com. Needs DAV version 0.3. 2012-12-01 17:12:41 -04:00
Joey Hess
757f041cd8 instrument webdav test 2012-12-01 14:32:50 -04:00
Joey Hess
0b6c889012 webapp: S3 and Glacier forms now have a select list of all currently-supported AWS regions. 2012-12-01 14:11:37 -04:00
Joey Hess
020a25abe1 avoid unnecessary Maybe 2012-11-30 00:55:59 -04:00
Joey Hess
ea5d7292e6 dropping from web 2012-11-29 17:01:07 -04:00
Joey Hess
3b35cde0e8 assistant: Retrival from glacier now handled. 2012-11-29 15:23:33 -04:00