Commit graph

195 commits

Author SHA1 Message Date
Joey Hess
c784ef4586 unify exception handling into Utility.Exception
Removed old extensible-exceptions, only needed for very old ghc.

Made webdav use Utility.Exception, to work after some changes in DAV's
exception handling.

Removed Annex.Exception. Mostly this was trivial, but note that
tryAnnex is replaced with tryNonAsync and catchAnnex replaced with
catchNonAsync. In theory that could be a behavior change, since the former
caught all exceptions, and the latter don't catch async exceptions.

However, in practice, nothing in the Annex monad uses async exceptions.
Grepping for throwTo and killThread only find stuff in the assistant,
which does not seem related.

Command.Add.undo is changed to accept a SomeException, and things
that use it for rollback now catch non-async exceptions, rather than
only IOExceptions.
2014-08-07 22:03:29 -04:00
Joey Hess
b4cf22a388 pushed checkPresent exception handling out of Remote implementations
I tend to prefer moving toward explicit exception handling, not away from
it, but in this case, I think there are good reasons to let checkPresent
throw exceptions:

1. They can all be caught in one place (Remote.hasKey), and we know
   every possible exception is caught there now, which we didn't before.
2. It simplified the code of the Remotes. I think it makes sense for
   Remotes to be able to be implemented without needing to worry about
   catching exceptions inside them. (Mostly.)
3. Types.StoreRetrieve.Preparer can only work on things that return a
   Bool, which all the other relevant remote methods already did.
   I do not see a good way to generalize that type; my previous attempts
   failed miserably.
2014-08-06 13:45:19 -04:00
Joey Hess
6f4592966d make testremote work with gcrypt repos
This involved making Remote.Gcrypt.gen expect a Repo with a regular,
non-gcrypt path. Since tht is what's stored as the Remote's gitrepo,
testremote can then modify it and feed it back into gen.
2014-08-04 08:42:04 -04:00
Joey Hess
1cd2273035 finally properly fixed ssh zombie leak
The leak was caused by the thread that sshd'd to send transferinfo
not waiting on its ssh. Doh.
2014-08-03 20:14:20 -04:00
Joey Hess
cdf61071bc optimise handling of unavailable repos
The exception handling resulted in git config --list being run twice
for unavailable repos. This dials it back down to running it only once.
2014-07-15 14:45:27 -04:00
Joey Hess
bd514eb65a catch exception when repo is really not available 2014-07-15 14:39:31 -04:00
Joey Hess
522a0922b8 sync: Fix git sync with local git remotes even when they don't have an annex.uuid set.
Catch an exception when ensureInitialized is run in a non-initted
repository. In this case, just read the git config, so that the Git.Repo
object is not LocalUnknown, which is what is used to represent remotes
on eg, drives that are not connected.

The assistant already got this right, and like with the assistant, this
causes an implicit git-annex init of the local remote on the second sync,
once the git-annex branch has been pushed to it.
See this comment for more analysis:
http://git-annex.branchable.com/todo/Recovering_from_a_bad_sync/#comment-64e469a2c1969829ee149cbb41b1c138

This commit was sponsored by jscit.
2014-07-15 14:27:43 -04:00
Joey Hess
a44fd2c019 export CreateProcess fields from Utility.Process
update code to avoid cwd and env redefinition warnings
2014-06-10 19:20:14 -04:00
Joey Hess
c07343e4f7 initremote/enableremote: Basic support for using with regular git remotes
initremote stores the location of an already existing git remote, and
enableremote setups up a remote using its stored location.
2014-05-22 13:42:17 -04:00
Joey Hess
c34b5e09f8 factor out getRemoteGitConfig 2014-05-16 16:08:20 -04:00
Joey Hess
0b899fa2f1 show a much longer message when annex-ignore is automatically set, to help the user fix their problem 2014-05-16 12:58:50 -04:00
Joey Hess
f00cb21037 Bring back rsync -p, but only when git-annex is running on a non-crippled file system. This is a better approach to fix #700282 while not unncessarily losing file permissions on non-crippled systems. 2014-04-17 14:31:42 -04:00
Joey Hess
e426fac273 add desktop notifications
Motivation: Hook scripts for nautilus or other file managers
need to provide the user with feedback that a file is being downloaded.

This commit was sponsored by THM Schoemaker.
2014-03-22 14:12:19 -04:00
Joey Hess
b63276309e clean up cleanup action enumeration 2014-03-13 19:06:26 -04:00
Joey Hess
4d06037fdd Fix zombie leak and general inneficiency when copying files to a local git repo.
Benchmarking this with 1000 small files being copied, the time reduced from
15.98s to 14.64s -- an 8% improvement in the non-data-transfer overhead of
git-annex copy.
2014-03-06 17:13:27 -04:00
Joey Hess
360ecb9f35 fix bare repo optimisation on Windows 2014-02-25 13:47:09 -04:00
Joey Hess
003fc2b7e1
add UrlOptions sum type 2014-02-24 22:00:25 -04:00
Joey Hess
c69d6eb035 Make annex.web-options be used in several places that call curl. 2014-02-24 21:29:37 -04:00
Joey Hess
089c0109a2 Added ways to configure rsync options to be used only when uploading or downloading from a remote. Useful to eg limit upload bandwidth. 2014-02-02 16:06:34 -04:00
Joey Hess
74b101d1dd reorg 2014-01-26 16:36:31 -04:00
Joey Hess
1ca111620d reorg 2014-01-26 16:32:55 -04:00
Joey Hess
5fc2d760ea Optimise non-bare http remotes; no longer does a 404 to the wrong url every time before trying the right url. Needs annex-bare to be set to false, which is done when initially probing the uuid of a http remote. 2014-01-26 13:03:25 -04:00
Joey Hess
207ac67aaa avoid needing a build-dep on hxt for Data.AssocList 2014-01-14 16:42:10 -04:00
Joey Hess
d07f2d7865 Fix a long-standing bug that could cause the wrong index file to be used when committing to the git-annex branch, if GIT_INDEX_FILE is set in the environment. This typically resulted in git-annex branch log files being committed to the master branch and later showing up in the work tree. (These log files can be safely removed.) 2014-01-14 15:36:33 -04:00
Joey Hess
c20f31a1ad add GETAVAILABILITY to external special remote protocol
And some reworking of types, and added an annex-availability git config
setting.
2014-01-13 14:41:10 -04:00
Joey Hess
7be69a2491 gcrypt, bup: Fix bug that prevented using these special remotes with encryption=pubkey.
I think both of these are all that's affected, but I went ahead and fixed
all the remotes that set their config to M.empty to instead store the
actual config. Who knows what will expect it to be actually present in
future, the Remote instance of getGpgEncParams came to..
2013-11-02 16:37:28 -04:00
Joey Hess
7ed8e87a34 assistant: Support repairing git remotes that are locally accessible
(eg, on removable drives)

gcrypt remotes are not yet handled.

This commit was sponsored by Sören Brunk.
2013-10-27 15:38:59 -04:00
Joey Hess
a6e9386d39 fix remote fsck to run in remote 2013-10-14 15:05:29 -04:00
Joey Hess
c78aaed317 ye olde inverted logic 2013-10-14 12:26:46 -04:00
Joey Hess
1ffb3bb0ba add remote fsck interface
Currently only implemented for local git remotes. May try to add support
to git-annex-shell for ssh remotes later. Could concevably also be
supported by some special remote, although that seems unlikely.

Cronner user this when available, and when not falls back to
fsck --fast --from remote

git annex fsck --from does not itself use this interface.
To do so, I would need to pass --fast and all other options that influence
fsck on to the git annex fsck that it runs inside the remote. And that
seems like a lot of work for a result that would be no better than
cd remote; git annex fsck
This may need to be revisited if git-annex-shell gets support, since it
may be the case that the user cannot ssh to the server to run git-annex
fsck there, but can run git-annex-shell there.

This commit was sponsored by Damien Diederen.
2013-10-11 16:03:18 -04:00
Joey Hess
747f5b123c url size fixes
addurl: Improve message when adding url with wrong size to existing file.
Before the message suggested the url didn't exist.

Fixed handling of URL keys that have no recorded size. Before, if the key
has no size, the url also had to not declare any size, which was unlikely
and wrong, or it was taken to not exist. This probably would mostly affect
keys that were added to the annex with addurl --relaxed.
2013-10-11 13:05:00 -04:00
Joey Hess
4e1e625fa6 fix transferring to gcrypt repo from direct mode repo
recvkey was told it was receiving a HMAC key from a direct mode repo,
and that confused it into rejecting the transfer, since it has no way to
verify a key using that backend, since there is no HMAC backend.

I considered making recvkey skip verification in the case of an unknown
backend. However, that could lead to bad results; a key can legitimately be
in the annex with a backend that the remote git-annex-shell doesn't know
about. Better to keep it rejecting if it cannot verify.

Instead, made the gcrypt special remote not set the direct mode flag when
sending (and receiving) files.

Also, added some recvkey messages when its checks fail, since otherwise
all that is shown is a confusing error message from rsync when the remote
git-annex-shell exits nonzero.
2013-10-01 14:38:46 -04:00
Joey Hess
12f6b9693a Send a git-annex user-agent when downloading urls.
Overridable with --user-agent option.

Not yet done for S3 or WebDAV due to limitations of libraries used --
nether allows a user-agent header to be specified.

This commit sponsored by Michael Zehrer.
2013-09-28 14:35:21 -04:00
Joey Hess
c1990702e9 hlint 2013-09-25 23:19:01 -04:00
Joey Hess
3192b059b5 add back lost check that git-annex-shell supports gcrypt 2013-09-24 17:51:12 -04:00
Joey Hess
f9e438c1bc factor out more ssh stuff from git remote
This has the dual benefits of making Remote.Git shorter, and letting
Remote.GCrypt use these utilities.
2013-09-24 13:37:41 -04:00
Joey Hess
e8e209f4e5 better probing for gcrypt repositories using new --check option
Now can tell if a repo uses gcrypt or not, and whether it's decryptable
with the current gpg keys.

This closes the hole that undecryptable gcrypt repos could have before been
combined into the repo in encrypted mode.
2013-09-19 12:53:24 -04:00
Joey Hess
5fe49b98f8 Support hot-swapping of removable drives containing gcrypt repositories.
To support this, a core.gcrypt-id is stored by git-annex inside the git
config of a local gcrypt repository, when setting it up.

That is compared with the remote's cached gcrypt-id. When different, a
drive has been changed. git-annex then looks up the remote config for
the uuid mapped from the core.gcrypt-id, and tweaks the configuration
appropriately. When there is no known config for the uuid, it will refuse to
use the remote.
2013-09-12 15:54:35 -04:00
Joey Hess
b64f5baf2d sync: support gcrypt 2013-09-09 10:02:15 -04:00
Joey Hess
00fb5705ff ignore gcrypt remotes w/o an annex-uuid 2013-09-08 15:19:14 -04:00
Joey Hess
7c1a9cdeb9 partially complete gcrypt remote (local send done; rest not)
This is a git-remote-gcrypt encrypted special remote. Only sending files
in to the remote works, and only for local repositories.

Most of the work so far has involved making initremote work. A particular
problem is that remote setup in this case needs to generate its own uuid,
derivied from the gcrypt-id. That required some larger changes in the code
to support.

For ssh remotes, this will probably just reuse Remote.Rsync's code, so
should be easy enough. And for downloading from a web remote, I will need
to factor out the part of Remote.Git that does that.

One particular thing that will need work is supporting hot-swapping a local
gcrypt remote. I think it needs to store the gcrypt-id in the git config of the
local remote, so that it can check it every time, and compare with the
cached annex-uuid for the remote. If there is a mismatch, it can change
both the cached annex-uuid and the gcrypt-id. That should work, and I laid
some groundwork for it by already reading the remote's config when it's
local. (Also needed for other reasons.)

This commit was sponsored by Daniel Callahan.
2013-09-07 18:38:00 -04:00
Joey Hess
a48a4e2f8a automatically derive an annex-uuid from a gcrypt-uuids 2013-09-05 16:02:39 -04:00
Joey Hess
06db8e0bd9 squash compiler warnings on Windows 2013-08-04 13:18:05 -04:00
Joey Hess
93f2371e09 get rid of __WINDOWS__, use mingw32_HOST_OS
The latter is harder for me to remember, but avoids build failures in code
used by the configure program.
2013-08-02 12:27:32 -04:00
Joey Hess
d2f40d3d76 Fix checking when content is present in a non-bare repository accessed via http.
I thought at first this was a Windows specific problem, but it's not;
this affects checking any non-bare repository exported via http. Which is
a potentially important use case!

The actual bug was the case where Right False was returned by the first url
short-curcuited later checks. But the whole method used felt like code
I'd no longer write, and the use of undefined was particularly disgusting.
So I rewrote it.

Also added an action display.

This commit was sponsored by Eric Hanchrow. Thanks!
2013-07-18 14:20:57 -04:00
Joey Hess
ea6fdc745f fix build on windows 2013-07-09 16:25:15 -04:00
Joey Hess
7e7b2daddf Windows: Fix url to object when using a http remote.
annexLocations uses OS-native directory separators, but for an url,
it needs to use / even on Windows.

This is an ugly workaround. Could parameterize a lot of stuff in
annexLocations to fix it better. I suspect this is probably the only place
it's needed though.
2013-07-07 13:35:56 -04:00
Joey Hess
3b1aedea3d Merge branch 'robustness' 2013-05-25 15:22:18 -04:00
Joey Hess
bf86b5ca16 improve robustness of fromDirect and replaceFile
Made fromDirect check that a file in the tree has good content (and is not
a broken symlink either) before copying it to another file that has the
same key.

Made replaceFile clean up the temp file if the action that creates it, or
the file replacement action fails.
2013-05-25 15:06:02 -04:00
Joey Hess
e3c1586997 Improve error handling when getting uuid of http remotes to auto-ignore, like with ssh remotes. 2013-05-25 01:47:19 -04:00