Commit graph

192 commits

Author SHA1 Message Date
Joey Hess
1cd2273035 finally properly fixed ssh zombie leak
The leak was caused by the thread that sshd'd to send transferinfo
not waiting on its ssh. Doh.
2014-08-03 20:14:20 -04:00
Joey Hess
cdf61071bc optimise handling of unavailable repos
The exception handling resulted in git config --list being run twice
for unavailable repos. This dials it back down to running it only once.
2014-07-15 14:45:27 -04:00
Joey Hess
bd514eb65a catch exception when repo is really not available 2014-07-15 14:39:31 -04:00
Joey Hess
522a0922b8 sync: Fix git sync with local git remotes even when they don't have an annex.uuid set.
Catch an exception when ensureInitialized is run in a non-initted
repository. In this case, just read the git config, so that the Git.Repo
object is not LocalUnknown, which is what is used to represent remotes
on eg, drives that are not connected.

The assistant already got this right, and like with the assistant, this
causes an implicit git-annex init of the local remote on the second sync,
once the git-annex branch has been pushed to it.
See this comment for more analysis:
http://git-annex.branchable.com/todo/Recovering_from_a_bad_sync/#comment-64e469a2c1969829ee149cbb41b1c138

This commit was sponsored by jscit.
2014-07-15 14:27:43 -04:00
Joey Hess
a44fd2c019 export CreateProcess fields from Utility.Process
update code to avoid cwd and env redefinition warnings
2014-06-10 19:20:14 -04:00
Joey Hess
c07343e4f7 initremote/enableremote: Basic support for using with regular git remotes
initremote stores the location of an already existing git remote, and
enableremote setups up a remote using its stored location.
2014-05-22 13:42:17 -04:00
Joey Hess
c34b5e09f8 factor out getRemoteGitConfig 2014-05-16 16:08:20 -04:00
Joey Hess
0b899fa2f1 show a much longer message when annex-ignore is automatically set, to help the user fix their problem 2014-05-16 12:58:50 -04:00
Joey Hess
f00cb21037 Bring back rsync -p, but only when git-annex is running on a non-crippled file system. This is a better approach to fix #700282 while not unncessarily losing file permissions on non-crippled systems. 2014-04-17 14:31:42 -04:00
Joey Hess
e426fac273 add desktop notifications
Motivation: Hook scripts for nautilus or other file managers
need to provide the user with feedback that a file is being downloaded.

This commit was sponsored by THM Schoemaker.
2014-03-22 14:12:19 -04:00
Joey Hess
b63276309e clean up cleanup action enumeration 2014-03-13 19:06:26 -04:00
Joey Hess
4d06037fdd Fix zombie leak and general inneficiency when copying files to a local git repo.
Benchmarking this with 1000 small files being copied, the time reduced from
15.98s to 14.64s -- an 8% improvement in the non-data-transfer overhead of
git-annex copy.
2014-03-06 17:13:27 -04:00
Joey Hess
360ecb9f35 fix bare repo optimisation on Windows 2014-02-25 13:47:09 -04:00
Joey Hess
003fc2b7e1
add UrlOptions sum type 2014-02-24 22:00:25 -04:00
Joey Hess
c69d6eb035 Make annex.web-options be used in several places that call curl. 2014-02-24 21:29:37 -04:00
Joey Hess
089c0109a2 Added ways to configure rsync options to be used only when uploading or downloading from a remote. Useful to eg limit upload bandwidth. 2014-02-02 16:06:34 -04:00
Joey Hess
74b101d1dd reorg 2014-01-26 16:36:31 -04:00
Joey Hess
1ca111620d reorg 2014-01-26 16:32:55 -04:00
Joey Hess
5fc2d760ea Optimise non-bare http remotes; no longer does a 404 to the wrong url every time before trying the right url. Needs annex-bare to be set to false, which is done when initially probing the uuid of a http remote. 2014-01-26 13:03:25 -04:00
Joey Hess
207ac67aaa avoid needing a build-dep on hxt for Data.AssocList 2014-01-14 16:42:10 -04:00
Joey Hess
d07f2d7865 Fix a long-standing bug that could cause the wrong index file to be used when committing to the git-annex branch, if GIT_INDEX_FILE is set in the environment. This typically resulted in git-annex branch log files being committed to the master branch and later showing up in the work tree. (These log files can be safely removed.) 2014-01-14 15:36:33 -04:00
Joey Hess
c20f31a1ad add GETAVAILABILITY to external special remote protocol
And some reworking of types, and added an annex-availability git config
setting.
2014-01-13 14:41:10 -04:00
Joey Hess
7be69a2491 gcrypt, bup: Fix bug that prevented using these special remotes with encryption=pubkey.
I think both of these are all that's affected, but I went ahead and fixed
all the remotes that set their config to M.empty to instead store the
actual config. Who knows what will expect it to be actually present in
future, the Remote instance of getGpgEncParams came to..
2013-11-02 16:37:28 -04:00
Joey Hess
7ed8e87a34 assistant: Support repairing git remotes that are locally accessible
(eg, on removable drives)

gcrypt remotes are not yet handled.

This commit was sponsored by Sören Brunk.
2013-10-27 15:38:59 -04:00
Joey Hess
a6e9386d39 fix remote fsck to run in remote 2013-10-14 15:05:29 -04:00
Joey Hess
c78aaed317 ye olde inverted logic 2013-10-14 12:26:46 -04:00
Joey Hess
1ffb3bb0ba add remote fsck interface
Currently only implemented for local git remotes. May try to add support
to git-annex-shell for ssh remotes later. Could concevably also be
supported by some special remote, although that seems unlikely.

Cronner user this when available, and when not falls back to
fsck --fast --from remote

git annex fsck --from does not itself use this interface.
To do so, I would need to pass --fast and all other options that influence
fsck on to the git annex fsck that it runs inside the remote. And that
seems like a lot of work for a result that would be no better than
cd remote; git annex fsck
This may need to be revisited if git-annex-shell gets support, since it
may be the case that the user cannot ssh to the server to run git-annex
fsck there, but can run git-annex-shell there.

This commit was sponsored by Damien Diederen.
2013-10-11 16:03:18 -04:00
Joey Hess
747f5b123c url size fixes
addurl: Improve message when adding url with wrong size to existing file.
Before the message suggested the url didn't exist.

Fixed handling of URL keys that have no recorded size. Before, if the key
has no size, the url also had to not declare any size, which was unlikely
and wrong, or it was taken to not exist. This probably would mostly affect
keys that were added to the annex with addurl --relaxed.
2013-10-11 13:05:00 -04:00
Joey Hess
4e1e625fa6 fix transferring to gcrypt repo from direct mode repo
recvkey was told it was receiving a HMAC key from a direct mode repo,
and that confused it into rejecting the transfer, since it has no way to
verify a key using that backend, since there is no HMAC backend.

I considered making recvkey skip verification in the case of an unknown
backend. However, that could lead to bad results; a key can legitimately be
in the annex with a backend that the remote git-annex-shell doesn't know
about. Better to keep it rejecting if it cannot verify.

Instead, made the gcrypt special remote not set the direct mode flag when
sending (and receiving) files.

Also, added some recvkey messages when its checks fail, since otherwise
all that is shown is a confusing error message from rsync when the remote
git-annex-shell exits nonzero.
2013-10-01 14:38:46 -04:00
Joey Hess
12f6b9693a Send a git-annex user-agent when downloading urls.
Overridable with --user-agent option.

Not yet done for S3 or WebDAV due to limitations of libraries used --
nether allows a user-agent header to be specified.

This commit sponsored by Michael Zehrer.
2013-09-28 14:35:21 -04:00
Joey Hess
c1990702e9 hlint 2013-09-25 23:19:01 -04:00
Joey Hess
3192b059b5 add back lost check that git-annex-shell supports gcrypt 2013-09-24 17:51:12 -04:00
Joey Hess
f9e438c1bc factor out more ssh stuff from git remote
This has the dual benefits of making Remote.Git shorter, and letting
Remote.GCrypt use these utilities.
2013-09-24 13:37:41 -04:00
Joey Hess
e8e209f4e5 better probing for gcrypt repositories using new --check option
Now can tell if a repo uses gcrypt or not, and whether it's decryptable
with the current gpg keys.

This closes the hole that undecryptable gcrypt repos could have before been
combined into the repo in encrypted mode.
2013-09-19 12:53:24 -04:00
Joey Hess
5fe49b98f8 Support hot-swapping of removable drives containing gcrypt repositories.
To support this, a core.gcrypt-id is stored by git-annex inside the git
config of a local gcrypt repository, when setting it up.

That is compared with the remote's cached gcrypt-id. When different, a
drive has been changed. git-annex then looks up the remote config for
the uuid mapped from the core.gcrypt-id, and tweaks the configuration
appropriately. When there is no known config for the uuid, it will refuse to
use the remote.
2013-09-12 15:54:35 -04:00
Joey Hess
b64f5baf2d sync: support gcrypt 2013-09-09 10:02:15 -04:00
Joey Hess
00fb5705ff ignore gcrypt remotes w/o an annex-uuid 2013-09-08 15:19:14 -04:00
Joey Hess
7c1a9cdeb9 partially complete gcrypt remote (local send done; rest not)
This is a git-remote-gcrypt encrypted special remote. Only sending files
in to the remote works, and only for local repositories.

Most of the work so far has involved making initremote work. A particular
problem is that remote setup in this case needs to generate its own uuid,
derivied from the gcrypt-id. That required some larger changes in the code
to support.

For ssh remotes, this will probably just reuse Remote.Rsync's code, so
should be easy enough. And for downloading from a web remote, I will need
to factor out the part of Remote.Git that does that.

One particular thing that will need work is supporting hot-swapping a local
gcrypt remote. I think it needs to store the gcrypt-id in the git config of the
local remote, so that it can check it every time, and compare with the
cached annex-uuid for the remote. If there is a mismatch, it can change
both the cached annex-uuid and the gcrypt-id. That should work, and I laid
some groundwork for it by already reading the remote's config when it's
local. (Also needed for other reasons.)

This commit was sponsored by Daniel Callahan.
2013-09-07 18:38:00 -04:00
Joey Hess
a48a4e2f8a automatically derive an annex-uuid from a gcrypt-uuids 2013-09-05 16:02:39 -04:00
Joey Hess
06db8e0bd9 squash compiler warnings on Windows 2013-08-04 13:18:05 -04:00
Joey Hess
93f2371e09 get rid of __WINDOWS__, use mingw32_HOST_OS
The latter is harder for me to remember, but avoids build failures in code
used by the configure program.
2013-08-02 12:27:32 -04:00
Joey Hess
d2f40d3d76 Fix checking when content is present in a non-bare repository accessed via http.
I thought at first this was a Windows specific problem, but it's not;
this affects checking any non-bare repository exported via http. Which is
a potentially important use case!

The actual bug was the case where Right False was returned by the first url
short-curcuited later checks. But the whole method used felt like code
I'd no longer write, and the use of undefined was particularly disgusting.
So I rewrote it.

Also added an action display.

This commit was sponsored by Eric Hanchrow. Thanks!
2013-07-18 14:20:57 -04:00
Joey Hess
ea6fdc745f fix build on windows 2013-07-09 16:25:15 -04:00
Joey Hess
7e7b2daddf Windows: Fix url to object when using a http remote.
annexLocations uses OS-native directory separators, but for an url,
it needs to use / even on Windows.

This is an ugly workaround. Could parameterize a lot of stuff in
annexLocations to fix it better. I suspect this is probably the only place
it's needed though.
2013-07-07 13:35:56 -04:00
Joey Hess
3b1aedea3d Merge branch 'robustness' 2013-05-25 15:22:18 -04:00
Joey Hess
bf86b5ca16 improve robustness of fromDirect and replaceFile
Made fromDirect check that a file in the tree has good content (and is not
a broken symlink either) before copying it to another file that has the
same key.

Made replaceFile clean up the temp file if the action that creates it, or
the file replacement action fails.
2013-05-25 15:06:02 -04:00
Joey Hess
e3c1586997 Improve error handling when getting uuid of http remotes to auto-ignore, like with ssh remotes. 2013-05-25 01:47:19 -04:00
Joey Hess
796c2f6bc8 remove unnecessary bracketIO 2013-05-19 18:15:29 -04:00
Joey Hess
03eec12cff fix 2013-05-14 13:58:17 -04:00
Joey Hess
17952a893e fix imports 2013-05-14 13:53:29 -04:00