Commit graph

1362 commits

Author SHA1 Message Date
Joey Hess
c784ef4586 unify exception handling into Utility.Exception
Removed old extensible-exceptions, only needed for very old ghc.

Made webdav use Utility.Exception, to work after some changes in DAV's
exception handling.

Removed Annex.Exception. Mostly this was trivial, but note that
tryAnnex is replaced with tryNonAsync and catchAnnex replaced with
catchNonAsync. In theory that could be a behavior change, since the former
caught all exceptions, and the latter don't catch async exceptions.

However, in practice, nothing in the Annex monad uses async exceptions.
Grepping for throwTo and killThread only find stuff in the assistant,
which does not seem related.

Command.Add.undo is changed to accept a SomeException, and things
that use it for rollback now catch non-async exceptions, rather than
only IOExceptions.
2014-08-07 22:03:29 -04:00
Joey Hess
32e4368377 S3: support chunking
The assistant defaults to 1MiB chunk size for new S3 special remotes.
Which will work around a couple of bugs:
  http://git-annex.branchable.com/bugs/S3_memory_leaks/
  http://git-annex.branchable.com/bugs/S3_upload_not_using_multipart/
2014-08-02 15:51:58 -04:00
Joey Hess
867fd116a7 better exception display 2014-07-26 23:01:44 -04:00
Joey Hess
4f9a29e7f6 use passed program name for kde hooks 2014-07-21 15:30:40 -04:00
Joey Hess
89a72f94e2 webapp: Automatically install Konqueror integration scripts to get and drop files.
Based on the example from the tip, but modified to cd into the repo before
running git-annex, since konqueror does not. Also, at least on my system,
the directory is ~/.kde, not ~/.kde4. (konqueror 4.12.4)

This commit was sponsored by Jürgen Peters.
2014-07-21 15:27:24 -04:00
Joey Hess
67016b5123 Install nautilus hooks even when ~/.local/share/nautilus/ does not yet exist, since it is not automatically created for Gnome 3 users. 2014-07-16 17:19:39 -04:00
Joey Hess
542a8288a8 fix build with path-pieces-0.1.4 2014-07-13 19:24:38 -04:00
Joey Hess
6fe35aab87 Really fix bug that caused the assistant to make many unncessary empty merge commits. 2014-07-09 15:18:00 -04:00
Joey Hess
4a66cd3f91 assistant: Fix bug, introduced in last release, that caused the assistant to make many unncessary empty merge commits. 2014-07-05 17:12:05 -04:00
Joey Hess
326fdffce9
remove unused import 2014-07-04 15:24:35 -04:00
Joey Hess
fc67925fd7
reorg
avoid Git.Command needing Utility.Batch which needs async

For github-backup etc
2014-07-04 12:18:49 -04:00
Joey Hess
d41849bc23
support commit.gpgsign
Support users who have set commit.gpgsign, by disabling gpg signatures for
git-annex branch commits and commits made by the assistant.

The thinking here is that a user sets commit.gpgsign intending the commits
that they manually initiate to be gpg signed. But not commits made in the
background, whether by a deamon or implicitly to the git-annex branch.
gpg signing those would be at best a waste of CPU and at worst would fail,
or flood the user with gpg passphrase prompts, or put their signature on
changes they did not directly do.

See Debian bug #753720.

Also makes all commits done by git-annex go through a few central control
points, to make such changes easier in future.

Also disables commit.gpgsign in the test suite.

This commit was sponsored by Antoine Boegli.
2014-07-04 11:53:51 -04:00
Joey Hess
c4e718d1f5
missing import 2014-07-03 19:49:26 -04:00
Joey Hess
51d6ec6744
force a sane umask when making temp gpg home dir
Someone reported:

gpg: WARNING: unsafe permissions on homedir `/var/folders/m6/zkd11n111m38ff37zbtgq0lr0000gp/T/git-annex-gpg.tmp.0'

Just a warning, but let's fix it anyway. Preumably the user has one of the
many insane and delightful umasks users sometimes use to shoot themselves
in their feet.
2014-07-03 15:43:09 -04:00
Joey Hess
9fb8038166 Windows: Assistant now logs to daemon.log.
Yes, this means that git annex webapp on windows execs git-annex, which
execs itself to set env, and the execs itself again to redirect logs.

This is disgusting. This is Windows(TM).
2014-06-17 19:27:54 -04:00
Joey Hess
8f98d28da4 Windows: Fix opening file browser from webapp when repo is in a directory with spaces. 2014-06-17 14:21:08 -04:00
Joey Hess
7a8f8b5ac9 refactor 2014-06-16 18:59:23 -04:00
Joey Hess
501cc8623a assistant: Fix one-way assistant->assistant sync in direct mode.
When in direct mode, update the master branch after committing to the
annex/direct/master branch. Also, update the synced/master branch.

This fixes a topology A->B where both A and B are in direct mode and
running the assistant, and a change is made to B. Before this fix, A pulled
the changes from B, but since they were only on the annex/direct/master
branch, it did not merge them.

Note that I considered making the assistant merge the
remotes/B/annex/direct/master, but decided to keep it simple and only merge
the sync branches as before.
2014-06-16 11:32:13 -04:00
Joey Hess
e4d7e2ebde fix for Windows file timestamp timezone madness
On Windows, changing the time zone causes the apparent mtime of files to
change. This confuses git-annex, which natually thinks this means the files
have actually been modified (since THAT'S WHAT A MTIME IS FOR, BILL <sheesh>).

Work around this stupidity, by using the inode sentinal file to detect if
the timezone has changed, and calculate a TSDelta, which will be applied
when generating InodeCaches.

This should add no overhead at all on unix. Indeed, I sped up a few
things slightly in the refactoring.

Seems to basically work! But it has a big known problem:
If the timezone changes while the assistant (or a long-running command)
runs, it won't notice, since it only checks the inode cache once, and
so will use the old delta for all new inode caches it generates for new
files it's added. Which will result in them seeming changed the next time
it runs.

This commit was sponsored by Vincent Demeester.
2014-06-12 13:42:21 -04:00
Joey Hess
8ff9938d97 Fix build with wai 0.3.0.
This version of wai changed the type of Middleware, so I cannot seem
to liftIO inside it. So, got rid of a lot of not really needed
complexity to use System.Log.Logger's logging stuff, and just use
the standard wai stdout logger when debug logging is enabled.

Format may change some, and it logs http to stdout instead of stderr
now. Doesn't matter for the webapp since both go to the same log anyway.
2014-06-11 01:29:00 -04:00
Joey Hess
a44fd2c019 export CreateProcess fields from Utility.Process
update code to avoid cwd and env redefinition warnings
2014-06-10 19:20:14 -04:00
Joey Hess
71b2a0a955 webapp: Include ssh port in mangled hostname.
This avoids a collision if different ssh ports are used on the same host
for some reason.

Note that it's ok to change the format of the mangled hostname; unmangling
only extracts the hostname from it, and once ssh is configured for a
mangled hostname, that config is not changed.
2014-06-05 12:50:15 -04:00
Fil
86d50149e9 typo ("Connnect") 2014-06-03 00:04:21 +02:00
Joey Hess
9c0f3ae7e1 assistant: Make sanity checker tmp dir cleanup code more robust. 2014-05-30 15:08:49 -04:00
Joey Hess
4f7f61e46e fix UI when enabling existing gcrypt repo
avoid editing repo for same reasons as in
52601eb606

avoid stomping on its description, even though no description exists until
after syncing is complete
2014-05-30 14:50:44 -04:00
Joey Hess
c92d43fb7b avoid stomping on existing description when enabling a special remote
Happened with eg, gcrypt remotes.
2014-05-30 14:30:43 -04:00
Joey Hess
86b9ae15dd also normalize /~/ 2014-05-30 14:23:21 -04:00
Joey Hess
52601eb606 don't show edit form after enabling ssh remote
Just after enabing a ssh remote, we've not synced with it yet, so its
description and group are not known. So, avoid showing edit form so user
doesn't see blank info. Instead, redirect to dashboard.
2014-05-30 14:17:20 -04:00
Joey Hess
9eaabf0382 webapp: avoid overwriting remote configs when enabling it
Avoid stomping on existing group and preferred content settings
when enabling or combining with an already existing remote.

Two level fix. First, use defaultStandardGroup rather than
setStandardGroup, so if there is an existing configuration in the git-annex
branch, it's not overwritten.

To handle pre-existing ssh remotes (including gcrypt), a second level is
needed, because before syncing with the remote, it's configuration won't be
available locally. (And syncing could take a long time.) So, in this case,
keep track of whether the remote is being created or enabled, and only set
configs when creating it.

This commit was sponsored by Anders Lannerback.
2014-05-30 14:03:04 -04:00
Joey Hess
7089e282b5 fetch from removable drive when adding
When adding a repo from a removable drive that already exists, fetch
from it before showing the edit form, so it will have the existing
configuration of that repo.
2014-05-30 12:48:01 -04:00
Joey Hess
de0cba18f6 webapp: Avoid stomping on existing group and preferred content settings when adding a local repository (or removable drive repository) that already exists.
This does mean that if the webapp is asked to add a git repository on
a removable drive that already exists, but is not yet a git-annex
repository, it will avoid putting it in any group. That unlikely edge case
is ok; the next step is the edit repository screen, which will show it's
not in any group and the user can pick one.
2014-05-30 12:32:44 -04:00
Joey Hess
2dd274e4ca webapp: When adding a new local repository, fix bug that caused its group and preferred content to be set in the current repository, even when not combining.
There was a tricky bit here, when it does combine, the edit form is shown,
and so the info needs to be committed to the new repository, but then
pulled into the current one. And caches need to be invalidated for it
to be visible in the edit form.
2014-05-29 20:17:05 -04:00
Joey Hess
b197ec8917 get rid of (completely safe) uses of Char8
Char8 often indicates an encoding bug. It didn't here, but I can avoid it
and not worry about it.
2014-05-27 20:26:10 -04:00
Joey Hess
a41ddabd73 webapp: When setting up a ssh remote, if the user inputs ~/foo, normalize that to foo, since it's in the home directory by default. 2014-05-27 14:33:27 -04:00
Joey Hess
d83c1a26cf
better work around for broken ghc in debian stable and some ubuntus 2014-05-23 13:00:29 -04:00
Joey Hess
2e1179df46 webapp: When setting up a ssh remote, record it using initremote, so that it can be easily enabled elsewhere.
This is the capstone in making the webapp remember ssh remotes
so they can be easily enabled in other clones of the repository.

Currently, the user will need to enter a password to enable the ssh remote,
but everything else is filled in automatically.

This commit was sponsored by Peter Lloyd.
2014-05-22 14:57:42 -04:00
Joey Hess
c11461b860 webapp: Support for enabling known git repositories on ssh servers.
The repository must have been added using initremote.

Turned out to be much much simpler than expected, because I was able to
reuse the existing code for enabling rsync and gcrypt remotes, which
was already sufficiently general that it will also work for ssh remotes.
Total win!

This commit was sponsored by an unknown bitcoin contributor.
2014-05-22 14:10:48 -04:00
Joey Hess
c6d690f350 include path in "Cannot find old distribution bundle; not upgrading" message 2014-05-21 13:27:40 -04:00
Joey Hess
fed509fb3e assistant: When there are multiple remotes giving different ways to access the same repository, honor remote cost settings and use the cheapest available.
Note that TransferInfo does not always contain the Remote, although
any transfer added to the TransferQueue does have a Remote in its
TransferInfo. The transferkeys command still accepts a UUID, which is
useful to handle upgrades, where an old assistant version runs the new
transferkeys.

This commit was sponsored by Kalle Svensson.
2014-05-19 16:19:33 -04:00
Joey Hess
d37675b1fd
work around utterly strange ubuntu ghc Conflicting definitions for `_' problem 2014-05-19 11:00:01 -04:00
Joey Hess
c0e7aeccd7 fix build with older version of yessod
Old yesod's hamlet parser does not understand f@(Data ...) syntax,
work around this.
2014-05-18 11:17:05 -04:00
Joey Hess
1a40a66d72 webapp: Fixed drag and drop to reorder the list of remotes.
Broken by 958312885f, in November!
I missed this because there's no strong type checking across the AJAX call. :(

Need to switch to Fay to avoid such bugs..
2014-05-15 22:01:32 -04:00
Joey Hess
dc72ea4ab5 deal with ssh key expiry
Not a perfect solution, but good enough, few users will wait 10 minutes in
the middle and see it expire, I hope.
2014-05-15 15:16:37 -04:00
Joey Hess
c705df5651 remove windows-specific rsync.net code, no longer necessary thanks to ssh password handling
Since ssh password prompting no longer happens on stdin, the
authorized_keys line can be sent on stdin. Yay!
2014-05-15 12:30:29 -04:00
Joey Hess
f41b585c29 force strict host key checking when host is known
Avoid any possibilty of prompting in ssh setup in webapp.

Prticularly on Windows this was a problem, it seemed to enter an infinite
loop. I think that ssh can sometimes use SSH_ASKPASS for y/n prompting,
when no controlling TTY is available, and since git-annex always answers
back with the host's password, not y/n, it looped.

This commit was sponsored by Simon Michael.
2014-05-14 18:13:53 -04:00
Joey Hess
a66c942645 refactor 2014-05-14 17:31:20 -04:00
Joey Hess
74f937cc55 this will hopefully make the ssh askpass work on windows 2014-05-14 17:13:20 -04:00
Joey Hess
a11176bab3 avoid clobbering existing env
This is necessary on windows, ssh couldn't resolve hostnames without env
for whatever reason.
2014-05-14 16:36:03 -04:00
Joey Hess
740de08461 fix windows build 2014-05-14 15:54:41 -04:00
Joey Hess
85e9e8c0cf webapp: Better ssh password prompting.
When setting up a remote on a ssh server, prompt for a password inside the
webapp, rather than relying on ssh's own password prompting in the terminal
the webapp was started from, or ssh-askpass.

Avoids double prompting for the ssh password (and triple-prompting on
windows for rsync.net), since the entered password is cached for 10 minutes
and this cached password is reused when setting up the repository, after
the initial probe.

When the user has an existing ssh key set up, they can choose to use it,
rather than entering a password. The webapp used to probe for this case
automatically, so this is a little harder, but it's an advanced user thing.

Note that this commit is known to break enabling existing rsync
repositories. It hs not been tested with gcrypt repositories. It's not been
successfully tested yet on Windows.

This commit was sponsored by Ralph Mayer.
2014-05-14 15:38:32 -04:00
Joey Hess
db8590791f Merge branch 'master' into sshpassword 2014-05-14 12:43:34 -04:00
Joey Hess
4e987d05a8 work around an EvilSplicer bug
This should fix the android build. The EvilSplicer cannot fix up ghc's
misformatting of a case expression with a construstor with a lot of
parameters, which spans multiple lines.
2014-05-10 12:27:21 -03:00
Joey Hess
2aed2d8510 Merge branch 'master' into bootstrap3
Conflicts:
	debian/changelog
2014-05-02 15:32:49 -03:00
Sören Brunk
58f92f148b fix more layout issues based on joey's feedback 2014-05-02 16:29:23 +02:00
Joey Hess
44b6d6c5e0 wip 2014-04-30 21:27:17 -04:00
Joey Hess
ac98853f05 add CredPair cache
Note that this does not yet use SecureMem. It would probably make sense for
the Password part of a CredPair to use SecureMem, and making that change
is better than passing in a String and having it converted to SecureMem in
this code.
2014-04-29 18:08:02 -04:00
Joey Hess
90eca40df2 assistant: Clean up stale tmp files on startup.
This commit was sponsored by Melissa Binde.
2014-04-27 22:22:46 -03:00
Sören Brunk
036aa129f7 fix navbar on error page 2014-04-27 23:57:57 +02:00
Sören Brunk
84eaf8b447 backport Yesod.Form.Bootstrap3 to Yesod 1.0.1 2014-04-27 20:18:36 +02:00
Joey Hess
f5b1c3841e avoid build warning from new dbus 2014-04-26 19:24:56 -04:00
Joey Hess
84a88aab9a avoid build warning from new dbus 2014-04-26 18:57:51 -04:00
Sören Brunk
54fe9af0bb create a local copy of Yesod.Form.Bootstrap3 2014-04-25 12:26:33 +02:00
Joey Hess
ca496b6a97 Standalone builds now check gpg signatures before upgrading. 2014-04-23 13:30:30 -04:00
Sören Brunk
ccc8afb59d don't close tags that are closed by hamlet 2014-04-21 19:47:21 +02:00
Sören Brunk
00c1cd0db1 bootstrap3 forms 2014-04-21 19:47:05 +02:00
Sören Brunk
d2b42c30ad bootstrap3 glyphicons 2014-04-21 19:45:22 +02:00
Sören Brunk
23091cc318 adapt templates to bootstrap3 layout and class changes 2014-04-21 19:45:18 +02:00
Sören Brunk
c8dccf9582 update bootstrap files to bootstrap3 2014-04-21 19:05:09 +02:00
Joey Hess
eba3a28a28 webapp: Support using git-annex on a remote server, which was installed from the standalone tarball or OSX app, and so does not have git-annex in PATH (and may also not have git or rsync in PATH).
* webapp: Support using git-annex on a remote server, which was installed
  from the standalone tarball or OSX app, and so does not have
  git-annex in PATH (and may also not have git or rsync in PATH).
* standalone tarball, OSX app: Install a ~/.ssh/git-annex-wrapper, which
  can be used to run git-annex, git, rsync, etc.
2014-04-20 18:39:10 -04:00
Joey Hess
512da29273 send remote-daemon a RELOAD after making a ssh remote
This doesn't work yet, because RELOAD is buggy and does not notice the new
remote.
2014-04-20 15:30:39 -04:00
Joey Hess
1a4c3caa96 avoid showing the connection nudge alert after creating git or gcrypt remote 2014-04-20 15:10:29 -04:00
Joey Hess
9724667a3d webapp: Fix UI for removing XMPP connection. 2014-04-20 12:46:33 -04:00
Joey Hess
e880d0d22c replace (Key, Backend) with Key
Only fsck and reinject and the test suite used the Backend, and they can
look it up as needed from the Key. This simplifies the code and also speeds
it up.

There is a small behavior change here. Before, all commands would warn when
acting on an annexed file with an unknown backend. Now, only fsck and
reinject show that warning.
2014-04-17 18:03:39 -04:00
Joey Hess
915d038bec reinit: New command that can initialize a new reposotory using the configuration of a previously known repository. Useful if a repository got deleted and you want to clone it back the way it was. 2014-04-15 20:13:35 -04:00
Joey Hess
138d25518d Merge branch 'master' into remotecontrol
Conflicts:
	doc/devblog/day_152__more_ssh_connection_caching.mdwn
2014-04-14 13:38:35 -04:00
Joey Hess
2ff9ba9f74
add missing Network.URI Ord instance for Debian stable 2014-04-14 13:25:49 -04:00
Joey Hess
ff192ace2f detect wicd network disconnection events 2014-04-12 18:36:48 -04:00
Joey Hess
bb44df206d NetWatcher: detect when networkmanager has lost network connection
This is a better approach to finding both when NM has lost a network
connection, and when a new network connection is made by NM.

Tested with network-manager 0.9.8.8.

This commit was sponsored by Cedric Staub.
2014-04-12 17:58:19 -04:00
Joey Hess
a33b30d0c4 remotedaemon: When network connection is lost, close all cached ssh connections.
This commit was sponsored by Cedric Staub.
2014-04-12 16:32:59 -04:00
Joey Hess
db38678595 webapp: Rework xmpp nudge to prompt for either xmpp or a ssh remote be set up.
This commit was sponsored by Nathan Howell.
2014-04-09 16:27:24 -04:00
Joey Hess
14349fb752 also show signal icon next to connected xmpp remotes 2014-04-09 15:51:00 -04:00
Joey Hess
33b8cff433 webapp: Show a network signal icon next to ssh remotes that it's currently connected with. 2014-04-09 15:26:41 -04:00
Joey Hess
fb73792f72 make git-remote-daemon ssh transport robust
* Remote system might be available, and connection get lost. Should
  reconnect, but needs to avoid bad behavior (ie, constant reconnect
  attempts.) Use exponential backoff.

* Detect if old system had a too old git-annex-shell, and show the user
  a nice message in the webapp. Required parsing error messages, so perhaps
  this code shoudl be removed once enough time has passed..

* Switch the protocol to using remote URI's, rather than remote names.
  Names change. Also avoids issues with serialization of names containing
  whitespace.

This is nearly ready for merge into master now. I'd still like to make the ssh
transport smarter about reusing ssh connection caching during git pull.

This commit was sponsored by Jim Paris.
2014-04-09 14:14:55 -04:00
Joey Hess
fac7bca05b assistant: Now detects immediately when other repositories push changes to
a ssh remote, and pulls.

XMPP is no longer needed in this configuration!

Requires the remote server have git-annex-shell with notifychanges support.

(untested)

This commit was sponsored by Geog Wechslberger.
2014-04-08 15:23:50 -04:00
Joey Hess
ade5d8ccc5 Fix nautilus script installation to not crash when the nautilus script dir does not exist. Instead, only install scripts when the directory already exists.
nautilus creates this directory each time run, so this avoids putting cruft
in user's home
2014-04-05 16:36:58 -04:00
Joey Hess
39996f8222 assistant: Adjust ssh authorized keys line generated in local pairing or for a remote ssh server to set environment variables in an alternative way that works with the non-POSIX fix shell, as well as POSIX shells. 2014-04-02 15:30:10 -04:00
Joey Hess
065248f3d2 Added required content configuration.
This includes checking when dropping files that any required content
configuration is satisfied. However, it does not yet include an active
check on the required content; the location log is trusted when checking
the required content expression.
2014-03-29 16:03:33 -04:00
Joey Hess
f2ddf9a299 reload preferred content when group preferred content log changes 2014-03-29 14:46:57 -04:00
Joey Hess
fe19e15040 reorg matcher types; no non-type code changes 2014-03-29 14:43:34 -04:00
Joey Hess
1731cb0840
use shebang lib 2014-03-23 08:17:03 -04:00
Johan Kiviniemi
284e6b5e8d installNautilus: “exec” git-annex and protect params with “--” 2014-03-23 07:28:28 +02:00
Joey Hess
f6eb967f1e webapp: Automatically install Nautilus integration scripts to get and drop files.
This commit was sponsored by Gian-Maria Daffre.
2014-03-22 15:51:30 -04:00
Joey Hess
e426fac273 add desktop notifications
Motivation: Hook scripts for nautilus or other file managers
need to provide the user with feedback that a file is being downloaded.

This commit was sponsored by THM Schoemaker.
2014-03-22 14:12:19 -04:00
Joey Hess
8d5a626a0b rename favicon.png to logo_16x16.png
This was never used as a favicon, it's just a png version of the
favicon.ico. It's actually used for the freedesktop menu file.
2014-03-16 15:52:16 -04:00
Joey Hess
66b8b9c094 webapp: Use securemem for constant time auth token comparisons.
Debian stable does not have securemem, but neither does it have warp-tls,
so just disable use of securemem when not building with https support.
2014-03-12 21:41:20 -04:00
Joey Hess
a7b3a2b097 another old warp-tls workaround 2014-03-12 12:32:43 -04:00
Joey Hess
5a37266080 use Monoid instance for Ordering 2014-03-09 14:08:45 -04:00
Joey Hess
54746463bb webapp: Added a "Sync now" item to each repository's menu. 2014-03-06 18:11:44 -04:00
Joey Hess
d32289bab0 assistant: Smarter log file rotation, which takes free disk space into account. 2014-03-05 18:26:53 -04:00
Joey Hess
a3fe8270ca annex.startupscan can be set to false to disable the assistant's startup scan. 2014-03-05 17:44:14 -04:00