git-annex/Utility
Joey Hess e5b4d447b6 assistant: Start a new git-annex transferkeys process after a network connection change
So that remotes that use a persistent network connection are restarted.

A remote might keep open a long duration network connection, and could
fail to deal well with losing the connection. This is particularly a
concern now that we have external special reotes. An external
special remote that is implemented naively might open the connection only
when PREPARE is sent, and if it loses connection, throw errors on each
request that is made.

(Note that the ssh connection caching should not have this problem; if the
long-duration ssh process loses connection, the named pipe is disconnected
and the next ssh attempt will reconnect. Also, XMPP already deals with
disconnection robustly in its own way.)

There's no way for git-annex to know if a lost network connection actually
affects a given remote, which might have a transfer in process. It does not
make sense to force kill the transferkeys process every time the NetWatcher
detects a change. (Especially because the NetWatcher sometimes polls 1
change per hour.)

In any case, the NetWatcher only detects connection to a network, not
disconnection. So if a transfer is in progress over the network, and the
network goes down, that will need to time out on its own.

An alternate approch that was considered is to use a separate transferkeys
process for each remote, and detect when a request fails, and assume that
means that process is in a failing state and restart it. The problem with
that approach is that if a resource is not available and a remote fails
every time, it degrades to starting a new transferkeys process for every
file transfer, which is too expensive.

Instead, this commit only handles the network reconnection case, and restarts
transferkeys only once the network has reconnected and another transfer needs
to be made. So, a transferkeys process will be reused for 1 hour, or until the
next network connection.

----

The NotificationBroadcaster was rewritten to use TMVars rather than MSampleVars,
to allow checking without blocking if a notification has been received.

----

This commit was sponsored by Tobias Brunner.
2014-01-06 16:03:39 -04:00
..
DirWatcher assistant: Fix OSX-specific bug that caused the startup scan to try to follow symlinks to other directories, and add their contents to the annex. 2013-12-18 15:05:29 -04:00
Applicative.hs pointlessness 2012-06-29 10:00:05 -04:00
Base64.hs allow to build when sandi is installed (don't use it yet) 2013-11-11 15:42:06 -04:00
Batch.hs typo 2013-12-29 22:40:23 -04:00
CoProcess.hs Avoid looping if long-running git cat-file or git hash-object crashes and keeps crashing when restarted. 2014-01-01 21:42:25 -04:00
CopyFile.hs get rid of __WINDOWS__, use mingw32_HOST_OS 2013-08-02 12:27:32 -04:00
Daemon.hs unmask async exceptions after fork 2013-10-10 16:05:44 -04:00
Data.hs clean up some ugly code 2013-09-27 19:52:36 -04:00
DataUnits.hs refactor and unify code 2013-07-19 19:39:14 -04:00
DBus.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
Directory.hs convert hacky shell linux mklibs code to haskell ; fixing symlink bug 2013-12-24 13:13:17 -04:00
DirWatcher.hs reorg 2013-12-04 23:09:54 -04:00
DiskFree.hs warning 2013-12-10 01:35:27 -04:00
Dot.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
Env.hs fix the day's windows permissions damage 2013-05-12 19:09:48 -04:00
Exception.hs clean up some ugly code 2013-09-27 19:52:36 -04:00
ExternalSHA.hs Use cryptohash rather than SHA for hashing. 2013-09-22 20:06:02 -04:00
FileMode.hs Revert "don't use writeFileProtected on windows" 2013-12-06 21:24:34 -04:00
FileSystemEncoding.hs Fix a few bugs involving filenames that are at or near the filesystem's maximum filename length limit. 2013-07-30 19:18:29 -04:00
Format.hs gpg secret keys list parsing 2013-09-16 12:57:39 -04:00
FreeDesktop.hs linux standalone auto-install icons 2013-07-09 20:50:41 -04:00
Gpg.hs remove *>=> and >=*> ; use <$$> instead 2013-09-27 19:58:48 -04:00
Hash.hs allow building w/o cryptohash 2013-10-03 12:33:38 -04:00
HumanNumber.hs refactor and unify code 2013-07-19 19:39:14 -04:00
HumanTime.hs parse "5" as 5 seconds 2013-10-26 12:07:00 -04:00
InodeCache.hs add, import, assistant: Better preserve the mtime of symlinks, when when adding content that gets deduplicated. 2013-09-25 16:07:11 -04:00
JSONStream.hs whitespace fixes 2012-12-13 00:45:27 -04:00
libdiskfree.c Makefile now builds using cabal, taking advantage of cabal's automatic detection of appropriate build flags. 2013-02-27 02:39:22 -04:00
libdiskfree.h Renamed diskfree.c to avoid OSX case insensativity bug. 2012-04-13 11:26:39 -04:00
libkqueue.c include sys/types.h 2013-04-24 10:39:52 -04:00
libkqueue.h fix prototype 2012-06-19 01:57:19 -04:00
libmounts.c cleanup 2012-07-19 21:20:38 -04:00
libmounts.h Got removable media mount detection working on Android. 2013-05-04 16:19:25 -04:00
LogFile.hs avoid more build warnings on Windows 2013-08-04 14:05:36 -04:00
Lsof.hs watcher: Detect at startup time when there is a stale .git/lock, and remove it so it does not interfere with the automatic commits of changed files. 2013-10-03 16:57:21 -04:00
Matcher.hs fix handling of Not in the matcher 2013-05-25 13:50:27 -04:00
Metered.hs external special remotes mostly implemented (untested) 2013-12-26 18:23:13 -04:00
Misc.hs add readFileStrictAnyEncoding 2013-11-20 13:41:13 -04:00
Monad.hs remove *>=> and >=*> ; use <$$> instead 2013-09-27 19:58:48 -04:00
Mounts.hsc avoid warnings when built with ghc 7.6 2013-06-02 15:01:58 -04:00
Network.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
NotificationBroadcaster.hs assistant: Start a new git-annex transferkeys process after a network connection change 2014-01-06 16:03:39 -04:00
OSX.hs squelch warning 2012-11-26 16:29:05 -04:00
Parallel.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
PartialPrelude.hs rename readMaybe to readish 2012-01-23 17:00:10 -04:00
Path.hs addurl, importfeed: Sanitize | and some other symbols and special characters. 2013-12-27 17:52:20 -04:00
Percentage.hs cleanup 2013-07-20 20:56:04 -04:00
Process.hs port transferkeys to windows; make stopping in progress transfers work too (probably) 2013-12-10 23:19:18 -04:00
QuickCheck.hs add schedule to vicfg 2013-10-07 17:11:13 -04:00
Quvi.hs golf 2013-11-25 00:02:48 -04:00
Rsync.hs rsync special remote: Fix fallback mode for rsync remotes that use hashDirMixed. Closes: #731142 2013-12-02 12:53:39 -04:00
SafeCommand.hs fix syntax 2013-08-02 12:42:14 -04:00
Scheduled.hs fix quickcheck range 2013-11-01 11:54:26 -04:00
Shell.hs fix use of wrong shebang when android is installing git-annex-shell wrapper on server 2013-05-06 15:58:13 -04:00
SRV.hs fix build with haskell DNS 1.0.0 2013-09-17 11:54:09 -04:00
SshConfig.hs assistant: Ensure that .ssh/config and .ssh/authorized_keys are not group or world writable when writing to those files, as that can make ssh refuse to use them, if it allows another user to write to them. 2014-01-03 17:44:12 -04:00
Tense.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
ThreadLock.hs reorg 2012-06-17 14:02:40 -04:00
ThreadScheduler.hs avoid blocked indefinitely on mvar on windows 2013-12-04 17:25:41 -04:00
TList.hs add two long-running XMPP push threads, no more inversion of control 2013-05-22 15:13:31 -04:00
Tmp.hs fix 2013-12-30 14:05:07 -04:00
Touch.hsc finished where indentation changes 2012-12-13 00:24:19 -04:00
Url.hs Added support for quvi 0.9. Slightly suboptimal due to limitations in its interface compared with the old version. 2013-11-24 23:44:30 -04:00
UserInfo.hs get rid of __WINDOWS__, use mingw32_HOST_OS 2013-08-02 12:27:32 -04:00
Verifiable.hs finished where indentation changes 2012-12-13 00:24:19 -04:00
WebApp.hs squash warning 2013-12-09 17:24:32 -04:00
Yesod.hs switch define used, not android specific 2013-12-18 02:06:15 -04:00