Commit graph

200 commits

Author SHA1 Message Date
Joey Hess
3ea708e03b Merge branch 'master' into assistant 2012-07-02 15:45:20 -04:00
Joey Hess
7daa269853 better pid file locking code 2012-07-02 13:47:32 -04:00
Joey Hess
74f0d67aa3 avoid untrappable exception if dirContentsRecursive is run on a directory
that doesn't exist, or cannot be read

The problem is its use of unsafeInterleaveIO, which causes its IO code
to run when the thunk is forced, outside any exception trapping the caller
may do.
2012-07-02 10:56:26 -04:00
Joey Hess
bea0ac0274 record transfers for git-annex-shell
Not yet tested and places git-annex-shell is run need to be modified to
pass the new field settings.

Note that rsyncServerSend was changed to fork, rather than directly exec
rsync, because it needs to keep the transfer lock held, and clean up the
transfer log when done.
2012-07-02 01:31:10 -04:00
Joey Hess
d1f49b0ad0 add fields to git-annex-shell 2012-07-02 00:53:00 -04:00
Joey Hess
7625319c2c Merge branch 'master' into assistant 2012-07-01 21:00:43 -04:00
Joey Hess
29335bf326 pointlessness 2012-06-29 10:00:05 -04:00
Joey Hess
638a321ca5 typo 2012-06-28 14:15:49 -04:00
Joey Hess
421f9ce0e2 fix kqueue build 2012-06-28 14:13:15 -04:00
Joey Hess
4888c5b042 improve thread termination handling
The reason the DirWatcher had to wait for program termination was because
it used withINotify, so when it finished, its watcher threads were killed.
But since I have two DirWatcher threads now, that was not good, and could
perhaps explain the MVar problem I saw yesterday. In any case, fixed this
part of the code by making the DirWatcher return a handle that can be used
to stop it, and now the main Assistant thread is the only one calling
waitForTermination.
2012-06-28 13:37:03 -04:00
Joey Hess
67c8ef7de2 use a TMVar
SampleMVar won't work; between getting the current value and changing
it, another thread could made a change, which would get lost.

TMVar works well; this update situation is handled by atomic transactions.
2012-06-26 19:21:44 -04:00
Joey Hess
5cfe91f06d add a push retry thread 2012-06-25 16:38:12 -04:00
Joey Hess
a71e7161fc golfing 2012-06-23 01:33:10 -04:00
Joey Hess
e699ce1841 added a merger thread
Wow! I can create a file in repo a, and it instantly* shows up in repo b!

* under 1 second anyway
2012-06-22 17:01:08 -04:00
Joey Hess
e9630e90de the syncer now pushes out changes to remotes, in parallel
Note that, since this always pushes branch synced/master to the remote, it
assumes that master has already gotten all the commits that are on the
remote merged in. Otherwise, fast-forward prevention may prevent the push.

That's probably ok, because the next stage is to automatically detect
incoming pushes and merge.
2012-06-22 15:49:48 -04:00
Joey Hess
28e28bc043 stub syncer thread and commit channel 2012-06-22 14:10:25 -04:00
Joey Hess
ad11de94e5 typo 2012-06-20 15:53:56 -04:00
Joey Hess
483b1b08c6 Merge branch 'master' into watch 2012-06-20 13:15:59 -04:00
Joey Hess
75b6ee81f9 avoid ByteString.Char8 where not needed
Its truncation behavior is a red flag, so avoid using it in these places
where only raw ByteStrings are used, without looking at the data inside.
2012-06-20 13:13:40 -04:00
Joey Hess
ee2acd474f [Word8] to filesystem encoded String
My, GHC makes this hard.
2012-06-20 12:51:25 -04:00
Joey Hess
5580af5789 add closingTracked flag 2012-06-19 10:22:36 -04:00
Joey Hess
e68b3c99f4 kqueue synthetic add events on startup 2012-06-19 10:08:06 -04:00
Joey Hess
2a61df23e7 kqueue recursive directory adding 2012-06-19 09:56:03 -04:00
Joey Hess
4ab9449cee add eventsCoalesce 2012-06-19 02:23:45 -04:00
Joey Hess
fd3e945932 fix prototype 2012-06-19 01:57:19 -04:00
Joey Hess
03b9341356 fix scheduling
Handle kevent interruptions in the haskell code, so it can yield to other threads
2012-06-19 04:52:55 +00:00
Joey Hess
02e9fdb0a5 kqueue build fix
new event dispatch seems a bit broken though
2012-06-19 04:07:48 +00:00
Joey Hess
7a09d74319 lifted out the kqueue and inotify to a generic DirWatcher interface
Kqueue code for dispatching events is not tested and probably doesn't
build.
2012-06-18 23:49:07 -04:00
Joey Hess
e164553272 robustness fixes 2012-06-19 02:13:26 +00:00
Joey Hess
5e9fdac92f update kqueue when new directories are added 2012-06-18 21:46:04 -04:00
Joey Hess
2bfcc0b09c kqueue: add directory content tracking, and change determination
This *may* now return Add or Delete Changes as appropriate. All I know
for sure is that it compiles.

I had hoped to avoid maintaining my own state about the content of the
directory tree, and rely on git to check what was changed. But I can't;
I need to know about new and deleted subdirectories to add them to the
watch list, and git doesn't deal with (empty) directories.

So, wrote all the code to scan directories, remember their past contents,
compare with current contents, generate appropriate Change events, and
update bookkeeping info appropriately.
2012-06-18 21:29:30 -04:00
Joey Hess
ae7d07ddcb close fds 2012-06-18 19:14:58 -04:00
Joey Hess
b141d9bcc8 retry interrupted kevent calls
Many thanks to geekosaur in #haskell for help with this.
2012-06-18 18:03:50 -04:00
Joey Hess
a11825a1f1 add test stub 2012-06-18 16:56:05 -04:00
Joey Hess
d680ff7ef0 kqueue code compiles on debian kfreebsd 2012-06-18 16:34:08 -04:00
Joey Hess
90d565149a flesh out kqueue library
Have not tried to build this yet. But barring minor mistakes, I think it's
good.
2012-06-18 16:18:59 -04:00
Joey Hess
89fcee03d0 add some utility functions for later
Will need to update the DirMap to add or remove subdirs.
2012-06-18 13:22:36 -04:00
Joey Hess
a39b73d118 recurse dirTree and open the directories for kqueue to watch 2012-06-18 13:01:58 -04:00
Joey Hess
dc3d9d1e98 added dirTree 2012-06-18 12:53:57 -04:00
Joey Hess
3c8a9043b6 skeleton C library for calling kqueue 2012-06-18 12:25:20 -04:00
Joey Hess
66344a3613 Enable diskfree on kfreebsd, using statvfs.
Could not reproduce the build failure I had seen related to this,
but the numbers were wrong with statfs64. Probably pulling from the wrong
place in the structure. statvfs seems to work..
2012-06-17 18:10:57 -04:00
Joey Hess
e84b78f40c reorg 2012-06-17 14:02:40 -04:00
Joey Hess
0052cec2b7 add lsof build deps
Check for it in configure; and add a --force option for people without it
who want to live dangerously.
2012-06-15 23:29:39 -04:00
Joey Hess
bb6074dfea work around a wrinkle in how lsof handles hard links to files that are open elsewhere
+d is probably more expensive, but I need it
2012-06-15 22:34:42 -04:00
Joey Hess
d91950ecba cleanup 2012-06-15 22:16:13 -04:00
Joey Hess
96ac25094b fix pid file writing
need to truncate, or part of previous longer pid may be left after writing
2012-06-15 20:42:53 -04:00
Joey Hess
85db1720e9 add lsof interface
Uses lsof -F0 to get machine-readable output
2012-06-15 15:13:31 -04:00
Joey Hess
4b2b5e820e add 2012-06-15 14:11:09 -04:00
Joey Hess
36d73b0017 slightly higher-level thread scheduling code
Including support for unbound thread sleeping. Haskell's max thread sleep
is 37 minutes, due to maxBound Int!
2012-06-13 17:53:19 -04:00
Joey Hess
12dbb9d1d0 plumb file status through to event handlers
The idea, not yet done, is to use this to detect when a file
has an old change time, and avoid expensive restaging of the file.

If git-annex watch keeps track of the last time it finished a full scan,
then any symlink that is older than that time must have been scanned
before, so need not be added. (Relying on moving, copying, etc of a file
all updating its change time.)

Anyway, this info is available for free since inotify already checks it,
so it might as well make it available.
2012-06-13 01:20:37 -04:00