Commit graph

216 commits

Author SHA1 Message Date
Joey Hess
cb0f435d94 adding removable drive repos now basically works 2012-08-05 14:49:47 -04:00
Joey Hess
3add2cd3ba wire up scan and transfer to newly added removable drive
remote setup still todo
2012-08-04 21:18:57 -04:00
Joey Hess
74fc9fcbe6 add alert when committing 2012-08-02 14:02:35 -04:00
Joey Hess
e21a32627f avoid bogus alert errors 2012-08-02 13:57:34 -04:00
Joey Hess
191ee3b697 awesome alert combining
Now an alert tracks files that have recently been added. As a large file
is added, it will have its own alert, that then combines with the tracker
when dones.

Also used for combining sanity checker alerts, as it could possibly want to
display a lot.
2012-08-02 09:03:04 -04:00
Joey Hess
3695cab949 avoid showing alert when there are no remotes to push to 2012-08-02 09:00:13 -04:00
Joey Hess
9a038b4a9b better ~/ handling 2012-08-02 07:50:13 -04:00
Joey Hess
ffeb060002 don't use hamlet for htmlshim
This allows me to not build-depend on blaze-markup, which was causing
me some trouble when tring to build with cabal on debian. Seems debian
ships Text.Blaze.Renderer.String in two packages.
2012-08-01 21:28:37 -04:00
Joey Hess
ecc168aba3 implemented firstrun repository creation and redirection to full webapp
Some of the trickiest code I've possibly ever written.
2012-08-01 16:10:26 -04:00
Joey Hess
04794eafc0 webapp now starts up when run not in a git repo 2012-07-31 12:17:31 -04:00
Joey Hess
b9b0097876 tweak types so the webapp can run without a threadstate when outside an annex 2012-07-31 11:19:40 -04:00
Joey Hess
2c8bbdf307 made navbar work
also added an About page and a stub Config page.
2012-07-31 02:30:26 -04:00
Joey Hess
58dfa3fa5b split up webapp files 2012-07-31 01:11:32 -04:00
Joey Hess
6e40aed948 fix noscript mode to not allocate notification ids on each refresh
Now the javascript does an ajax call at the start to request the url
to use to poll, and the notification id is generated then, once we know
javascript is working.
2012-07-30 22:24:19 -04:00
Joey Hess
9b1ed7a720 Merge branch 'assistant' into assistant-longpolling-rewrite 2012-07-30 19:40:16 -04:00
Joey Hess
5de897e8d7 remove the "running" alert 2012-07-30 18:26:36 -04:00
Joey Hess
2e54a611c7 Merge branch 'assistant' into assistant-longpolling-rewrite 2012-07-30 18:19:24 -04:00
Joey Hess
254c174bba fix transfers display logic 2012-07-30 18:18:53 -04:00
Joey Hess
502bc5d5f8 rewrote longpolling, trying to avoid duplication
does not work though. stupid JS
2012-07-30 18:01:41 -04:00
Joey Hess
b2e359a15d fix kqueue build 2012-07-30 16:32:32 -04:00
Joey Hess
78b3dada5b better connection close handling
Depending on how the webapp was started up and whether the user clicked on
any links in it, window.close() may be disallowed by browser security
policy. Also if that fails, display a modal dialog that nicely blackens out
the webapp.

TODO: avoid Escape closing it. Bootstrap's docs are unclear about how to do
that.
2012-07-30 14:50:32 -04:00
Joey Hess
a994130843 implement server-side alert closing
Rather than using bootstrap's client-side closing.
Now closed alerts stay closed.
2012-07-30 14:08:22 -04:00
Joey Hess
1f671ee40c spruce up display of the repo list 2012-07-30 13:31:19 -04:00
Joey Hess
8d2667715b prune old filler alerts 2012-07-30 12:21:53 -04:00
Joey Hess
40c9973675 fix push status, broken when inParallel was adapted for -threaded
Before pushing ran in its own process, so exitSuccess was the right thing
to do, but with the threaded code, that's caught as an exception.
2012-07-30 11:52:44 -04:00
Joey Hess
3dce75fb23 make old activiy alerts stay visible
They're updated to show whether the activity succeeded or failed.

This adds several TODOs to the code to fix later.
2012-07-30 02:39:24 -04:00
Joey Hess
ec0493fa4d filter out dead repos from the intro 2012-07-30 00:31:33 -04:00
Joey Hess
895b068e35 tweak intro 2012-07-29 22:11:01 -04:00
Joey Hess
326617ad2f add intro 2012-07-29 21:54:23 -04:00
Joey Hess
0186f06744 tweak Alert closability and construction 2012-07-29 19:41:17 -04:00
Joey Hess
d62b157194 better ordering of alerts 2012-07-29 19:05:51 -04:00
Joey Hess
d52c932424 moved all alert messages into one file
Makes it easier to edit for consistent voice etc.
2012-07-29 18:09:24 -04:00
Joey Hess
b2dc8fdb06 add more alerts
Nearly all long-running actions now display an alert.
2012-07-29 17:53:18 -04:00
Joey Hess
ce7889ba86 debuggery 2012-07-29 14:10:17 -04:00
Joey Hess
c4023f7858 probably fixes http://git-annex.branchable.com/bugs/lsof__47__committer_thread_loops_occassionally/ 2012-07-29 13:55:07 -04:00
Joey Hess
ebd8362d58 fix bug in transfer initiation checking
Putting the transfer on the currentTransfers atomically introduced a bug:
It checks to see if the transfer is in progress, and cancels it.

Fixed by moving that check inside the STM transaction.
2012-07-29 13:37:26 -04:00
Joey Hess
09e77a0cf0 add some alerts 2012-07-29 13:22:08 -04:00
Joey Hess
e1d4bfe671 typo; was waiting on the wrong notifier for the sidebar! 2012-07-29 12:37:45 -04:00
Joey Hess
c2f3e66d8c show alerts in the sidebar
This has a bug -- it seems long polling can only wait on one page at a
time. Need to re-unify the notifiers.
2012-07-29 11:31:06 -04:00
Joey Hess
57203e3981 refactor 2012-07-29 08:52:57 -04:00
Joey Hess
62dac85880 update the sidebar by long polling
Needs to use a different NotificationBroadcaster, and not replace the
whole sidebar div, but instead add in new content. However, it's 3:30 am.
2012-07-29 03:23:17 -04:00
Joey Hess
38ade1af70 better noscript UI 2012-07-29 00:55:22 -04:00
Joey Hess
376f8443c1 add a separate page for noscript browsers
This may be customised differently than the main page later on, but
for now the important thing is that this constantly refreshed page does not
allocate a new NotificationHandle each time it's loaded.
2012-07-29 00:08:14 -04:00
Joey Hess
a498be7f98 renamed /status to /transfers
Also fixed a bug; the ident for the div was regnerated each time
/status was called. This only was the same as the original ident due to
luck.
2012-07-28 23:55:41 -04:00
Joey Hess
9b18dc2a39 tune javascript refresh delays
WebApp now shows changes with no delay. Comparing a running git-annex get
and the webapp side-by-side, they both show each new transfer at the same
time.
2012-07-28 21:25:56 -04:00
Joey Hess
6a9abf6526 add NotificationID to StatusR, and use it to block 2012-07-28 21:21:22 -04:00
Joey Hess
e31277d38a send notifications when the TransferQueue is changed
The fun part was making it move things from TransferQueue to currentTransfers
entirely atomically. Which will avoid inconsistent display if the WebApp
renders the current status at just the wrong time. STM to the rescue!
2012-07-28 18:47:24 -04:00
Joey Hess
3cc1885793 move DaemonStatus manipulation out of the Annex monad to IO
I've convinced myself that nothing in DaemonStatus can deadlock,
as it always keepts the TMVar full. That was the only reason it was in the
Annex monad.
2012-07-28 18:02:11 -04:00
Joey Hess
c0ca6f44ac template reorg 2012-07-27 20:47:48 -04:00
Joey Hess
ecd63e2981 template cleanup
use julius's nice #id and .class things
2012-07-27 20:25:28 -04:00
Joey Hess
7717501fee add alert close buttons 2012-07-27 16:28:00 -04:00
Joey Hess
02ec8ea012 much better webapp startup of the assistant
This avoids forking another process, avoids polling, fixes a race,
and avoids a rare forkProcess thread hang that I saw once time
when starting the webapp.
2012-07-27 15:33:24 -04:00
Joey Hess
0f6292920a webapp now displays the real running and queued transfers
yowza!!!
2012-07-27 11:47:34 -04:00
Joey Hess
7e3c1e008d webapp now uses twitter bootstrap
mocked up the main screen, and am actually pretty happy with it!
2012-07-27 04:48:50 -04:00
Joey Hess
1983ca2852 added jquery to static site
Had to switch to toWaiAppPlain to avoid a seeming bug in toWaiApp;
chromium only received a partial copy of jquery. Always the same length
each time, which makes me think it's a bug in the compression, although
a bug in the autohead middleware is also a possibility.

Anyway, there's little need for compression for a local webapp. Not wasting
time compressing things is probably a net gain.

Similarly, I've not worried about minifying this yet. Although that would
avoid bloating the git-annex binary quite so much.
2012-07-26 23:55:51 -04:00
Joey Hess
615dc09ffc use widgetFile 2012-07-26 22:54:31 -04:00
Joey Hess
e40f94cbcd add threadState member, will need this later to access the daemonStatus 2012-07-26 21:51:56 -04:00
Joey Hess
f5ef46d01e cleaned up refreshing code into a widget
Very happy to have a reusable autoUpdate widget that can make any Yesod
widget automatically refresh!

Also added support for non-javascript browsers, falling back to meta
refresh.

Also, the home page is now rendered with the webapp status on it, before
any refreshing is done.
2012-07-26 21:03:46 -04:00
Joey Hess
9fd03c65f9 webapp now does long polling
The webapp is now a constantly updating clock! I accomplished this amazing
feat using "long polling", with some jquery and a little custom java
script.

There are more modern techniques, but this one works everywhere.
2012-07-26 17:56:24 -04:00
Joey Hess
95f4b192f0 on second thought, the session cookie is still useful to support setMessage 2012-07-26 12:41:20 -04:00
Joey Hess
a453be4195 disable client session cookie 2012-07-26 12:10:53 -04:00
Joey Hess
b89b801567 update 2012-07-26 05:20:52 -04:00
Joey Hess
9d6b59d0e2 use the secret token for authentication, and add to all dynamic urls 2012-07-26 04:50:09 -04:00
Joey Hess
b36804d648 generate random token and launch webapp using it 2012-07-26 03:38:20 -04:00
Joey Hess
3ac2cf09e5 added a custom defaultLayout, static site, and favicon
Broke hamlet out into standalone files.

I don't like the favicon display; it should be served from /favicon.ico,
but I could only get the static site to serve /static/favicon.ico, so
I had to use a <link rel=icon> to pull it in. I looked at
Yesod.Default.Handlers.getFaviconR, but it doesn't seem to embed
the favicon into the binary?
2012-07-26 02:45:01 -04:00
Joey Hess
6a8540c1a2 tweak 2012-07-26 00:39:25 -04:00
Joey Hess
805d50c69d use hamlet at least for the static html 2012-07-25 23:50:29 -04:00
Joey Hess
1ffef3ad75 git annex webapp now opens a browser to the webapp
Also, starts the assistant if it wasn't already running.
2012-07-25 23:13:01 -04:00
Joey Hess
e6ce54de82 build fixes 2012-07-25 22:04:10 -04:00
Joey Hess
32d3cffc4c run yesod, and launch webapp on startup 2012-07-25 21:26:13 -04:00
Joey Hess
95c80b6440 laziness fix
Now scanning runs fully interleaved with transferring.
2012-07-25 14:54:09 -04:00
Joey Hess
b665ffe36f implement simple working copy based scan
Works.. could be more efficient.
2012-07-25 14:16:53 -04:00
Joey Hess
a9dbfdf28d better transfer queue management
Allow transfers to be added with blocking until the queue is sufficiently
small.

Better control over which end of the queue to add a transfer to.
2012-07-25 13:12:34 -04:00
Joey Hess
6107328a6b avoid spawning new transfer thread until a slot becomes available 2012-07-25 12:07:30 -04:00
Joey Hess
522f568450 add TransferScanner thread
Efficiently finding transfers that need to be done to get two repos back
in sync seems like an interesting problem.
2012-07-22 23:16:56 -04:00
Joey Hess
26e4e65307 filter out special remotes when pulling 2012-07-22 15:09:40 -04:00
Joey Hess
e4f714d1be pull from newly mounted git remotes 2012-07-22 15:06:18 -04:00
Joey Hess
b48d7747a3 debugging improvements
add timestamps to debug messages

Add lots of debug output in the assistant's threads.
2012-07-20 19:29:59 -04:00
Joey Hess
42e73537d1 detect KDE automounting
Best dbus events I could find were setupDone from org.kde.Solid.Device.
There may be some spurious events, but that's ok, the code will only
check to see if new mounts are available.

It does not try to auto-start this service if it's not running.
2012-07-20 18:14:57 -04:00
Joey Hess
2fce3940b5 catch all errors 2012-07-20 02:16:09 -04:00
Joey Hess
d9f26115c3 use dbus to activate GduVolumeMonitor if it's not already running 2012-07-20 01:59:21 -04:00
Joey Hess
6b4fe507f6 only use dbus when there's a client connected we know will send mount events 2012-07-19 23:34:33 -04:00
Joey Hess
0496a3971d store whole Mntents
This way, if a mount point was already mounted, but something else
gets mounted there, it'll be seen as a new mount.
2012-07-19 21:25:26 -04:00
Joey Hess
107a7b9388 try to make Utility.Mounts portable
This is an unholy mashup, but it just might work. It works on Linux,
that's all I've tested. :)
2012-07-19 20:38:58 -04:00
Joey Hess
f20a40f9d4 MountWatcher thread
Currently only prints mount points when mounts happen.
2012-07-19 13:04:33 -04:00
Joey Hess
cf47bb3f50 run file transfers in threads, not processes
This should fix OSX/BSD issues with not noticing transfer information
files with kqueue. Now that threads are used, the thread can manage the
transfer slot allocation and deallocation by itself; much cleaner.
2012-07-18 19:15:34 -04:00
Joey Hess
eea0a3616c add thread id field to transferinfo
Also converted its timestand to posix seconds, like is used in the other
log files.
2012-07-18 18:42:41 -04:00
Joey Hess
f520a2c103 add missing imports 2012-07-18 18:29:33 -04:00
Joey Hess
b7d3cefde9 merge two shouldTransfer checks 2012-07-17 12:06:35 -04:00
Joey Hess
c8691d76aa bugfix 2012-07-07 11:17:20 -06:00
Joey Hess
cd168c6cba fix transferrer thread's use of transfer slots and transfer info files
Check first if a transfer needs to be done, using the location log only
(for speed), and avoid occupying a slot if not. Always write a transfer
info file, and keep it open throughout the tranfer process.

Now transfers to remotes seem reliable.
2012-07-07 10:50:20 -06:00
Joey Hess
cc6f660752 fix transfer slots blocking and refilling when transfers are stopped
There's a bug, if a transfer process notices it needs to do nothing,
it never starts the transfer, so the slot is never freed.
2012-07-06 21:59:45 -06:00
Joey Hess
d954a0ce59 fixed close-together transfer race
The issue involved forking and they trying to read from a MVar. Reading the
MVar 1st fixed it.
2012-07-06 18:48:51 -06:00
Joey Hess
62876502c5 wait on child transfer processes, and invalidate cache
There's still a bug; if the child updates its transfer info file,
then the data from it will superscede the TransferInfo, losing the
info that we should wait on this child.
2012-07-06 16:44:13 -06:00
Joey Hess
4a10795144 logic error 2012-07-06 15:07:42 -06:00
Joey Hess
3d30a45e72 simplified
background transferrs seem to work now
2012-07-06 14:54:07 -06:00
Joey Hess
430ad8ce85 it builds again
Currently nothing waits on transfer processes.

(Second drive of the day fried. Not concentrating very well.)
2012-07-06 16:41:37 -04:00
Joey Hess
8795a392c3 fix 2012-07-06 16:30:55 -04:00
Joey Hess
721748135b fix build (almost) 2012-07-06 14:42:45 -04:00
Joey Hess
a92f5589fc unfinished (and unbuildable) work toward separate transfer processes 2012-07-05 18:57:06 -06:00
Joey Hess
9eaba58dd9 run transfer with copy of annex state
This should have made it run concurrently with other annex actions,
but I'm still seeing it serialize. Perhaps I need to forkProcess?
2012-07-05 16:07:49 -06:00
Joey Hess
2136ee4adb logic error 2012-07-05 14:45:21 -06:00
Joey Hess
71b5ad8398 wrote transfer thread
finally!
2012-07-05 14:34:20 -06:00
Joey Hess
6af319d8cd enqueue Downloads when new symlinks appear to content we don't have 2012-07-05 10:58:49 -06:00
Joey Hess
83c66ccaf8 queue Uploads of newly added files to remotes
Added knownRemotes to DaemonStatus. This list is not entirely trivial to
calculate, and having it here should make it easier to add/remove remotes
on the fly later on. It did require plumbing the daemonstatus through to
some more threads.
2012-07-05 10:21:22 -06:00
Joey Hess
b4917bd18f add transfer watching thread
Worked the 1st try!
2012-07-03 10:58:40 -04:00
Joey Hess
a3636602ab MVar deadlock problem seems to be fixed by previous commit 2012-06-28 13:47:15 -04:00
Joey Hess
4888c5b042 improve thread termination handling
The reason the DirWatcher had to wait for program termination was because
it used withINotify, so when it finished, its watcher threads were killed.
But since I have two DirWatcher threads now, that was not good, and could
perhaps explain the MVar problem I saw yesterday. In any case, fixed this
part of the code by making the DirWatcher return a handle that can be used
to stop it, and now the main Assistant thread is the only one calling
waitForTermination.
2012-06-28 13:37:03 -04:00
Joey Hess
40f357fdcf tweak 2012-06-28 13:04:02 -04:00
Joey Hess
fb51d99951 merge conflict resolution now working
Avoid MVar deadlock issue, which I don't understand.
Have not taken the time to debug it fully, because it turns out I don't
need to resolve merge conflicts when a new branch ref is written... I
think.

Ensure the git-annex branch is merged when doing a manual pull.
Otherwise it can get out of sync, since git-annex normally only merges it
once per run.
2012-06-27 20:50:50 -04:00
Joey Hess
59b5266ad1 actually fetch from remote when doing a manual pull
forgot to do this
2012-06-27 20:30:04 -04:00
Joey Hess
783bee285f automatic conflict resolution for assistant 2012-06-27 20:06:21 -04:00
Joey Hess
67c8ef7de2 use a TMVar
SampleMVar won't work; between getting the current value and changing
it, another thread could made a change, which would get lost.

TMVar works well; this update situation is handled by atomic transactions.
2012-06-26 19:21:44 -04:00
Joey Hess
05c4dfb941 fixup merges now done when needed 2012-06-25 20:16:30 -04:00
Joey Hess
5cfe91f06d add a push retry thread 2012-06-25 16:38:12 -04:00
Joey Hess
0b146f9ecc reorg threads 2012-06-25 16:10:24 -04:00