Commit graph

274 commits

Author SHA1 Message Date
Joey Hess
e58d19b533 run full transfer scan on all remotes at startup
Or when a remote first becomes available after startup.
2012-08-24 13:46:10 -04:00
Joey Hess
9fafddc7eb make higher priorities and full scans be preserved when adding 2012-08-24 13:28:20 -04:00
Joey Hess
72e110ce5d avoid requeueing a download from a remote that no longer has a key 2012-08-24 13:08:38 -04:00
Joey Hess
cea55b70fc remove slightly misleading extended message 2012-08-23 19:10:20 -04:00
Joey Hess
f3721d89bc also notice dbus unmount events 2012-08-23 18:58:54 -04:00
Joey Hess
4a4f8064ae make the NetWatcher always rescan networked remotes every 30 minutes
There are multiple reasons to do this:

* The local network may be up solid, but a route to a networked remote
  is having trouble. Any transfers to it that fail should be retried.
* Someone might have wicd running, but like to bring up new networks
  by hand too. This way, it'll eventually notice them.
2012-08-23 16:13:43 -04:00
Joey Hess
37b960d3b0 don't rely on hasKeyCheap in the transfer scan
The problem with using it here is that, if a removable drive is scanned
and gets disconnected during the scan, testing for all the files will
indicate it doesn't have them, and the scan is logged as completed
successfully, without necessary transfers being queued.
2012-08-23 15:48:31 -04:00
Joey Hess
6420fa6c08 remove pointless bracket 2012-08-23 15:48:14 -04:00
Joey Hess
715a9a2f8e keep logs of failed transfers, and requeue them when doing a non-full scan
of a remote
2012-08-23 15:24:15 -04:00
Joey Hess
487bdf0e24 add transfer scanned flag files 2012-08-23 13:42:26 -04:00
Joey Hess
ab8cb05989 scan cheapest remotes first
This way, we get transfers from cheapest remotes.
2012-08-23 13:41:38 -04:00
Joey Hess
546ba8b7e1 better name 2012-08-22 15:37:26 -04:00
Joey Hess
5c3e14649e avoid unnecessary transfer scans when syncing a disconnected remote
Found a very cheap way to determine when a disconnected remote has
diverged, and has new content that needs to be transferred: Piggyback on
the git-annex branch update, which already checks for divergence.

However, this does not check if new content has appeared locally while
disconnected, that should be transferred to the remote.

Also, this does not handle cases where the two git repos are in sync,
but their content syncing has not caught up yet.

This code could have its efficiency improved:

* When multiple remotes are synced, if any one has diverged, they're
  all queued for transfer scans.
* The transfer scanner could be told whether the remote has new content,
  the local repo has new content, or both, and could optimise its scan
  accordingly.
2012-08-22 15:05:57 -04:00
Joey Hess
5d577c32a9 move some git operations outside the annex monad to avoid blocking other threads 2012-08-22 14:36:58 -04:00
Joey Hess
68659f4998 refactor 2012-08-22 14:32:17 -04:00
Joey Hess
5a68acb521 add NetWatcher thread
This deals with interruptions in network connectevity, by listening
for a new network interface coming up (using dbus to see when
network-manager or wicd do it), and forcing a rescan of
2012-08-21 19:58:53 -04:00
Joey Hess
95bfbcd3e3 make canCheckSymlink check in a parent directory if the directory DNE 2012-08-16 16:50:21 -07:00
Joey Hess
a73e271d60 run resumed transfers immediately, do not wait for free transfer slot
The resumed transfer still uses a slot, so will delay other, queued
transfers from starting.
2012-08-12 12:36:08 -04:00
Joey Hess
b6b8f6da9c implement resuming of paused transfers
Currently waits for a new transfer slot to open up, which probably needs to
change..
2012-08-12 12:11:20 -04:00
Joey Hess
37eed5d8d0 fix button display for paused transfer 2012-08-10 18:50:21 -04:00
Joey Hess
8ba9830653 implement pausing of transfers
A paused transfer's thread keeps running, keeping the slot in use.
This is intentional; pausing a transfer should not let other
queued transfers to run in its place.
2012-08-10 18:42:44 -04:00
Joey Hess
21bd92f077 send update notificaton when removing a queued transfer 2012-08-10 16:00:24 -04:00
Joey Hess
0d80406b2b remove dummy 2012-08-10 15:54:12 -04:00
Joey Hess
a76078a78e process group killing
This seems to work pretty well.

Handled the process groups like this:

- git-annex processes started by the assistant for transfers are run in their
  own process groups.
- otherwise, rely on the shell to allocate a process group for git-annex

There is potentially a problem if some other program runs git-annex
directly (not using sh -c) The program and git-annex would then be in
the same process group. If that git-annex starts a transfer and it's
canceled, the program would also get killed. May or may not be a desired
result.

Also, the new updateTransferInfo probably closes a race where it was
possible for the thread id to not be recorded in the transfer info, if
the transfer info file from the transfer process is read first.
2012-08-10 15:52:22 -04:00
Joey Hess
d5e06e7b89 fork off git-annex copy for transfers
This doesn't quite work, because canceling a transfer sends a signal
to git-annex, but not to rsync (etc).

Looked at making git-annex run in its own process group, which could then
be killed, and would kill child processes. But, rsync checks if it's
process group is the foreground process group and doesn't show progress if
not, and when git has run git-annex, if git-annex makes a new process
group, that is not the case. Also, if git has run git-annex, ctrl-c
wouldn't be propigated to it if it made a new process group.

So this seems like a blind alley, but recording it here just in case.
2012-08-10 14:14:08 -04:00
Joey Hess
20203b45b9 transfer canceling
Should work (untested) for transfers being run by other processes.

Not yet by transfers being run by the assistant. killThread does not
kill processes forked off by a thread. To fix this, will probably
need to make `git annex getkey` and `git annex sendkey` commands that
operate on keys, and write their own transfer info. Then the assistant
can run them, and kill them, as needed.
2012-08-08 17:55:56 -04:00
Joey Hess
09449792fa factor out onclick javascript 2012-08-08 17:07:38 -04:00
Joey Hess
94fcd0cf59 add routes to pause/start/cancel transfers
This commit includes a paydown on technical debt incurred two years ago,
when I didn't know that it was bad to make custom Read and Show instances
for types. As the routes need Read and Show for Transfer, which includes a
Key, and deriving my own Read instance of key was not practical,
I had to finally clean that up.

So the compact Key read and show functions are now file2key and key2file,
and Read and Show are now derived instances.

Changed all code that used the old instances, compiler checked.
(There were a few places, particularly in Command.Unused, and the test
suite where the Show instance continue to be used for legitimate
comparisons; ie show key_x == show key_y (though really in a bloom filter))
2012-08-08 16:20:24 -04:00
Joey Hess
77cd327170 fix display of icon in block alert heading 2012-08-06 17:15:27 -04:00
Joey Hess
8f1a9ef8b5 added an alert after a file transfer 2012-08-06 17:09:23 -04:00
Joey Hess
05ed196ce5 better alert message generation 2012-08-06 15:41:42 -04:00
Joey Hess
94e92a1b58 make alerts change tense when they finish 2012-08-06 15:00:46 -04:00
Joey Hess
aab3a01a71 merge two repository screens into one 2012-08-05 19:55:06 -04:00
Joey Hess
5ae1f75a39 handle case of adding populated drive to just created repo
The just created repo has no master branch commits yet. This is now
handled, merging in the master branch from the populated drive.
2012-08-05 16:35:30 -04:00
Joey Hess
a3f76fe696 wording
De-emphasize "clone", because it's not that simple. The removable drive may
already have an annex with content; if so it'll get synced in.
2012-08-05 15:57:30 -04:00
Joey Hess
34fc0d358e fix crashes when run in a git repo that has been initted but has no master branch yet 2012-08-05 15:53:47 -04:00
Joey Hess
7478872a09 fix crash when just one remote needs to be scanned
The TMVar is supposed to be left empty once the map is empty, but the code
neglected to do that, so the next time takeMVar got an empty map, which
is not handled since that was supposed to never happen..

Also, avoid any possibility of this crash. If an empty map somehow creeps
in, just retry.
2012-08-05 15:18:56 -04:00
Joey Hess
ac71ab7bd7 avoid head 2012-08-05 15:10:26 -04:00
Joey Hess
cb0f435d94 adding removable drive repos now basically works 2012-08-05 14:49:47 -04:00
Joey Hess
3add2cd3ba wire up scan and transfer to newly added removable drive
remote setup still todo
2012-08-04 21:18:57 -04:00
Joey Hess
e125ce74b8 work toward adding new repos on removable drives
This actually does add a new repo, but it doesn't yet set up
remotes, or sync to it.
2012-08-04 18:17:16 -04:00
Joey Hess
e0c3958d9a improved config 2012-08-03 20:40:34 -04:00
Joey Hess
1bd2be549f add extra sidebar widget
html is slightly broken
2012-08-03 20:10:32 -04:00
Joey Hess
b1a5a4f985 moving toward configuring new repos in the webapp 2012-08-03 14:36:16 -04:00
Joey Hess
a323b0c46d update
default description will be added elsewhere
2012-08-03 10:44:13 -04:00
Joey Hess
89120261e0 run the file browser in a separate thread to avoid blocking the browser 2012-08-03 10:18:57 -04:00
Joey Hess
1f89712e6b add a navbar button that opens the repo in the desktop's native file browser
This should work on linux (xdg-open) and OSX (open). If the program
is not in $PATH, it falls back to opening a browser window/tab with file:///

The only tricky bit is the javascript code, that handles clicking on the
link. This is to avoid unnecessary page refreshes. Until I added the
return false at the end, the <a>'s normal click event also fired, so two
file browsers opened. I have not checked portability extensively.
2012-08-03 10:02:56 -04:00
Joey Hess
74fc9fcbe6 add alert when committing 2012-08-02 14:02:35 -04:00
Joey Hess
e21a32627f avoid bogus alert errors 2012-08-02 13:57:34 -04:00
Joey Hess
a6e4283fed add slight delay in between sidebar updates, to avoid excessive churn
Tested and 0.01 seconds is not perceivable as a delay when interacting with
the UI.
2012-08-02 13:55:38 -04:00
Joey Hess
d2f9759443 refactor 2012-08-02 13:50:35 -04:00
Joey Hess
d2b48cacdb add some strictness annotations
on general principles
2012-08-02 09:20:21 -04:00
Joey Hess
1f2127c520 trim long filenames (have to fit on the sidebar)
30 characters would mostly work, but 20 is safer due to some wider letters
like 'w'. Of course this is very heuristic based on filesize anyway.

(Bootstrap does a surprisingly bad job at dealing with overlong words
in the sidebar.)
2012-08-02 09:15:08 -04:00
Joey Hess
191ee3b697 awesome alert combining
Now an alert tracks files that have recently been added. As a large file
is added, it will have its own alert, that then combines with the tracker
when dones.

Also used for combining sanity checker alerts, as it could possibly want to
display a lot.
2012-08-02 09:03:04 -04:00
Joey Hess
3695cab949 avoid showing alert when there are no remotes to push to 2012-08-02 09:00:13 -04:00
Joey Hess
22d7447ced fix tilde expansion 2012-08-02 07:55:10 -04:00
Joey Hess
9a038b4a9b better ~/ handling 2012-08-02 07:50:13 -04:00
Joey Hess
112ce4f49c support XDG_DATA_DIR 2012-08-02 07:47:39 -04:00
Joey Hess
60da0d6ad2 full autostart support
git annex assistant --autostart will start separate daemons in each
listed autostart repo

running the webapp outside any git-annex repo will open it on the
first listed autostart repo
2012-08-02 00:42:33 -04:00
Joey Hess
ffeb060002 don't use hamlet for htmlshim
This allows me to not build-depend on blaze-markup, which was causing
me some trouble when tring to build with cabal on debian. Seems debian
ships Text.Blaze.Renderer.String in two packages.
2012-08-01 21:28:37 -04:00
Joey Hess
7606f3e7c1 cleanup 2012-08-01 16:31:16 -04:00
Joey Hess
ecc168aba3 implemented firstrun repository creation and redirection to full webapp
Some of the trickiest code I've possibly ever written.
2012-08-01 16:10:26 -04:00
Joey Hess
1efe4f3332 only use smart default on first run 2012-07-31 21:34:29 -04:00
Joey Hess
b9afb7785e typo 2012-07-31 21:12:58 -04:00
Joey Hess
e81e8913d9 default repository location
Unifying poll results, it's Annex in lowercase. :)

When cwd is HOME, use ~/Desktop/annex, unless there's no Desktop directory;
then use use ~/annex

If cwd is not $HOME, use cwd
2012-07-31 21:06:30 -04:00
Joey Hess
bab80bf24a full input validation for repository path
Expands ~ , checks for every crazy input problem I can think of
2012-07-31 20:56:10 -04:00
Joey Hess
c950e8fba0 move out to template 2012-07-31 18:33:19 -04:00
Joey Hess
4b5ffe8f9b implemented the addrepository form
shiny!
2012-07-31 17:57:08 -04:00
Joey Hess
c70496dc7f improve first run screen 2012-07-31 16:13:09 -04:00
Joey Hess
04794eafc0 webapp now starts up when run not in a git repo 2012-07-31 12:17:31 -04:00
Joey Hess
b9b0097876 tweak types so the webapp can run without a threadstate when outside an annex 2012-07-31 11:19:40 -04:00
Joey Hess
e9d9d9d5ea add icons 2012-07-31 03:10:16 -04:00
Joey Hess
f0a88e1203 change url 2012-07-31 02:36:18 -04:00
Joey Hess
02b345249b stub 2012-07-31 02:35:07 -04:00
Joey Hess
2c8bbdf307 made navbar work
also added an About page and a stub Config page.
2012-07-31 02:30:26 -04:00
Joey Hess
5fed026bcd reorg templates 2012-07-31 01:24:49 -04:00
Joey Hess
58dfa3fa5b split up webapp files 2012-07-31 01:11:32 -04:00
Joey Hess
6e40aed948 fix noscript mode to not allocate notification ids on each refresh
Now the javascript does an ajax call at the start to request the url
to use to poll, and the notification id is generated then, once we know
javascript is working.
2012-07-30 22:24:19 -04:00
Joey Hess
9b1ed7a720 Merge branch 'assistant' into assistant-longpolling-rewrite 2012-07-30 19:40:16 -04:00
Joey Hess
5de897e8d7 remove the "running" alert 2012-07-30 18:26:36 -04:00
Joey Hess
2e54a611c7 Merge branch 'assistant' into assistant-longpolling-rewrite 2012-07-30 18:19:24 -04:00
Joey Hess
254c174bba fix transfers display logic 2012-07-30 18:18:53 -04:00
Joey Hess
502bc5d5f8 rewrote longpolling, trying to avoid duplication
does not work though. stupid JS
2012-07-30 18:01:41 -04:00
Joey Hess
b2e359a15d fix kqueue build 2012-07-30 16:32:32 -04:00
Joey Hess
5469bd6e42 remove old filler that is effectively the same as new filler 2012-07-30 15:33:12 -04:00
Joey Hess
78b3dada5b better connection close handling
Depending on how the webapp was started up and whether the user clicked on
any links in it, window.close() may be disallowed by browser security
policy. Also if that fails, display a modal dialog that nicely blackens out
the webapp.

TODO: avoid Escape closing it. Bootstrap's docs are unclear about how to do
that.
2012-07-30 14:50:32 -04:00
Joey Hess
a994130843 implement server-side alert closing
Rather than using bootstrap's client-side closing.
Now closed alerts stay closed.
2012-07-30 14:08:22 -04:00
Joey Hess
1f671ee40c spruce up display of the repo list 2012-07-30 13:31:19 -04:00
Joey Hess
9c9db6feb8 make filler closeable 2012-07-30 12:23:40 -04:00
Joey Hess
8d2667715b prune old filler alerts 2012-07-30 12:21:53 -04:00
Joey Hess
40c9973675 fix push status, broken when inParallel was adapted for -threaded
Before pushing ran in its own process, so exitSuccess was the right thing
to do, but with the threaded code, that's caught as an exception.
2012-07-30 11:52:44 -04:00
Joey Hess
3dce75fb23 make old activiy alerts stay visible
They're updated to show whether the activity succeeded or failed.

This adds several TODOs to the code to fix later.
2012-07-30 02:39:24 -04:00
Joey Hess
ec0493fa4d filter out dead repos from the intro 2012-07-30 00:31:33 -04:00
Joey Hess
74cf65a4dd avoid first person 2012-07-29 22:18:58 -04:00
Joey Hess
895b068e35 tweak intro 2012-07-29 22:11:01 -04:00
Joey Hess
326617ad2f add intro 2012-07-29 21:54:23 -04:00
Joey Hess
0186f06744 tweak Alert closability and construction 2012-07-29 19:41:17 -04:00
Joey Hess
d62b157194 better ordering of alerts 2012-07-29 19:05:51 -04:00
Joey Hess
d52c932424 moved all alert messages into one file
Makes it easier to edit for consistent voice etc.
2012-07-29 18:09:24 -04:00
Joey Hess
b2dc8fdb06 add more alerts
Nearly all long-running actions now display an alert.
2012-07-29 17:53:18 -04:00