Merge branch 'master' into assistant
This commit is contained in:
Joey Hess
commit
de792c35e4
9 changed files with 178 additions and 7 deletions
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="jtang"
|
||||
ip="79.97.135.214"
|
||||
subject="comment 10"
|
||||
date="2012-07-29T10:49:05Z"
|
||||
content="""
|
||||
This is looking good, no more segfaulting.
|
||||
"""]]
|
||||
49
doc/bugs/lsof__47__committer_thread_loops_occassionally.mdwn
Normal file
49
doc/bugs/lsof__47__committer_thread_loops_occassionally.mdwn
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
I've noticed this to occur occassionally
|
||||
|
||||
<pre>
|
||||
laplace:atest jtang$ ls
|
||||
1@ 2@ 3@ 4@ 5@ readme.txt@
|
||||
laplace:atest jtang$ git annex watch -d --foreground
|
||||
watch . [2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","show-ref","git-annex"]
|
||||
[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","show-ref","--hash","refs/heads/git-annex"]
|
||||
[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","log","refs/heads/git-annex..f85faa60e73efabc2e92f837b19c3918d3ab030f","--oneline","-n1"]
|
||||
[2012-07-29 11:49:26 IST] chat: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","cat-file","--batch"]
|
||||
(scanning...) [2012-07-29 11:49:26 IST] Assistant: all threads started
|
||||
[2012-07-29 11:49:26 IST] Merger: watching /Users/jtang/sandbox/atest/.git/refs/heads/synced
|
||||
[2012-07-29 11:49:26 IST] TransferWatcher: watching for transfers
|
||||
[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"]
|
||||
[2012-07-29 11:49:26 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","add","--update"]
|
||||
[2012-07-29 11:49:26 IST] Merger: merging changes into Just refs/heads/master
|
||||
[2012-07-29 11:49:26 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","merge","--no-edit","refs/heads/synced/master"]
|
||||
(started...) [2012-07-29 11:49:26 IST] Watcher: watching .
|
||||
[2012-07-29 11:49:26 IST] WebApp: running on port 60042
|
||||
Already up-to-date.
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./1
|
||||
[2012-07-29 11:49:26 IST] chat: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","cat-file","--batch"]
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./2
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./3
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./4
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./5
|
||||
[2012-07-29 11:49:26 IST] Watcher: add symlink ./readme.txt
|
||||
[2012-07-29 11:49:27 IST] Committer: committing 6 changes
|
||||
(Recording state in git...)
|
||||
[2012-07-29 11:49:27 IST] feed: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","update-index","-z","--index-info"]
|
||||
[2012-07-29 11:49:27 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","commit","--allow-empty-message","-m","","--allow-empty","--quiet"]
|
||||
[2012-07-29 11:49:28 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"]
|
||||
[2012-07-29 11:49:28 IST] Pusher: pushing to []
|
||||
[2012-07-29 11:49:28 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","branch","-f","synced/master"]
|
||||
[2012-07-29 11:49:28 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"]
|
||||
[2012-07-29 11:49:28 IST] Merger: merging changes into Just refs/heads/master
|
||||
[2012-07-29 11:49:28 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","merge","--no-edit","refs/heads/synced/master"]
|
||||
Already up-to-date.
|
||||
[2012-07-29 11:49:43 IST] Watcher: file added ./.gitignore
|
||||
[2012-07-29 11:49:43 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"]
|
||||
[2012-07-29 11:49:44 IST] Committer: delaying commit of 0 changes
|
||||
[2012-07-29 11:49:45 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"]
|
||||
[2012-07-29 11:49:46 IST] Committer: delaying commit of 0 changes
|
||||
[2012-07-29 11:49:47 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"]
|
||||
[2012-07-29 11:49:48 IST] Committer: delaying commit of 0 changes
|
||||
[2012-07-29 11:49:49 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"]
|
||||
</pre>
|
||||
|
||||
I ran " git annex watch -d --foreground" to watch what was going one, and just created a .gitignore file and the the commiter/lsof thread just loops over and over.... I only noticed as my laptop battery had drained at somepoint when git-annex was running in the background.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.153.8.133"
|
||||
subject="probably a kqueue specific problem..."
|
||||
date="2012-07-29T18:10:12Z"
|
||||
content="""
|
||||
I don't think this would eat your battery badly; it's looping for sure, but with a hardcoded 1 second delay I put in to guard against it eating all CPU.
|
||||
|
||||
I tried to fix this in commit c4023f785834bc237e5fcdb69e275bbae10dd40b, but I sort of doubt I did.
|
||||
I made one more commit that will at least tell us what file it is trying to check over and over with lsof.
|
||||
"""]]
|
||||
|
|
@ -26,6 +26,10 @@ References:
|
|||
* <http://stackoverflow.com/questions/5151858/running-a-haskell-program-on-the-android-os>
|
||||
* <http://www.reddit.com/r/haskell/comments/ful84/haskell_on_android/>
|
||||
|
||||
I've heard anecdoally that ipwnstudios not only has an IPhone GHC port,
|
||||
but also Android. Need to get in touch with them.
|
||||
<http://ipwnstudios.com/>
|
||||
|
||||
### Android specific features
|
||||
|
||||
The app should be aware of power status, and avoid expensive background
|
||||
|
|
|
|||
68
doc/design/assistant/blog/day_46__notification_pools.mdwn
Normal file
68
doc/design/assistant/blog/day_46__notification_pools.mdwn
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
Focus today was writing a notification broadcaster library. This is a way to
|
||||
send a notification to a set of clients, any of which can be blocked
|
||||
waiting for a new notification to arrive. A complication is that any number
|
||||
of clients may be be dead, and we don't want stale notifications for those
|
||||
clients to pile up and leak memory.
|
||||
|
||||
It took me 3 tries to find the solution, which turns out to be head-smackingly
|
||||
simple: An array of SampleVars, one per client.
|
||||
|
||||
Using SampleVars means that clients only see the most recent notification,
|
||||
but when the notification is just "the assistant's state changed somehow;
|
||||
display a refreshed rendering of it", that's sufficient.
|
||||
|
||||
----
|
||||
|
||||
First use of that was to make the thread that woke up every 10 minutes
|
||||
and checkpointed the daemon status to disk also wait for a notification
|
||||
that it changed. So that'll be more current, and use less IO.
|
||||
|
||||
----
|
||||
|
||||
Second use, of course, was to make the WebApp block long polling clients
|
||||
until there is really a change since the last time the client polled.
|
||||
|
||||
To do that, I made one change to my Yesod routes:
|
||||
|
||||
[[!format diff """
|
||||
-/status StatusR GET
|
||||
+/status/#NotificationId StatusR GET
|
||||
"""]]
|
||||
|
||||
Now I find another reason to love Yesod, because after doing that,
|
||||
I hit "make".. and fixed the type error. And hit make.. and fixed
|
||||
the type error. And then it just freaking worked! Html was generated with
|
||||
all urls to /status including a `NotificationId`, and the handler for
|
||||
that route got it and was able to use it:
|
||||
|
||||
[[!format haskell """
|
||||
{- Block until there is an updated status to display. -}
|
||||
b <- liftIO $ getNotificationBroadcaster webapp
|
||||
liftIO $ waitNotification $ notificationHandleFromId b nid
|
||||
"""]]
|
||||
|
||||
And now the WebApp is able to display transfers in realtime!
|
||||
When I have both the WebApp and `git annex get` running on the same screen,
|
||||
the WebApp displays files that git-annex is transferring about as fast
|
||||
as the terminal updates.
|
||||
|
||||
The [[progressbars]] still need to be sorted out, but otherwise
|
||||
the WebApp is a nice live view of file transfers.
|
||||
|
||||
---
|
||||
|
||||
I also had some fun with Software Transactional Memory. Now when the
|
||||
assistant moves a transfer from its queue of transfers to do, to its map of
|
||||
transfers that are currently running, it does so in an atomic transaction.
|
||||
This will avoid the transfer seeming to go missing (or be listed twice) if
|
||||
the webapp refreshes at just the wrong point in time. I'm really starting
|
||||
to get into STM.
|
||||
|
||||
----
|
||||
|
||||
Next up, I will be making the WebApp maintain a list of notices, displayed
|
||||
on its sidebar, scrolling new notices into view, and removing ones the user
|
||||
closes, and ones that expire. This will be used for displaying errors, as
|
||||
well as other communication with the user (such as displaying a notice
|
||||
while a git sync is in progress with a remote, etc). Seems worth doing now,
|
||||
so the basic UI of the WebApp is complete with no placeholders.
|
||||
14
doc/design/assistant/blog/day_47__alert_messages.mdwn
Normal file
14
doc/design/assistant/blog/day_47__alert_messages.mdwn
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
Some days I spend 2 hours chasing red herrings (like "perhaps my JSON ajax
|
||||
calls arn't running asynchronoously?") that turn out to be a simple
|
||||
one-word typo. This was one of them.
|
||||
|
||||
However, I did get the sidebar displaying alert messages, which can be
|
||||
easily sent to the user from any part of the assistant. This includes
|
||||
transient alerts of things it's doing, which disappear once the action
|
||||
finishes, and long-term alerts that are displayed until the user closes
|
||||
them. It even supports rendering arbitrary Yesod widgets as alerts, so
|
||||
they can also be used for asking questions, etc.
|
||||
|
||||
Time for a screencast!
|
||||
|
||||
<video controls src="http://joeyh.name/screencasts/git-annex-webapp.ogg"></video>
|
||||
|
|
@ -0,0 +1,18 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkpEY8WTFDhjIVTWG38Ph7ppmuXUTJAHAg"
|
||||
nickname="Justin"
|
||||
subject="selective sync"
|
||||
date="2012-07-28T04:08:00Z"
|
||||
content="""
|
||||
hey joey
|
||||
|
||||
great work!!
|
||||
|
||||
will partial content work like selective sync in dropbox
|
||||
|
||||
use case: on desktop i have photos/mp3s/docs, but would only want to sync the docs to my netbook
|
||||
|
||||
cheers
|
||||
|
||||
justin
|
||||
"""]]
|
||||
|
|
@ -16,7 +16,7 @@ The webapp is a web server that displays a shiny interface.
|
|||
|
||||
## interface
|
||||
|
||||
* list of files uploading and downloading
|
||||
* list of files uploading and downloading **done**
|
||||
* progress bars for each file
|
||||
* drag and drop to reorder
|
||||
* cancel and pause
|
||||
|
|
@ -27,15 +27,14 @@ The webapp is a web server that displays a shiny interface.
|
|||
* there could be a UI to export a file, which would make it be served up
|
||||
over http by the web app
|
||||
* Display any relevant warning messages. One is the `inotify max_user_watches`
|
||||
exceeded message. Need to lift such messages into DaemonStatus
|
||||
so the WebApp can include them in its rendering of DaemonStatus.
|
||||
exceeded message.
|
||||
|
||||
## implementation
|
||||
|
||||
* perhaps define a custom `errorHandler`, which could avoid the potential
|
||||
of leaking auth tokens on error pages. Or make the test suite test for
|
||||
leakage.
|
||||
* possibly lose the ugly auth= token past the first page,
|
||||
and use a client-side session. It could be encrypted using the token
|
||||
as the `encryptKey`. Note: Would need to set the session duration
|
||||
to infinite (how?)
|
||||
* Fix notification handle leakage on pages other than the main page.
|
||||
The javascript should use AJAX to request handles, that way
|
||||
they won't be allocated at all in noscript.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
The `git annex sync` command provides an easy way to keep several
|
||||
repositories in sync.
|
||||
|
||||
Often git is used in a centralized fashion with a central bare repositry
|
||||
Often git is used in a centralized fashion with a central bare repository
|
||||
which changes are pulled and pushed to using normal git commands.
|
||||
That works fine, if you don't mind having a central repository.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue