diff --git a/doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_10_6c872dff4fcc63c16bf69d1e96891c89._comment b/doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_10_6c872dff4fcc63c16bf69d1e96891c89._comment new file mode 100644 index 0000000000..952ffadc4d --- /dev/null +++ b/doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_10_6c872dff4fcc63c16bf69d1e96891c89._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="jtang" + ip="79.97.135.214" + subject="comment 10" + date="2012-07-29T10:49:05Z" + content=""" +This is looking good, no more segfaulting. +"""]] diff --git a/doc/bugs/lsof__47__committer_thread_loops_occassionally.mdwn b/doc/bugs/lsof__47__committer_thread_loops_occassionally.mdwn new file mode 100644 index 0000000000..eb8a0ead0a --- /dev/null +++ b/doc/bugs/lsof__47__committer_thread_loops_occassionally.mdwn @@ -0,0 +1,49 @@ +I've noticed this to occur occassionally + +
+laplace:atest jtang$ ls
+1@  2@	3@  4@	5@  readme.txt@
+laplace:atest jtang$ git annex watch -d --foreground
+watch . [2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","show-ref","git-annex"] 
+[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","show-ref","--hash","refs/heads/git-annex"] 
+[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","log","refs/heads/git-annex..f85faa60e73efabc2e92f837b19c3918d3ab030f","--oneline","-n1"] 
+[2012-07-29 11:49:26 IST] chat: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","cat-file","--batch"] 
+(scanning...) [2012-07-29 11:49:26 IST] Assistant: all threads started
+[2012-07-29 11:49:26 IST] Merger: watching /Users/jtang/sandbox/atest/.git/refs/heads/synced
+[2012-07-29 11:49:26 IST] TransferWatcher: watching for transfers
+[2012-07-29 11:49:26 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"] 
+[2012-07-29 11:49:26 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","add","--update"] 
+[2012-07-29 11:49:26 IST] Merger: merging changes into Just refs/heads/master
+[2012-07-29 11:49:26 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","merge","--no-edit","refs/heads/synced/master"] 
+(started...) [2012-07-29 11:49:26 IST] Watcher: watching .
+[2012-07-29 11:49:26 IST] WebApp: running on port 60042
+Already up-to-date.
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./1
+[2012-07-29 11:49:26 IST] chat: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","cat-file","--batch"] 
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./2
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./3
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./4
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./5
+[2012-07-29 11:49:26 IST] Watcher: add symlink ./readme.txt
+[2012-07-29 11:49:27 IST] Committer: committing 6 changes
+(Recording state in git...)
+[2012-07-29 11:49:27 IST] feed: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","update-index","-z","--index-info"] 
+[2012-07-29 11:49:27 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","commit","--allow-empty-message","-m","","--allow-empty","--quiet"] 
+[2012-07-29 11:49:28 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"] 
+[2012-07-29 11:49:28 IST] Pusher: pushing to []
+[2012-07-29 11:49:28 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","branch","-f","synced/master"] 
+[2012-07-29 11:49:28 IST] read: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","symbolic-ref","HEAD"] 
+[2012-07-29 11:49:28 IST] Merger: merging changes into Just refs/heads/master
+[2012-07-29 11:49:28 IST] call: git ["--git-dir=/Users/jtang/sandbox/atest/.git","--work-tree=/Users/jtang/sandbox/atest","merge","--no-edit","refs/heads/synced/master"] 
+Already up-to-date.
+[2012-07-29 11:49:43 IST] Watcher: file added ./.gitignore
+[2012-07-29 11:49:43 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"] 
+[2012-07-29 11:49:44 IST] Committer: delaying commit of 0 changes
+[2012-07-29 11:49:45 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"] 
+[2012-07-29 11:49:46 IST] Committer: delaying commit of 0 changes
+[2012-07-29 11:49:47 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"] 
+[2012-07-29 11:49:48 IST] Committer: delaying commit of 0 changes
+[2012-07-29 11:49:49 IST] read: lsof ["-F0can","+d","/Users/jtang/sandbox/atest/.git/annex/tmp/"] 
+
+ +I ran " git annex watch -d --foreground" to watch what was going one, and just created a .gitignore file and the the commiter/lsof thread just loops over and over.... I only noticed as my laptop battery had drained at somepoint when git-annex was running in the background. diff --git a/doc/bugs/lsof__47__committer_thread_loops_occassionally/comment_1_f8d1720aa26c719609720acf0772606e._comment b/doc/bugs/lsof__47__committer_thread_loops_occassionally/comment_1_f8d1720aa26c719609720acf0772606e._comment new file mode 100644 index 0000000000..521d2e0cb6 --- /dev/null +++ b/doc/bugs/lsof__47__committer_thread_loops_occassionally/comment_1_f8d1720aa26c719609720acf0772606e._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.153.8.133" + subject="probably a kqueue specific problem..." + date="2012-07-29T18:10:12Z" + content=""" +I don't think this would eat your battery badly; it's looping for sure, but with a hardcoded 1 second delay I put in to guard against it eating all CPU. + +I tried to fix this in commit c4023f785834bc237e5fcdb69e275bbae10dd40b, but I sort of doubt I did. +I made one more commit that will at least tell us what file it is trying to check over and over with lsof. +"""]] diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn index f2711a4794..65569e4037 100644 --- a/doc/design/assistant/android.mdwn +++ b/doc/design/assistant/android.mdwn @@ -26,6 +26,10 @@ References: * * +I've heard anecdoally that ipwnstudios not only has an IPhone GHC port, +but also Android. Need to get in touch with them. + + ### Android specific features The app should be aware of power status, and avoid expensive background diff --git a/doc/design/assistant/blog/day_46__notification_pools.mdwn b/doc/design/assistant/blog/day_46__notification_pools.mdwn new file mode 100644 index 0000000000..483ad95b1c --- /dev/null +++ b/doc/design/assistant/blog/day_46__notification_pools.mdwn @@ -0,0 +1,68 @@ +Focus today was writing a notification broadcaster library. This is a way to +send a notification to a set of clients, any of which can be blocked +waiting for a new notification to arrive. A complication is that any number +of clients may be be dead, and we don't want stale notifications for those +clients to pile up and leak memory. + +It took me 3 tries to find the solution, which turns out to be head-smackingly +simple: An array of SampleVars, one per client. + +Using SampleVars means that clients only see the most recent notification, +but when the notification is just "the assistant's state changed somehow; +display a refreshed rendering of it", that's sufficient. + +---- + +First use of that was to make the thread that woke up every 10 minutes +and checkpointed the daemon status to disk also wait for a notification +that it changed. So that'll be more current, and use less IO. + +---- + +Second use, of course, was to make the WebApp block long polling clients +until there is really a change since the last time the client polled. + +To do that, I made one change to my Yesod routes: + +[[!format diff """ + -/status StatusR GET + +/status/#NotificationId StatusR GET +"""]] + +Now I find another reason to love Yesod, because after doing that, +I hit "make".. and fixed the type error. And hit make.. and fixed +the type error. And then it just freaking worked! Html was generated with +all urls to /status including a `NotificationId`, and the handler for +that route got it and was able to use it: + +[[!format haskell """ + {- Block until there is an updated status to display. -} + b <- liftIO $ getNotificationBroadcaster webapp + liftIO $ waitNotification $ notificationHandleFromId b nid +"""]] + +And now the WebApp is able to display transfers in realtime! +When I have both the WebApp and `git annex get` running on the same screen, +the WebApp displays files that git-annex is transferring about as fast +as the terminal updates. + +The [[progressbars]] still need to be sorted out, but otherwise +the WebApp is a nice live view of file transfers. + +--- + +I also had some fun with Software Transactional Memory. Now when the +assistant moves a transfer from its queue of transfers to do, to its map of +transfers that are currently running, it does so in an atomic transaction. +This will avoid the transfer seeming to go missing (or be listed twice) if +the webapp refreshes at just the wrong point in time. I'm really starting +to get into STM. + +---- + +Next up, I will be making the WebApp maintain a list of notices, displayed +on its sidebar, scrolling new notices into view, and removing ones the user +closes, and ones that expire. This will be used for displaying errors, as +well as other communication with the user (such as displaying a notice +while a git sync is in progress with a remote, etc). Seems worth doing now, +so the basic UI of the WebApp is complete with no placeholders. diff --git a/doc/design/assistant/blog/day_47__alert_messages.mdwn b/doc/design/assistant/blog/day_47__alert_messages.mdwn new file mode 100644 index 0000000000..81551fa95b --- /dev/null +++ b/doc/design/assistant/blog/day_47__alert_messages.mdwn @@ -0,0 +1,14 @@ +Some days I spend 2 hours chasing red herrings (like "perhaps my JSON ajax +calls arn't running asynchronoously?") that turn out to be a simple +one-word typo. This was one of them. + +However, I did get the sidebar displaying alert messages, which can be +easily sent to the user from any part of the assistant. This includes +transient alerts of things it's doing, which disappear once the action +finishes, and long-term alerts that are displayed until the user closes +them. It even supports rendering arbitrary Yesod widgets as alerts, so +they can also be used for asking questions, etc. + +Time for a screencast! + + diff --git a/doc/design/assistant/partial_content/comment_1_58c4faa321a5bb71adf9fdee079849f4._comment b/doc/design/assistant/partial_content/comment_1_58c4faa321a5bb71adf9fdee079849f4._comment new file mode 100644 index 0000000000..aa65cd6b9a --- /dev/null +++ b/doc/design/assistant/partial_content/comment_1_58c4faa321a5bb71adf9fdee079849f4._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkpEY8WTFDhjIVTWG38Ph7ppmuXUTJAHAg" + nickname="Justin" + subject="selective sync" + date="2012-07-28T04:08:00Z" + content=""" +hey joey + +great work!! + +will partial content work like selective sync in dropbox + +use case: on desktop i have photos/mp3s/docs, but would only want to sync the docs to my netbook + +cheers + +justin +"""]] diff --git a/doc/design/assistant/webapp.mdwn b/doc/design/assistant/webapp.mdwn index 018d70886b..ebf1689963 100644 --- a/doc/design/assistant/webapp.mdwn +++ b/doc/design/assistant/webapp.mdwn @@ -16,7 +16,7 @@ The webapp is a web server that displays a shiny interface. ## interface -* list of files uploading and downloading +* list of files uploading and downloading **done** * progress bars for each file * drag and drop to reorder * cancel and pause @@ -27,15 +27,14 @@ The webapp is a web server that displays a shiny interface. * there could be a UI to export a file, which would make it be served up over http by the web app * Display any relevant warning messages. One is the `inotify max_user_watches` - exceeded message. Need to lift such messages into DaemonStatus - so the WebApp can include them in its rendering of DaemonStatus. + exceeded message. ## implementation -* perhaps define a custom `errorHandler`, which could avoid the potential - of leaking auth tokens on error pages. Or make the test suite test for - leakage. * possibly lose the ugly auth= token past the first page, and use a client-side session. It could be encrypted using the token as the `encryptKey`. Note: Would need to set the session duration to infinite (how?) +* Fix notification handle leakage on pages other than the main page. + The javascript should use AJAX to request handles, that way + they won't be allocated at all in noscript. diff --git a/doc/sync.mdwn b/doc/sync.mdwn index 765c1e43fd..057dcb355e 100644 --- a/doc/sync.mdwn +++ b/doc/sync.mdwn @@ -1,7 +1,7 @@ The `git annex sync` command provides an easy way to keep several repositories in sync. -Often git is used in a centralized fashion with a central bare repositry +Often git is used in a centralized fashion with a central bare repository which changes are pulled and pushed to using normal git commands. That works fine, if you don't mind having a central repository.