Merge branch 'master' into assistant

Command/InitRemote.hs

@@ -38,18 +38,19 @@ start (name:ws) = do
 	t <- findType fullconfig
 
 	showStart "initremote" name
-	next $ perform t u $ M.union config c
+	next $ perform t u name $ M.union config c
 
 	where
 		config = Logs.Remote.keyValToConfig ws
 
-perform :: RemoteType -> UUID -> R.RemoteConfig -> CommandPerform
-perform t u c = do
+perform :: RemoteType -> UUID -> String -> R.RemoteConfig -> CommandPerform
+perform t u name c = do
 	c' <- R.setup t u c
-	next $ cleanup u c'
+	next $ cleanup u name c'
 
-cleanup :: UUID -> R.RemoteConfig -> CommandCleanup
-cleanup u c = do
+cleanup :: UUID -> String -> R.RemoteConfig -> CommandCleanup
+cleanup u name c = do
+	describeUUID u name
 	Logs.Remote.configSet u c
         return True
 
@@ -61,7 +62,6 @@ findByName name = do
 	where
 		generate = do
 			uuid <- liftIO genUUID
-			describeUUID uuid name
 			return (uuid, M.insert nameKey name M.empty)
 
 findByName' :: String ->  M.Map UUID R.RemoteConfig -> Maybe (UUID, R.RemoteConfig)

debian/changelog

@@ -1,3 +1,10 @@
+git-annex (3.20120722) UNRELEASED; urgency=low
+
+  * initremote: Avoid recording remote's description before checking
+    that its config is valid.
+
+ -- Joey Hess <joeyh@debian.org>  Fri, 27 Jul 2012 21:04:47 -0400
+
 git-annex (3.20120721) unstable; urgency=low
 
   * get, move, copy: Now refuse to do anything when the requested file

doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_5_5e27737a5bb0e9e46c98708700318e67._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
+ nickname="Jimmy"
+ subject="comment 5"
+ date="2012-07-27T12:20:07Z"
+ content="""
+After some debugging, I looked at the Utility/libkqueue.c and used it as a test, it seems to be hanging/segfaulting around the call to that library. Annoyingly I get segfaults from the library every so often on OSX, it's pretty a random event.
+"""]]

doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_6_1f92da712232d050e085a4f39063d7a6._comment

@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.20"
+ subject="comment 6"
+ date="2012-07-27T16:34:45Z"
+ content="""
+Are you seeing libkqueue crash when it's called from a debugger or C program, rather than from Haskell?
+
+Are you building for 32 or 64 bit? You might try getting the 32 bit version of GCC (or The Haskell Platform) and see if it does better. There is a known GCC crashes on 64 bit OSX involving C libraries,
+although this bug report doesn't seem to apply, since we're not using ghci <http://hackage.haskell.org/trac/ghc/ticket/7040>.
+
+Are you building with cabal, or using the Makefile?
+
+You might try reverting git commit da4c506d61115236f3e43dd0bd17f30cd54df950
+
+You might try disabling the -threaded option in the cabal file or Makefile.
+
+I ssh'd to the OSX box I have an account on, and confirmed that git-annex watch still works there as of the current head of the `assistant` branch. That's a 64 bit GHC system, FWIW.
+
+Do you see the crash when building from the `master` branch, or only `assistant`? Master has the watch command, but it's much out of date, so this will tell if the problem was introduced recently... and you might still have to bisect it since I can't reproduce it. :(
+"""]]

doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_7_4153dc8029c545f8e86584a38bd536fb._comment

@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
+ nickname="Jimmy"
+ subject="comment 7"
+ date="2012-07-27T17:23:24Z"
+ content="""
+I'm using make and a 64bit version of haskell-platform, it's fine on the master branch. It's just crashing on the assistant branch, I'm just thinking out loud, but could I share the binaries that I have with you (I'd like to grab your binaries too) to see see if its just some silly problem with my build environment.
+
+I'm seeing the crash when I'm running git-annex (in haskell), when I run libkqueue in a debugger it behaves randomly, is mostly succeeds, but every so often it fails. A back trace reveals nothing so I am a bit at a loss.
+
+I've tried disabling the threaded option, but it still crashes, I will give it another try later when I get home. The problem seems to occur on my desktop mac in work and my home mac, however it is fine on my linux machines.
+
+Could I ask which version of OSX do you have access to? is it 10.6 or 10.7 ?
+
+"""]]

doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_8_f85b6eb5bfd28ffc6973fb4ab0fe4337._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.20"
+ subject="comment 8"
+ date="2012-07-27T18:10:06Z"
+ content="""
+I've reproduced a crash on OSX, involving not kqueue, but the WebApp's use of getaddrinfo. I've fixed that, but several things you've said in this bug report don't 100% add up to this being the same crash you've been seeing (for one thing, this can't affect `git annex watch`), so I'll wait for you to confirm.
+"""]]

doc/bugs/Watch_command_as_of_commit_6cecc26206c4a539999b04664136c6f785211a41_segfaults/comment_9_c747c488461c98cd285b51d3afc2c3eb._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="jtang"
+ ip="79.97.135.214"
+ subject="comment 9"
+ date="2012-07-27T18:17:45Z"
+ content="""
+Ah, it's alive, I'm testing on my home machine right now and it's functioning as expected. I've tested on my work machine as well and the watch command works as expected! (short of the existing system limits which fails for my bigger annexes).
+
+Apologies for sending you on the wrong path with the kqueue path. I reckon if I give this a good testing over the weekend is a good idea before closing this bug, I will report back in a few days on this after giving a thrashing of files!
+"""]]

doc/bugs/git_annex_initremote_needs_some___34__error_checking__34__.mdwn

@@ -60,3 +60,6 @@ bloom filter size: 16 mebibytes (0% full)
 backend usage: 
 x00:atest jtang$ 
 </pre>
+
+> Indeed, I broke that in June by making it record the name in a much too
+> early stage. Now fixed. [[done]] --[[Joey]]

doc/bugs/yesod-default_is_needed_as_a_dependancy.mdwn

@@ -0,0 +1,10 @@
+Seems like there is a needed dependancy, yesod-default is needed on OSX.
+
+<pre>
+Utility/Yesod.hs:10:8:
+    Could not find module `Yesod.Default.Util'
+    Use -v to see a list of the files searched for.
+make: *** [git-annex] Error 1
+</pre>
+
+> Only on OSX apparently. Weird. Added. [[done]] --[[Joey]] 

doc/design/assistant/blog/day_44__webapp_basics/comment_1_d5fb67f373038e9f583cb2e1992bef67._comment

@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="http://jasonwoof.com/"
+ nickname="JasonWoof"
+ subject="Your security solution is fine"
+ date="2012-07-27T06:37:03Z"
+ content="""
+I think making the html file redirect is just fine. desktopcouch does something like this. They talk about it being there to help you find the port number, but it contains the security token too.
+
+They also have some javascript to delay the redirect for a while so you can bookmark the redirect file. You can click to set a cookie to skip the delay in future.
+
+It's been a while since I used desktopcouch, but I found an old redirect file laying around. Here it is:
+
+http://jasonwoof.com/downloads/desktopcouch-redirect.txt
+
+The code is pretty short, you might find something useful there.
+
+      - Jason
+"""]]

doc/design/assistant/blog/day_45__long_polling.mdwn

@@ -0,0 +1,66 @@
+The webapp now displays actual progress bars, for the actual transfers
+that the assistant is making! And it's seriously shiny.
+
+[[!img full.png]]
+
+Yes, I used Bootstrap. I can see why so many people are using it,
+that the common complaint is everything looks the same. I spent a few hours
+mocking up the transfer display part of the WebApp using Bootstrap, and
+arrived at something that doesn't entirely suck remarkably quickly.
+
+The really sweet thing about Bootstrap is that when I resized my browser to
+the shape of a cell phone, it magically redrew the WebApp like so:
+
+[[!img phone.png]]
+
+---
+
+To update the display, the WebApp uses two techniques. On noscript
+browsers, it just uses a meta refresh, which is about the best I can do. I
+welcome feedback; it might be better to just have an "Update" button in
+this case.
+
+With javascript enabled, it uses long polling, done over AJAX. There are
+some other options I considered, including websockets, and server-sent
+events. Websockets seem too new, and while there's a WAI module supporting
+server-sent events, and even an example of them in the Yesod book, the
+module is not packaged for Debian yet. Anyway, long polling is the most
+widely supported, so a good starting place. It seems to work fine too, I
+don't really anticipate needing the more sophisticated methods.
+
+(Incidentially, this's the first time I've ever written code that uses AJAX.)
+
+Currently the status display is rendered in html by the web server, and
+just updated into place by javascript. I like this approach since it
+keeps the javascript code to a minimum and the pure haskell code to a
+maximum. But who knows, I may have to switch to JSON that gets rendered by
+javascript, for some reason, later on.
+
+---
+
+I was very happy with Yesod when I managed to factor out a
+general purpose widget that adds long-polling and meta-refresh to any
+other widget. I was less happy with Yesod when I tried to include
+jquery on my static site and it kept serving up a truncated version of it.
+Eventually worked around what's seemingly a bug in the default WAI
+middleware, by disabling that middleware.
+
+----
+
+Also yesterday I realized there were about 30 comments stuck in moderation on
+this website. I thought I had a feed of those, but obviously I didn't. I've
+posted them all, and also read them all.
+
+----
+
+Next up is probably some cleanup of bugs and minor todos. Including
+figuring out why `watch` has started to segfault on OSX when it was
+working fine before.
+
+After that, I need to build a way to block the long polling request
+until the DaemonStatus and/or TransferQueue change from the version
+previously displayed by the WebApp. An interesting concurrency problem..
+
+Once I have that working, I can reduce the current 3 second delay between
+refreshes to a very short delay, and the WebApp will update in
+near-realtime as changes come in.

doc/design/assistant/blog/day_45__long_polling/comment_1_994bec0978324e268666073e8ff4f6ae._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://jasonwoof.com/"
+ nickname="JasonWoof"
+ subject="auth token length"
+ date="2012-07-27T18:52:19Z"
+ content="""
+Your auth token looks a little short. Aren't you worried about people brute-forcing it? ;)
+"""]]

doc/design/assistant/blog/day_45__long_polling/comment_2_dfa164c86290899139491acccddd8b2b._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.20"
+ subject="comment 2"
+ date="2012-07-27T18:55:51Z"
+ content="""
+Heh, I consider it overflowing most address fields a bonus, as you don't have to worry when making screenshots. :)
+
+Of course, it changes each app run too..
+"""]]

doc/forum/Looking_at_the_webapp_on_OSX/comment_5_c735841bc230efc61594ea013fc2902b._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
+ nickname="Jimmy"
+ subject="comment 5"
+ date="2012-07-27T15:53:50Z"
+ content="""
+I can confirm that the watch command is crashing when i start it up manually.
+"""]]

doc/forum/Looking_at_the_webapp_on_OSX/comment_6_0e489fbfc89d282e9eb47f1b814ff70c._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="jtang"
+ ip="79.97.135.214"
+ subject="comment 6"
+ date="2012-07-27T18:21:17Z"
+ content="""
+FYI, the webapp starts up and sends me to a web browser with the correct page ;) this is after the other issue of [[Watch command as of commit 6cecc26206c4a539999b04664136c6f785211a41 segfaults]] got fixed.
+"""]]