Merge branch 'master' into assistant

doc/bugs/Build_failure_at_commit_1efe4f3.mdwn

@@ -0,0 +1,45 @@
+Applying this
+
+<pre>
+laplace:git-annex jtang$ git diff
+diff --git a/Assistant/WebApp/Configurators.hs b/Assistant/WebApp/Configurators.hs
+index b9630b1..bf36e59 100644
+--- a/Assistant/WebApp/Configurators.hs
++++ b/Assistant/WebApp/Configurators.hs
+@@ -101,7 +101,7 @@ checkRepositoryPath p = do
+  -
+  - If run in another directory, the user probably wants to put it there. -}
+ defaultRepositoryPath :: Bool -> IO FilePath
+-defaultRepositoryPath firstrun = do
++defaultRepositoryPath firstRun = do
+        cwd <- liftIO $ getCurrentDirectory
+        home <- myHomeDir
+        if home == cwd && firstRun
+</pre>
+
+Causes this to occur,
+
+<pre>
+Assistant/WebApp/Configurators.hs:114:17:
+    Couldn't match expected type `Control.Monad.Trans.RWS.Lazy.RWST
+                                    (Maybe (Env, FileEnv), WebApp, [Yesod.Form.Types.Lang])
+                                    Enctype
+                                    Ints
+                                    (GHandler WebApp WebApp)
+                                    t0'
+                with actual type `Text'
+    Expected type: String
+                   -> Control.Monad.Trans.RWS.Lazy.RWST
+                        (Maybe (Env, FileEnv), WebApp, [Yesod.Form.Types.Lang])
+                        Enctype
+                        Ints
+                        (GHandler WebApp WebApp)
+                        t0
+      Actual type: String -> Text
+    In the first argument of `(.)', namely `T.pack'
+    In the first argument of `(<$>)', namely
+      `T.pack . addTrailingPathSeparator'
+make: *** [git-annex] Error 1
+</pre>
+
+> [[fixed|done]] --[[Joey]] 

doc/bugs/journal_commit_error_when_using_annex.mdwn

@@ -0,0 +1,21 @@
+Sorry for the noise - notabug - [[done]]
+
+Setup:
+
+* git-annex version: 3.20120629 on Debian unstable.
+* external RAID1 array connected via usb
+
+I did: <code> git-annex add \<a specific album of ogg files\> </code>
+
+When it gets to the fourth song, there is a long delay and a bunch of "journal commit I/O error" from the kernel start appearing. Then GNOME helpfully tells me that my drive was just connected (implying that it thought it was disconnected momentarily)
+
+Log outputs:
+
+Both trying to add whole album, failing at song 4
+
+* <http://paste.mitechie.com/show/745/>
+* <http://paste.mitechie.com/show/746/>
+
+Just trying to add song 4 individually
+
+* <http://paste.mitechie.com/show/747/> (lots o' journal commit I/O errors that time)

doc/bugs/journal_commit_error_when_using_annex/comment_1_38f60ca3503ea1530c4bd2cde5c9182f._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
+ nickname="Justin"
+ subject="comment 1"
+ date="2012-08-01T23:06:03Z"
+ content="""
+That's not a git-annex error, your drive is dying.
+
+I'm sure if you tried just playing the song you'll get the same errors.
+"""]]

doc/bugs/journal_commit_error_when_using_annex/comment_2_6de455a67f37d9ee0a307a78123781bf._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://creativecommons.net/greg/identity"
+ ip="64.79.125.70"
+ subject="comment 2"
+ date="2012-08-01T23:22:03Z"
+ content="""
+Probably right, the song in question is not playing fully and causing same error.
+
+Time to go into crises mode...
+"""]]

doc/design/assistant/blog/day_40__dbus/comment_5_40b6b9d741d3081203f0cc94eb8dc3ea._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://smcv.pseudorandom.co.uk/"
+ nickname="smcv"
+ subject="comment 5"
+ date="2012-07-31T09:58:13Z"
+ content="""
+It's `org.gtk` because gvfs, GLib and Gtk are all products of gtk.org (there is no separate glib.org).
+
+It's `Private` because the gvfs developers don't consider its D-Bus interface to be stable API, I believe. The official API for it is the C API in GIO, part of GLib.
+
+To poll for changes to /proc/mounts on Linux, you open it for reading and poll() for POLLERR (not sure why it gets to be different, but it is). See gio/gunixmounts.c in recent GLib.
+"""]]

doc/design/assistant/blog/day_49__first_run_experience.mdwn

@@ -0,0 +1,37 @@
+Started work on the interface displayed when the webapp is started 
+with no existing git-annex repository. All this needs to do is walk the user
+through setting up a repository, as simply as possible.
+
+A tricky part of this is that most of git-annex runs in the Annex monad,
+which requires a git-annex repository. Luckily, much of the webapp
+does not run in Annex, and it was pretty easy to work around the parts that
+do. Dodged a bullet there.
+
+There will, however, be a tricky transition from this first run webapp,
+to a normally fully running git-annex assistant and webapp. I think the
+first webapp will have to start up all the normal threads once it makes the
+repository, and then redirect the user's web browser to the full webapp.
+
+Anyway, the UI I've made is very simple: A single prompt, for the
+directory where the repository should go. With, eventually, tab completion,
+sanity checking (putting the repository in "/" is not good, and making it
+all of "$HOME" is probably unwise).
+
+[[!img screenshot/firstrun.png]]
+
+Ideally most users will accept the default, which will be something
+like `/home/username/Desktop/Annex`, and be through this step in seconds.
+
+Suggestions for a good default directory name appreciated.. Putting it on a
+folder that will appear on the desktop seems like a good idea, when there's
+a Desktop directory. I'm unsure if I should name it something specific like
+"GitAnnex", or something generic like "Synced".
+
+Time for the first of probably many polls!
+
+What should the default directory name used by the git-annex assistant be?
+
+[[!poll open=no 19 "Annex" 7 "GitAnnex" 10 "Synced" 0 "AutoSynced" 1 "Shared" 10 "something lowercase!" 1 "CowboyNeal" 1 "Annexbox"]]
+
+(Note: This is a wiki. You can edit this page to add your own
+[[ikiwiki/directive/poll]] options!)

doc/design/assistant/blog/day_49__first_run_experience/comment_1_e146cf06c8dd6303dd6a991f152a73fe._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://nico.kaiser.me/"
+ nickname="Nico Kaiser"
+ subject="Directory location"
+ date="2012-08-01T07:51:36Z"
+ content="""
+I don't think the Annex directory (or whatever it will be called – \"Annex\" seems like a good choice in my opinion) should be inside the \"Desktop\" folder by default. All other xdg-user-dirs (Documents, Downloads, etc.) are in $HOME, and i think Desktop should not be cluttered with folders.
+"""]]

doc/design/assistant/blog/day_49__first_run_experience/comment_2_5d6adcf6782c02283bef6189582ee467._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.154.3.236"
+ subject="comment 2"
+ date="2012-08-01T14:04:08Z"
+ content="""
+Hmm, my thought on putting it under Desktop is that'll let me install git-annex and tell my mom (really!) that files dragged to there are shared. Don't the other xdg directories get used as default save/open locations for things like word processors, so not need to be on the desktop?
+
+Although that may not be necessary. I've realized I can, at least on linux, easily make a button on the webapp that pops open the directory in the desktop's file manager.
+
+I guess another way of looking at the question is: Where does dropbox put its folder?
+"""]]

doc/design/assistant/blog/day_49__first_run_experience/comment_3_7ac2e34c2a7bc9b57488ca0c91307d32._comment

@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawllHWUwOT-UVe5bGkk8G87bpeRAzy-5T7w"
+ nickname="Rick"
+ subject="Folder location"
+ date="2012-08-01T14:18:21Z"
+ content="""
+I'm of two minds about where the Annex folder should be.
+
+For beginners, the desktop is ideal. No doubt about it. And more experienced users can override the default path during the install.
+
+On the other hand, many of the other 'synced folder' services place their folder in the user's home. Both Dropbox and Skydrive do this. It's where I'd put it for consistency, and to keep my desktop free of clutter.
+
+It's a close call, but if push came to shove, I'd probably go with the user home folder.
+"""]]

doc/design/assistant/blog/day_50__directory_name.mdwn

@@ -0,0 +1,20 @@
+Based on the results of yesterday's poll, the WebApp defaults to
+`~/Desktop/annex` when run in the home directory. If there's no `Desktop`
+directory, it uses just `~/annex`. And if run from some other place than
+the home directory, it assumes you want to use cwd. Of course, you can
+change this default, but I think it's a good one for most use cases.
+
+----
+
+My work today has all been on making **one second** of the total lifetime
+of the WebApp work. It's the very tricky second in between clicking on
+"Make repository" and being redirected to a WebApp running in your new
+repository. The trickiness involves threads, and MVars, and
+multiple web servers, and I don't want to go into details here.
+I'd rather forget. ;-)
+
+Anyway, it works; you can run "git annex webapp" and be walked right
+through to having a usable repository! Now I need to see about adding
+that to the desktop menus, and making "git annex webapp", when run a second
+time, remembering where your repository is. I'll use
+`~/.config/git-annex/repository` for storing that.

doc/design/assistant/blog/day_50__directory_name/comment_1_782cec95a8558a05b2b38a2d2302214d._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
+ nickname="Richard"
+ subject="comment 1"
+ date="2012-08-01T21:19:03Z"
+ content="""
+Please use `$XDG_CONFIG_HOME` and fall back to `~/.config` if that's not defined.
+"""]]

doc/design/assistant/blog/day_50__directory_name/comment_2_2b8ceb0a26f25e8ed2711bcbe7225a58._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://piotr.ozarowski.pl/"
+ nickname="POX"
+ subject="XDG"
+ date="2012-08-02T10:30:55Z"
+ content="""
+same for XDG_DESKTOP_DIR (for pl_PL ~/.config/user-dirs.dirs file contains XDG_DESKTOP_DIR=\"$HOME/Pulpit\" instead of Desktop)
+"""]]

doc/design/assistant/blog/day_51__desktop.mdwn

@@ -0,0 +1,34 @@
+Now installing git-annex automatically generates a freedesktop.org .desktop
+file, and installs it, either system-wide (root) or locally (user). So
+`Menu -> Internet -> Git Annex` will start up the web app.
+
+(I don't entirely like putting it on the Internet menu, but the
+Accessories menu is not any better (and much more crowded here), 
+and there's really no menu where it entirely fits.)
+
+I generated that file by writing a generic library to deal with
+freedesktop.org desktop files and locations. Which seemed like overkill at
+the time, but then I found myself continuing to use that library. Funny how
+that happens.
+
+So, there's also another .desktop file that's used to autostart the
+`git-annex assistant` daemon when the user logs into the desktop.
+
+This even works when git-annex is installed to the ugly non-PATH location
+`.cabal/bin/git-annex` by Cabal! To make that work, it records the path
+the binary is at to a freedesktop.org data file, at install time.
+
+---
+
+That should all work in Gnome, KDE, XFCE, etc. Not Mac OSX I'm guessing...
+
+---
+
+Also today, I added a sidebar notification when the assistant notices new
+files. To make that work well, I implemented merging of related sidebar
+action notifications, so the effect is that there's one notification that
+collectes a list of recently added files, and transient notifications that
+show up if a really big file is taking a while to checksum.
+
+I'm pleased that the notification interface is at a point where I was able
+to  implement all that, entirely in pure functional code.

doc/design/assistant/webapp.mdwn

@@ -20,6 +20,7 @@ The webapp is a web server that displays a shiny interface.
 * progress bars for each file
 * drag and drop to reorder
 * cancel and pause
+* button to open file browser on repo (`xdg-open $DIR`)
 * keep it usable w/o javascript, and accessible to blind, etc
 
 ## other features
@@ -29,14 +30,17 @@ The webapp is a web server that displays a shiny interface.
 * Display any relevant warning messages. One is the `inotify max_user_watches`
   exceeded message.
 
+## first start
+
+* make git repo **done**
+* generate a nice description like "joey@hostname Desktop/annex"
+* record repository that was made, and use it next time run
+* write a pid file, to prevent more than one first-start process running
+  at once
+
 ## implementation
 
 * possibly lose the ugly auth= token past the first page,
   and use a client-side session. It could be encrypted using the token
   as the `encryptKey`. Note: Would need to set the session duration
   to infinite (how?)
-* Fix notification handle leakage on pages other than the main page.
-  The javascript should use AJAX to request handles, that way
-  they won't be allocated at all in noscript.
-* Hook up notificaton close button to a callback that removes the notification
-  from the list. Otherwise reloading brings them back.

doc/forum/DS__95__Store_files_are_not_added.mdwn

@@ -0,0 +1,3 @@
+The "git annex add" command adds new file to the annex. However ".DS_Store" files are ignored by git-annex. Is there a list of files that are being ignored? 
+
+Maybe sometimes it's useful to add .DS_Store extended attribute data to the annex to ensure a complete sync of Mac files...

doc/forum/DS__95__Store_files_are_not_added/comment_1_30687306da9bd35ec02a806193c5e240._comment

@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 1"
+ date="2012-08-02T23:29:38Z"
+ content="""
+git-annex ignores whatever git does. See `git-ls-files` and the gitignore configuration.
+"""]]

doc/forum/Delete_unused_files__47__metadata.mdwn

@@ -0,0 +1,7 @@
+After moving some files (about 1G, some big and some small files) in and out the annex, I noticed that the size of the repository has grown quite a bit. My empty repository now is over 100 MB (even after  "git annex dropunused ..." and "git gc").
+
+Most of this size is not Git metadata but many small files in the git-annex branch that seem to hold information about files I deleted (even in the other known repositories).
+
+So is there a way to get rid of these useless but space consuming information?
+
+(Maybe there is no (elegant) way to remove the symlink versions from the Git history (is there?), but it would already be nice if those small git-annex metadata files could be removed)

doc/forum/Git_repositories_in_the_annex__63__.mdwn

@@ -0,0 +1,5 @@
+First of all thanks for the great work. git-annex looks very nice already!
+
+I wonder if it's possible to add Git repositories to the annex. In my current Dropbox setup I drop some small Git repositories into the Dropbox to keep them in sync on two computers. 
+
+This is obviously not possible with git-annex, as the annex itself is a Git repository and sub-repositories (directories with .git folders) are ignored. But in some cases people might want to be able to drop "anything" into the annex, maybe even Git repositories (e.g. the "nomad" usecase)...

doc/forum/Wishlist:_Don__39__t_make_files_readonly.mdwn

@@ -0,0 +1,3 @@
+Maybe this explained somewhere else, but what is the purpose of marking files readonly. To me this is an annoyance that doesn't make sense. I want to be able to change my files, having to go through an unlock step, to do that seems like unnecessary work. Interestingly, Microsoft's Team Foundation Server source control does the same thing and I don't like it there either.
+
+In addition why replace files with symlinks? Why not just leave the real files in place, or do the reverse and put the symlink to the file in the repository.

doc/forum/git_unannex_speed.mdwn

@@ -0,0 +1 @@
+It was fast to git annex a bunch of files. But git unannex seems a lot slower. Is there a faster way to get files out of git annex? Or to replace the symlinks with real files and then I could just remove the .git directory? I shouldn't have put so many in as a test but they are there now.

doc/forum/git_unannex_speed/comment_1_10cf326248f4e89e1f75bf97d7574763._comment

@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 1"
+ date="2012-08-02T16:58:41Z"
+ content="""
+It currently commits once per file, which is slow. `Command/Unannex.hs` explains why this is necessary. If I think of a way to avoid that, I will.
+"""]]

doc/install/Debian/comment_3_4d922e11249627634ecc35bba4044d9e._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkstq9oH1vHXY_VP0nYO9Gg3eKnKerDGRI"
+ nickname="Hadi"
+ subject="ARM"
+ date="2012-07-31T15:13:06Z"
+ content="""
+is there any package for Debian armhf? I'd love to install git-annex on my raspberry pi
+"""]]

doc/install/Debian/comment_4_2a93ab18b05ccb90e7acc5885866fca2._comment

@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 4"
+ date="2012-07-31T15:41:43Z"
+ content="""
+Yes, git-annex is available for every Debian architecture which supports Haskell, including all arm ports:
+
+<pre>git-annex | 3.20120629         | wheezy            | source, amd64, armel, armhf, i386, kfreebsd-amd64, kfreebsd-i386, mips, mipsel, powerpc, s390, s390x, sparc</pre>
+"""]]

doc/todo/windows_support/comment_3_bd0a12f4c9b884ab8a06082842381a01._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://xolus.net/openid/max"
+ nickname="B0FH"
+ subject="What about NTFS support ?"
+ date="2012-08-02T17:45:10Z"
+ content="""
+Has git-annex been tested with an NTFS-formatted disk under Linux ? NTFS is supposed to be case-sensitive and to allow symlinks, and these are supposed to work with ntfs3g.
+"""]]