Merge branch 'master' into assistant

Command/Sync.hs

@@ -16,6 +16,7 @@ import qualified Remote
 import qualified Annex
 import qualified Annex.Branch
 import qualified Git.Command
+import qualified Git.Merge
 import qualified Git.Branch
 import qualified Git.Ref
 import qualified Git
@@ -170,7 +171,7 @@ mergeAnnex = do
 mergeFrom :: Git.Ref -> CommandCleanup
 mergeFrom branch = do
 	showOutput
-	inRepo $ Git.Command.runBool "merge" [Param $ show branch]
+	inRepo $ Git.Merge.mergeNonInteractive branch
 
 changed :: Remote -> Git.Ref -> Annex Bool
 changed remote b = do

Git/Merge.hs

@@ -0,0 +1,17 @@
+{- git merging
+ -
+ - Copyright 2012 Joey Hess <joey@kitenet.net>
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Git.Merge where
+
+import Common
+import Git
+import Git.Command
+
+{- Avoids recent git's interactive merge. -}
+mergeNonInteractive :: Ref -> Repo -> IO Bool
+mergeNonInteractive branch = runBool "merge"
+	[Param "--no-edit", Param $ show branch]

debian/changelog

@@ -6,6 +6,7 @@ git-annex (3.20120616) UNRELEASED; urgency=low
     Available on Linux, BSDs, and OSX!
   * Enable diskfree on kfreebsd, using kqueue.
   * unused: Fix crash when key names contain invalid utf8.
+  * sync: Avoid recent git's interactive merge.
 
  -- Joey Hess <joeyh@debian.org>  Tue, 12 Jun 2012 11:35:59 -0400
 

doc/bugs/Possible_issues_with_git_1.7.10_and_newer___40__merge_command_now_asks_for_a_commit_message__34__.mdwn

@@ -0,0 +1,18 @@
+running 'git annex sync' doesn't merge the branches as expected (from the
+limited testing I have done) with git 1.7.10, the behaviour of merge has
+changed, it now asks for a commit message. I would expect setting
+_GIT_MERGE_AUTOEDIT=no_ should resolve this issue.
+
+I had to manually do a merge (or set that variable) to get the branches
+back in sync again, this confused me a bit when git-annex watch was running
+in the background on a remote and it did not pick up the changes.
+
+> Yeah, I tend to miss these since the first thing I did when this
+> misfeature was being posted was to write a mail discouraging them from
+> doing it (sadly ignored), and then set in ~/.environment:
+
+	# My time is more valuable than git's new, bad default
+	GIT_MERGE_AUTOEDIT=no
+	export GIT_MERGE_AUTOEDIT
+
+> Anyway, I've made sync run merge with --no-edit now. [[done]] --[[Joey]]

doc/design/assistant/blog/day_15__its_aliiive.mdwn

@@ -0,0 +1,33 @@
+Syncing works! I have two clones, and any file I create in the first
+is immediately visible in the second. Delete that file from the second, and
+it's immediately removed from the first.
+
+Most of my work today felt like stitching existing limbs onto a pre-existing
+monster. Took the committer thread, that waits for changes and commits them,
+and refashioned it into a pusher thread, that waits for commits and pushes
+them. Took the watcher thread, that watches for files being made,
+and refashioned it into a merger thread, that watches for git refs being
+updated. Pulled in bits of the `git annex sync` command to reanimate this.
+
+It may be a shambling hulk, but it works.
+
+Actually, it's not much of a shambling hulk; I refactored my code after
+copying it. ;)
+
+I think I'm up to 11 threads now in the new
+`git annex assistant` command, each with its own job, and each needing
+to avoid stepping on the other's toes. I did see one MVar deadlock error
+today, which I have not managed to reproduce after some changes. I think
+the committer thread was triggering the merger thread, which probably
+then waited on the Annex state MVar the committer thread had held.
+
+Anyway, it even pushes to remotes in parallel, and keeps track of remotes
+it failed to push to, although as of yet it doesn't do any attempt at
+periodically retrying.
+
+One bug I need to deal with is that the push code assumes any change
+made to the remote has already been pushed back to it. When it hasn't, 
+the push will fail due to not being a fast-forward. I need to make it
+detect this case and pull before pushing.
+
+(I've pushed this work out in a new `assistant branch`.)

doc/design/assistant/syncing.mdwn

@@ -5,13 +5,16 @@ all the other git clones, at both the git level and the key/value level.
 
 1. Can use `git annex sync`, which already handles bidirectional syncing.
    When a change is committed, launch the part of `git annex sync` that pushes
-   out changes.
+   out changes. **done**; changes are pushed out to all remotes in parallel
 1. Watch `.git/refs/remotes/` for changes (which would be pushed in from
    another node via `git annex sync`), and run the part of `git annex sync`
    that merges in received changes, and follow it by the part that pushes out
    changes (sending them to any other remotes).
    [The watching can be done with the existing inotify code! This avoids needing
-   any special mechanism to notify a remote that it's been synced to.]
+   any special mechanism to notify a remote that it's been synced to.]  
+   **done**
+1. Periodically retry pushes that failed. Also, detect if a push failed
+   due to not being up-to-date, pull, and repush.
 2. Use a git merge driver that adds both conflicting files,
    so conflicts never break a sync.
 3. Investigate the XMPP approach like dvcs-autosync does, or other ways of

doc/forum/Wishlist:_logging_to_file_when_running_as_a_daemon___40__for_the_assistant__41__.mdwn

@@ -0,0 +1 @@
+Logging to file would be nice when running git-annex as a daemon so when something fails or when certain events happens one can look at the logs to see what has failed, unless I'm missing something this probably should be on the wishlist or roadmap.

doc/special_remotes/directory/comment_1_e8a53592adb13f7d7f212a2eb5a18a31._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlc1og3PqIGudOMkFNrCCNg66vB7s-jLpc"
+ nickname="Paul"
+ subject="how is this different than rsync?"
+ date="2012-06-22T22:10:19Z"
+ content="""
+Thanks for this great tool! I was wondering what the differences are between using `type=directory`, `type=rsync`, or a bare git repo for directories? 
+
+I guess I can't use just a regular repo because my USB drive is formatted as `vfat` -- which threw me for a loop the first time I heard about `git-annex` about a year ago, because I followed the walkthrough, and it didn't work as expected and gave up (now I know it was just a case of PEBKAC). It might be worth adding a note about [vfat](http://git-annex.branchable.com/bugs/fat_support/) to the \"Adding a remote\" section of the [walkthrough](http://git-annex.branchable.com/walkthrough/), since the unstated assumption there is that the USB drive is formatted as a filesystem that supports symlinks.
+
+Thanks again, my scientific data management just got a lot more sane!
+"""]]