Merge branch 'master' into xmpp

Logs/Transfer.hs

@@ -96,6 +96,9 @@ download u key file shouldretry a = runTransfer (Transfer Download u key) file s
  -
  - If the transfer action returns False, the transfer info is 
  - left in the failedTransferDir.
+ -
+ - An upload can be run from a read-only filesystem, and in this case
+ - no transfer information or lock file is used.
  -}
 runTransfer :: Transfer -> Maybe FilePath -> RetryDecider -> (MeterUpdate -> Annex Bool) -> Annex Bool
 runTransfer t file shouldretry a = do
@@ -107,7 +110,7 @@ runTransfer t file shouldretry a = do
 	unless ok $ failed info
 	return ok
 	where
-		prep tfile mode info = do
+		prep tfile mode info = catchMaybeIO $ do
 			fd <- openFd (transferLockFile tfile) ReadWrite (Just mode)
 				defaultFileFlags { trunc = True }
 			locked <- catchMaybeIO $
@@ -116,7 +119,8 @@ runTransfer t file shouldretry a = do
 				error $ "transfer already in progress"
 			writeTransferInfoFile info tfile
 			return fd
-		cleanup tfile fd = do
+		cleanup _ Nothing = noop
+		cleanup tfile (Just fd) = do
 			void $ tryIO $ removeFile tfile
 			void $ tryIO $ removeFile $ transferLockFile tfile
 			closeFd fd
@@ -149,7 +153,7 @@ runTransfer t file shouldretry a = do
 mkProgressUpdater :: Transfer -> TransferInfo -> Annex (MeterUpdate, FilePath, MVar Integer)
 mkProgressUpdater t info = do
 	tfile <- fromRepo $ transferFile t
-	createAnnexDirectory $ takeDirectory tfile
+	_ <- tryAnnex $ createAnnexDirectory $ takeDirectory tfile
 	mvar <- liftIO $ newMVar 0
 	return (liftIO . updater tfile mvar, tfile, mvar)
 	where
@@ -157,7 +161,7 @@ mkProgressUpdater t info = do
 			if (bytes - oldbytes >= mindelta)
 				then do
 					let info' = info { bytesComplete = Just bytes }
-					writeTransferInfoFile info' tfile
+					_ <- tryIO $ writeTransferInfoFile info' tfile
 					return bytes
 				else return oldbytes
 		{- The minimum change in bytesComplete that is worth

Remote/Bup.hs

@@ -111,7 +111,7 @@ bupSplitParams r buprepo k src = do
 	let os = map Param $ words o
 	showOutput -- make way for bup output
 	return $ bupParams "split" buprepo 
-		(os ++ [Param "-n", Param (bupRef k), src])
+		(os ++ [Param "-n", Param (bupRef k)] ++ src)
 
 store :: Git.Repo -> BupRepo -> Key -> AssociatedFile -> MeterUpdate -> Annex Bool
 store r buprepo k _f _p = do

debian/changelog

@@ -19,6 +19,8 @@ git-annex (3.20121018) UNRELEASED; urgency=low
   * webapp: Switched to using the same multicast IP address that avahi uses.
   * bup: Don't pass - to bup-split to make it read stdin; bup 0.25
     does not accept that.
+  * bugfix: Don't fail transferring content from read-only repos.
+    Closes: #691341
 
  -- Joey Hess <joeyh@debian.org>  Wed, 17 Oct 2012 14:24:10 -0400
 

doc/bugs/tmp_file_handling/comment_1_0300c11ee3f94a9e7c832671e16f5511._comment

@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="http://www.openid.albertlash.com/openid/"
+ ip="74.96.185.87"
+ subject="What to do?"
+ date="2012-10-24T15:47:16Z"
+ content="""
+I've got a stale .git/annex/tmp/rsynctmp file that just won't go away, and I think its because I cancelled an in-progress rsync, then did something else, as you describe.
+
+Is it ok to manually remove the file?
+
+Thanks!
+Albert
+"""]]

doc/design.mdwn

@@ -2,3 +2,5 @@ git-annex's high-level design is mostly inherent in the data that it
 stores in git, and alongside git. See [[internals]] for details.
 
 See [[encryption]] for design of encryption elements.
+
+See [[assistant]] for the design site for the git-annex [[/assistant]].

doc/design/assistant.mdwn

@@ -26,6 +26,7 @@ We are, approximately, here:
 * [[desymlink]]
 * [[deltas]]
 * [[leftovers]]
+* [[other todo items|todo]]
 
 ## blog
 

doc/design/assistant/blog/day_112__and_now_for_something_completely_different/comment_1_5e4fe1538d9ae1c450b0a6602fc6d29b._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmQ4Oe5-qOANRuZel9kDvtBfQG1zlEcIzw"
+ nickname="Dominik"
+ subject="PEP is ok"
+ date="2012-10-24T05:15:38Z"
+ content="""
+PEP and pubsub are really widely used. You will have a hard time finding a server that doesn't support it.
+
+Also note XMPPs ability to send status to only selected buddies and even resources. You can globally appear as offline while sending availability to myownjid@example.com/thatannexthere.
+"""]]

doc/design/assistant/blog/day_112__and_now_for_something_completely_different/comment_2_c5a734f611ecc95729904e645583ee43._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmQ4Oe5-qOANRuZel9kDvtBfQG1zlEcIzw"
+ nickname="Dominik"
+ subject="Remote control clients"
+ date="2012-10-24T05:18:02Z"
+ content="""
+Then of course there is XEP-0146, which has some advantages to it, like controlling annex from the context menu of another Jabber client. 
+"""]]

doc/design/assistant/todo.mdwn

@@ -0,0 +1,4 @@
+This is a subset of [[/todo]] for items tagged for the assistant.
+Link items to [[todo/done]] when done.
+
+[[!inline pages="tagged(design/assistant)" show=0 archive=yes]]

doc/forum/shared_cipher_tries_to_use_gpg/comment_2_f3260aea3a5bb9b95a9bdf1d0dfce090._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlFUH5H4OUj9vMJIoXQs8bheiptgANQ6fU"
+ nickname="lee"
+ subject="A little too old"
+ date="2012-10-23T20:47:40Z"
+ content="""
+That's the problem, then. I have 3.20120406. For my purposes a gpg key with no passphrase works, though. Thanks.
+"""]]

doc/special_remotes/bup/comment_2_612b038c15206f9f3c2e23c7104ca627._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.154.0.23"
+ subject="comment 2"
+ date="2012-10-23T20:01:43Z"
+ content="""
+@Albert, thanks for reporting this bug (but put them in [[bugs]] in future please).
+
+This is specific to using the bup special remote with encryption. Without encryption it works. And no, it won't manage to deduplicate anything that's encrypted, as far as I know. 
+
+I think bup-split must have used - for stdin in the past, but now, it just reads from stdin when no file is specified, so I've updated git-annex.
+"""]]

doc/todo/incremental_fsck.mdwn

@@ -17,3 +17,8 @@ clear the sticky bit. --[[Joey]]
 > * --max-age=30d  Once the incremental fsck completes and was started 30 days ago,
 >   start a new one.
 > * --time-limit --size-limit --file-limit: Limit how long the fsck runs.
+
+>> Calling this [[done]]. The `--incremental-schedule` option
+>> allows scheduling time between incremental fscks. `--time-limit` is
+>> done. I implemented `--smallerthan` independently. Not clear what
+>> `--file-limit` would be. --[[Joey]]

doc/todo/wishlist:_disable_automatic_commits.mdwn

@@ -0,0 +1,13 @@
+When using the [[/assistant]] on some of my repositories, I would like to retain manual control over the granularity and contents of the commit history.  Some motivating reasons:
+
+* manually specified commit messages makes the history easier to follow
+* make a series of minor changes to a file over a period of a few hours would result in a single commit rather than capturing intermediate incomplete edits
+* manual choice of which files to annex (based on predicted usage) could be useful, e.g. a repo might contain a 4MB PDF which you want available in *every* remote even without `git annex get`, and also some 2MB images which are only required in some remotes
+
+Obviously this needs to be configurable at least per repository, and ideally perhaps even per remote, since usage habits can vary from machine to machine (e.g. I could choose to commit manually from my desktop machine which has a nice comfy keyboard and large screen, but this would be too much pain to do from my tiny netbook).
+
+In fact, this is vaguely related to [[design/assistant/partial_content]], since the usefulness of the commit history depends on the context of the data being manipulated, which in turn depends on which subdirectories are being touched.  So any mechanism for disabling sync per directory could potentially be reused for disabling auto-commit per directory.
+
+According to Joey, it should be easy to arrange for the watcher thread not to run, but would need some more work for the assistant to notice manual commits in order to sync them; however the assistant already does some crazy inotify watching of git refs, in order to detect incoming pushes, so detecting manual commits wouldn't be a stretch.
+
+[[!tag design/assistant]]