assistant: Be smarter about avoiding unncessary transfers.

Just before starting a transfer, do one last check that it's still
preferred content.

I was just doing this for uploads, as part of the smarter flood filling
bug, but realized it's also possible for a download that was preferred
content to change to not be before the download begins, so check that too.

Assistant/Threads/Transferrer.hs

@@ -21,6 +21,7 @@ import qualified Remote
 import Types.Key
 import Locations.UserConfig
 import Assistant.Threads.TransferWatcher
+import Annex.Wanted
 
 import System.Process (create_group)
 
@@ -103,21 +104,40 @@ startTransfer program t info = case (transferRemote info, associatedFile info) o
 			, File file
 			]
 
-{- Checks if the file to download is already present, or the remote
+{- Called right before a transfer begins, this is a last chance to avoid
- - being uploaded to isn't known to have the file. -}
+ - unnecessary transfers.
+ -
+ - For downloads, we obviously don't need to download if the already
+ - have the object.
+ -
+ - Smilarly, for uploads, check if the remote is known to already have
+ - the object.
+ -
+ - Also, uploads get queued to all remotes, in order of cost.
+ - This may mean, for example, that an object is uploaded over the LAN
+ - to a locally paired client, and once that upload is done, a more
+ - expensive transfer remote no longer wants the object. (Since
+ - all the clients have it already.) So do one last check if this is still
+ - preferred content.
+ -
+ - We'll also do one last preferred content check for downloads. An
+ - example of a case where this could be needed is if a download is queued
+ - for a file that gets moved out of an archive directory -- but before
+ - that download can happen, the file is put back in the archive.
+ -}
 shouldTransfer :: Transfer -> TransferInfo -> Annex Bool
 shouldTransfer t info
 	| transferDirection t == Download =
-		not <$> inAnnex key
+		(not <$> inAnnex key) <&&> wantGet True file
-	| transferDirection t == Upload =
+	| transferDirection t == Upload = case transferRemote info of
-		{- Trust the location log to check if the
-		 - remote already has the key. This avoids
-		 - a roundtrip to the remote. -}
-		case transferRemote info of
 		Nothing -> return False
-			Just remote -> 
+		Just r -> notinremote r
-				notElem (Remote.uuid remote)
+			<&&> wantSend True file (Remote.uuid r)
-					<$> loggedLocations key
 	| otherwise = return False
   where
 	key = transferKey t
+	file = associatedFile info
+
+	{- Trust the location log to check if the remote already has
+	 - the key. This avoids a roundtrip to the remote. -}
+	notinremote r = notElem (Remote.uuid r) <$> loggedLocations key

debian/changelog

@@ -48,6 +48,7 @@ git-annex (4.20130228) UNRELEASED; urgency=low
     combines all their changes into a single commit.
   * assistant: Fix ~/.ssh/git-annex-shell wrapper to work when the
     ssh key does not force a command.
+  * assistant: Be smarter about avoiding unncessary transfers.
 
   * webapp: DTRT when told to create a git repo that already exists.
   * webapp: Work around bug in Warp's slowloris attack prevention code,

doc/bugs/smarter_flood_filling.mdwn

@@ -12,7 +12,9 @@ this.
 1. Ensure that locally paired computers have a lower cost than network
    transfer remotes.
 2. When queuing uploads, queue transfers to the lowest cost remotes first.
+   (already done)
 3. Just before starting a transfer, re-check if the transfer is still wanted.
+   (done)
 
 Now, unnecessary transfers to tranfer repos are avoided if it can send
 the file locally instead.