make sync update adjusted branch to hide/unhide

This completes initial support for --hide-missing, although the
assistant still needs to be updated and it perhaps needs to be sped up,
and maybe there needs to be a way for git-annex get to operate on
missing files. Opened some more todos for those things.

This commit was sponsored by Henrik Riomar.

Annex/AdjustedBranch.hs

@@ -19,9 +19,10 @@ module Annex.AdjustedBranch (
 	fromAdjustedBranch,
 	getAdjustment,
 	enterAdjustedBranch,
+	updateAdjustedBranch,
 	adjustBranch,
 	adjustToCrippledFileSystem,
-	updateAdjustedBranch,
+	mergeToAdjustedBranch,
 	propigateAdjustedCommits,
 	AdjustedClone(..),
 	checkAdjustedClone,
@@ -150,7 +151,7 @@ enterAdjustedBranch :: Adjustment -> Annex Bool
 enterAdjustedBranch adj = inRepo Git.Branch.current >>= \case
 	Just currbranch -> case getAdjustment currbranch of
 		Just curradj | curradj == adj ->
-			reenterAdjustedBranch adj currbranch
+			updateAdjustedBranch adj (AdjBranch currbranch)
 				(fromAdjustedBranch currbranch)
 		_ -> go currbranch
 	Nothing -> do
@@ -197,8 +198,8 @@ checkoutAdjustedBranch (AdjBranch b) checkoutparams = do
  - deleting and rebuilding the adjusted branch, and then checking it out.
  - But, it can be implemented more efficiently than that.
  -}
-reenterAdjustedBranch :: Adjustment -> Branch -> OrigBranch -> Annex Bool
-reenterAdjustedBranch adj@(PresenceAdjustment _ _) currbranch origbranch = do
+updateAdjustedBranch :: Adjustment -> AdjBranch -> OrigBranch -> Annex Bool
+updateAdjustedBranch adj@(PresenceAdjustment _ _) (AdjBranch currbranch) origbranch = do
 	b <- preventCommits $ \commitlck -> do
 		-- Avoid losing any commits that the adjusted branch has that
 		-- have not yet been propigated back to the origbranch.
@@ -319,8 +320,8 @@ findAdjustingCommit (AdjBranch b) = go =<< catCommit b
 {- Update the currently checked out adjusted branch, merging the provided
  - branch into it. Note that the provided branch should be a non-adjusted
  - branch. -}
-updateAdjustedBranch :: Branch -> (OrigBranch, Adjustment) -> [Git.Merge.MergeConfig] -> Annex Bool -> Git.Branch.CommitMode -> Annex Bool
-updateAdjustedBranch tomerge (origbranch, adj) mergeconfig canresolvemerge commitmode = catchBoolIO $
+mergeToAdjustedBranch :: Branch -> (OrigBranch, Adjustment) -> [Git.Merge.MergeConfig] -> Annex Bool -> Git.Branch.CommitMode -> Annex Bool
+mergeToAdjustedBranch tomerge (origbranch, adj) mergeconfig canresolvemerge commitmode = catchBoolIO $
 	join $ preventCommits go
   where
 	adjbranch@(AdjBranch currbranch) = originalToAdjusted origbranch adj

Assistant/Threads/Committer.hs

@@ -229,7 +229,7 @@ commitStaged msg = do
 		Right _ -> do
 			ok <- Command.Sync.commitStaged Git.Branch.AutomaticCommit msg
 			when ok $
-				Command.Sync.updateSyncBranch =<< getCurrentBranch
+				Command.Sync.updateBranches =<< getCurrentBranch
 			return ok
 
 {- OSX needs a short delay after a file is added before locking it down,

CHANGELOG

@@ -7,7 +7,8 @@ git-annex (6.20181012) UNRELEASED; urgency=medium
     useful with --hide-missing to hide/unhide files after their content
     has been dropped or received.
   * git-annex sync --content supports --hide-missing; it can
-    be used to get the content of hidden files.
+    be used to get the content of hidden files, and it updates the
+    adjusted branch to hide/unhide files as necessary.
   * Removed the old Android app.
   * Removed support for building with very old ghc < 8.0.1,
     and with yesod < 1.4.3, and without concurrent-output,

Command/Sync.hs

@@ -19,7 +19,7 @@ module Command.Sync (
 	pushBranch,
 	updateBranch,
 	syncBranch,
-	updateSyncBranch,
+	updateBranches,
 	seekExportContent,
 ) where
 
@@ -227,7 +227,7 @@ mergeConfig =
 
 merge :: CurrBranch -> [Git.Merge.MergeConfig] -> ResolveMergeOverride -> Git.Branch.CommitMode -> Git.Branch -> Annex Bool
 merge currbranch mergeconfig resolvemergeoverride commitmode tomerge = case currbranch of
-	(Just b, Just adj) -> updateAdjustedBranch tomerge (b, adj) mergeconfig canresolvemerge commitmode
+	(Just b, Just adj) -> mergeToAdjustedBranch tomerge (b, adj) mergeconfig canresolvemerge commitmode
 	(b, _) -> autoMergeFrom tomerge b mergeconfig canresolvemerge commitmode
   where
 	canresolvemerge = case resolvemergeoverride of
@@ -342,17 +342,29 @@ needMerge (Just branch, madj) = ifM (allM id checks)
 
 pushLocal :: CurrBranch -> CommandStart
 pushLocal b = do
-	updateSyncBranch b
+	updateBranches b
 	stop
 
-updateSyncBranch :: CurrBranch -> Annex ()
-updateSyncBranch (Nothing, _) = noop
-updateSyncBranch (Just branch, madj) = do
+updateBranches :: CurrBranch -> Annex ()
+updateBranches (Nothing, _) = noop
+updateBranches (Just branch, madj) = do
 	-- When in an adjusted branch, propigate any changes made to it
-	-- back to the original branch.
-	maybe noop (propigateAdjustedCommits branch) madj
+	-- back to the original branch. The adjusted branch may also need
+	-- to be updated to hide/expose files.
+	case madj of
+		Nothing -> noop
+		Just adj -> do
+			let origbranch = branch
+			propigateAdjustedCommits origbranch adj
+			when (adjustmentHidesFiles adj) $ do
+				showSideAction "updating adjusted branch"
+				let adjbranch = originalToAdjusted origbranch adj
+				unlessM (updateAdjustedBranch adj adjbranch origbranch) $
+					warning $ unwords [ "Updating adjusted branch failed." ]
+					
 	-- Update the sync branch to match the new state of the branch
 	inRepo $ updateBranch (syncBranch branch) branch
+
 	-- In direct mode, we're operating on some special direct mode
 	-- branch, rather than the intended branch, so update the intended
 	-- branch.

doc/todo/assistant_support_hide-missing.mdwn

@@ -0,0 +1,11 @@
+While `git annex sync` supports branches set up by `git annex adjust
+--hide-missing`, the assisitant does not yet.
+
+This would invole:
+
+* The assistant also needs to scan the original branch when looking for
+  files to download.
+
+* The assistant ought to update the adjusted branch at some point after
+  downloads, but it's not clear when. Perhaps this will need to be deferred
+  until it can be done more cheaply, so it can do it after every file.

doc/todo/better_way_to_get_missing_files.mdwn

@@ -0,0 +1,34 @@
+Currently in a `git annex adjust --hide-missing` branch, the only way to
+get missing files is to run `git annex sync --content`, with appropriate preferred
+content settings or with `-C path` to get a specific path. This is kind of
+klunky for ad-hoc getting files.
+
+It would be possible to make `git annex get`, `git annex copy --from remote`, etc
+operate on hidden files. Some of the infrastructure is there; it may be as
+simple as using LsFiles.inRepoOrBranch instead of LsFiles.inRepo to seek
+the files.
+
+Two open questions:
+
+* Would it make more sense for this to be the default behavior when in the
+  adjusted branch, or for it to be an option, like "--branch=master"?
+
+  One advantage to the latter is that it would also be usable in bare
+  repos, or if you just for some reason want to get files present in some
+  other branch than the current one.
+
+  The former might be easier, but it could also be kind of surprising.
+  `git annex get .` would download tons of files, which `ls` doesn't
+  display.
+
+  Of course, both things could be supported too.
+
+* After getting the files, they'll still be hidden until the branch is
+  updated to include them. When should that happen?
+
+  The current method of re-running `git annex adjust --hide-missing` will
+  expose them, but the way that works is probably too slow to run after
+  every command. Even if it were fast, it would generate a lot of dangling
+  objects over time. So leave the update up to the user to run the command
+  when they want it? But then the user may get confused, why did it
+  download files and they didn't appear?

doc/todo/hide_missing_files.mdwn

@@ -25,3 +25,5 @@ Thanks for any advice! --[[anarcat]]
 > > cleaner, of course, if adjusted branches could do this. --[[anarcat]]
 
 Other discussions: [[forum/How_to_hide_broken_symlinks]], where the idea of using views to hide missing files is introduced and [[forum/How_do_I_hide_files_not_present_in_the_local_annex__63__]], where it is said that placeholders shouldn't be used in direct mode... --[[anarcat]]
+
+> [[done]]! --[[Joey]]

doc/todo/hide_missing_files/comment_4_e3bf8aaecc0f612873609c92814dcd12._comment

@@ -11,13 +11,4 @@ My implementation is not as fast as it could be; each update
 is a linear scan of the whole original branch and rebuild of the adjusted
 branch. But it all works so we'll defer speeding this up to later bug
 reports about it being too slow. ;)
-
-What still needs to be done:
-
-* `git annex sync` needs to update the adjusted branch.
-* The assistant also needs to scan the original branch when looking for
-  files to download.
-* The assistant ought to update the adjusted branch at some point after
-  downloads, but it's not clear when. Perhaps this will need to be deferred
-  until it can be done more cheaply, so it can do it after every file.
 """]]