Merge branch 'master' into preferred

Annex/Import.hs

@@ -163,40 +163,44 @@ buildImportCommit remote importtreeconfig importcommitconfig importable =
 buildImportCommit' :: Remote -> ImportCommitConfig -> Maybe Sha -> History Sha -> Annex (Maybe Sha)
 buildImportCommit' remote importcommitconfig mtrackingcommit imported@(History ti _) =
 	case mtrackingcommit of
-		Nothing -> Just <$> mkcommits imported
+		Nothing -> Just <$> mkcommitsunconnected imported
 		Just trackingcommit -> do
 			-- Get history of tracking branch to at most
 			-- one more level deep than what was imported,
 			-- so we'll have enough history to compare,
 			-- but not spend too much time getting it.
-			let maxdepth = succ (historyDepth imported)
+			let maxdepth = succ importeddepth
 			inRepo (getHistoryToDepth maxdepth trackingcommit)
 				>>= go trackingcommit
   where
-	go _ Nothing = Just <$> mkcommits imported
+	go _ Nothing = Just <$> mkcommitsunconnected imported
 	go trackingcommit (Just h)
 		-- If the tracking branch head is a merge commit
 		-- and one side of the merge matches the history,
 		-- nothing new needs to be committed.
-		| t == ti && any (sametodepth imported) (S.toList s) = return Nothing
+		| t == ti && any sametodepth (S.toList s) = return Nothing
 		-- If the tracking branch matches the history,
 		-- nothing new needs to be committed.
 		-- (This is unlikely to happen.)
-		| sametodepth imported h' = return Nothing
+		| sametodepth h' = return Nothing
 		| otherwise = do
 			importedcommit <- case getRemoteTrackingBranchImportHistory h of
-				Nothing ->
-					mkcommits imported
-				Just oldimported@(History oldhc _) -> do
-					let oldimportedtrees = mapHistory historyCommitTree oldimported
-					mknewcommits oldhc oldimportedtrees imported
+				Nothing -> mkcommitsunconnected imported
+				Just oldimported@(History oldhc _)
+					| importeddepth == 1 ->
+						mkcommitconnected imported oldimported
+					| otherwise -> do
+						let oldimportedtrees = mapHistory historyCommitTree oldimported
+						mknewcommits oldhc oldimportedtrees imported
 			ti' <- addBackNonPreferredContent remote ti
 			Just <$> makeRemoteTrackingBranchMergeCommit'
 				trackingcommit importedcommit ti'
 	  where
 		h'@(History t s) = mapHistory historyCommitTree h
 
-	sametodepth a b = a == truncateHistoryToDepth (historyDepth a) b
+	importeddepth = historyDepth imported
+
+	sametodepth b = imported == truncateHistoryToDepth importeddepth b
 
 	mkcommit parents tree = inRepo $ Git.Branch.commitTree
 		(importCommitMode importcommitconfig)
@@ -204,8 +208,16 @@ buildImportCommit' remote importcommitconfig mtrackingcommit imported@(History t
 		parents
 		tree
 
-	mkcommits (History importedtree hs) = do
-		parents <- mapM mkcommits (S.toList hs)
+	-- Start a new history of import commits, not connected to any
+	-- prior import commits.
+	mkcommitsunconnected (History importedtree hs) = do
+		parents <- mapM mkcommitsunconnected (S.toList hs)
+		mkcommit parents importedtree
+
+	-- Commit the new history connected with the old history.
+	-- Used when the import is not versioned, so the history depth is 1.
+	mkcommitconnected (History importedtree _) (History oldhc _) = do
+		let parents = [historyCommit oldhc]
 		mkcommit parents importedtree
 
 	-- Reuse the commits from the old imported History when possible.

CHANGELOG

@@ -7,6 +7,8 @@ git-annex (7.20190508) UNRELEASED; urgency=medium
     annex.jobs=cpus, or using option --jobs=cpus or -Jcpus.
   * Honor preferred content of a special remote when exporting trees to it;
     unwanted files are filtered out of the tree that is exported.
+  * Improve shape of commit tree when importing from unversioned special
+    remotes.
 
  -- Joey Hess <id@joeyh.name>  Mon, 06 May 2019 13:52:02 -0400
 

doc/bugs/Android_install_doesn__39__t_permanently_add_to___36__PATH/comment_8_c15caba920b49cebf88ebd36caa053b7._comment

@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e"
+ nickname="branchable"
+ avatar="http://cdn.libravatar.org/avatar/ae41dba34ee6000056f00793c695be75"
+ subject="comment 8"
+ date="2019-05-20T15:23:45Z"
+ content="""
+I understand, and of course it's your prerogative to take that position.  While there are many UNIX shells out there, I'm pretty sure that bash and zsh are the two which combined take the lion share of the market.
+
+If you're worried about the impact of adding `cp .profile .zprofile` to the install, a reasonable alternative would be to document that the install relies on setting up `$PATH` via `.profile` - this confusion only arose in the first place because that mechanism is somewhat hidden rather than taking place directly in the install script.
+"""]]

doc/bugs/surprising_import_tree_merge_result.mdwn

@@ -13,9 +13,3 @@ raw/photos/something
 This is quite surprising. It seems somehow when it modified the old
 tree, it rooted the new at the location the old had been moved to in
 master.
-
-----
-
-Also, commit cd55a5be377020b7261c3a737b5e32a01e22f4b8 had no parent, so
-git diff --stat shows it as adding every file that was in the tree before
-as well as the imported files, which seems wrong at least visually.

doc/devblog/day_588__export_preferred_content.mdwn

@@ -0,0 +1,8 @@
+Made `git annex export --to remote` honor the preferred content of the
+remote. In a nice bit of code reuse, `adjustTree` was just what I needed to
+filter unwanted content out of the exported tree.
+
+Then a hard problem: When a tree is exported with some non-preferred
+content filtered out, importing from the remote generates a tree that is
+lacking those files, but merging that tree would delete the files
+from the working tree. Solving that took the rest of the day.

doc/todo/import_from_special_remote_large_git_log.mdwn

@@ -0,0 +1,22 @@
+After import from a special remote and merge, git log --stat shows
+a large diff, because every file that got imported from the special 
+remote is put on a commit with no parent, so the diff shows it as if those
+files are newly added.
+
+(Of course when using S3 with versioning, the commit tree does have
+parents, but still a root commit with no parent.)
+
+This does not seem avoidable for the initial import, even if the remote was
+populated by an earlier export, that starts a new, disconnected history for
+reasons explained in the import tree design. 
+
+Subsequent imports though could fix it, by setting the parent of the new
+import to the previous import.
+
+For versioned imports, it reuses commits from the old imported history,
+and adds more commits on top of those, so this is mostly not a problem
+there. If the old and new imported histories are disjoint, a commit or
+commits will be made with no parent, but that seems acceptable; it's an
+edge case and it's replicating the information from the remote.
+
+> [[done]] --[[Joey]]