From 8bec41ceedefd2c2121849461a61066100f756c0 Mon Sep 17 00:00:00 2001 From: "branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e" Date: Mon, 20 May 2019 15:23:45 +0000 Subject: [PATCH 1/4] Added a comment --- ...omment_8_c15caba920b49cebf88ebd36caa053b7._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/bugs/Android_install_doesn__39__t_permanently_add_to___36__PATH/comment_8_c15caba920b49cebf88ebd36caa053b7._comment diff --git a/doc/bugs/Android_install_doesn__39__t_permanently_add_to___36__PATH/comment_8_c15caba920b49cebf88ebd36caa053b7._comment b/doc/bugs/Android_install_doesn__39__t_permanently_add_to___36__PATH/comment_8_c15caba920b49cebf88ebd36caa053b7._comment new file mode 100644 index 0000000000..65b5546ed7 --- /dev/null +++ b/doc/bugs/Android_install_doesn__39__t_permanently_add_to___36__PATH/comment_8_c15caba920b49cebf88ebd36caa053b7._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e" + nickname="branchable" + avatar="http://cdn.libravatar.org/avatar/ae41dba34ee6000056f00793c695be75" + subject="comment 8" + date="2019-05-20T15:23:45Z" + content=""" +I understand, and of course it's your prerogative to take that position. While there are many UNIX shells out there, I'm pretty sure that bash and zsh are the two which combined take the lion share of the market. + +If you're worried about the impact of adding `cp .profile .zprofile` to the install, a reasonable alternative would be to document that the install relies on setting up `$PATH` via `.profile` - this confusion only arose in the first place because that mechanism is somewhat hidden rather than taking place directly in the install script. +"""]] From 357f420ba3b27c815d7fdd86291719d1a63c569e Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 20 May 2019 16:46:14 -0400 Subject: [PATCH 2/4] devblog --- doc/devblog/day_588__export_preferred_content.mdwn | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/devblog/day_588__export_preferred_content.mdwn diff --git a/doc/devblog/day_588__export_preferred_content.mdwn b/doc/devblog/day_588__export_preferred_content.mdwn new file mode 100644 index 0000000000..fd2cf1d6c2 --- /dev/null +++ b/doc/devblog/day_588__export_preferred_content.mdwn @@ -0,0 +1,8 @@ +Made `git annex export --to remote` honor the preferred content of the +remote. In a nice bit of code reuse, `adjustTree` was just what I needed to +filter unwanted content out of the exported tree. + +Then a hard problem: When a tree is exported with some non-preferred +content filtered out, importing from the remote generates a tree that is +lacking those files, but merging that tree would delete the files +from the working tree. Solving that took the rest of the day. From 5af9e7f3d09655ce66f6123e5b9e1e2c257ffd9b Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 21 May 2019 11:10:13 -0400 Subject: [PATCH 3/4] break out a todo --- .../surprising_import_tree_merge_result.mdwn | 6 ------ ...ort_from_special_remote_large_git_log.mdwn | 20 +++++++++++++++++++ 2 files changed, 20 insertions(+), 6 deletions(-) create mode 100644 doc/todo/import_from_special_remote_large_git_log.mdwn diff --git a/doc/bugs/surprising_import_tree_merge_result.mdwn b/doc/bugs/surprising_import_tree_merge_result.mdwn index 7831eb2a5d..3d2f37f3c2 100644 --- a/doc/bugs/surprising_import_tree_merge_result.mdwn +++ b/doc/bugs/surprising_import_tree_merge_result.mdwn @@ -13,9 +13,3 @@ raw/photos/something This is quite surprising. It seems somehow when it modified the old tree, it rooted the new at the location the old had been moved to in master. - ----- - -Also, commit cd55a5be377020b7261c3a737b5e32a01e22f4b8 had no parent, so -git diff --stat shows it as adding every file that was in the tree before -as well as the imported files, which seems wrong at least visually. diff --git a/doc/todo/import_from_special_remote_large_git_log.mdwn b/doc/todo/import_from_special_remote_large_git_log.mdwn new file mode 100644 index 0000000000..f1b43221fa --- /dev/null +++ b/doc/todo/import_from_special_remote_large_git_log.mdwn @@ -0,0 +1,20 @@ +After import from a special remote and merge, git log --stat shows +a large diff, because every file that got imported from the special +remote is put on a commit with no parent, so the diff shows it as if those +files are newly added. + +(Of course when using S3 with versioning, the commit tree does have +parents, but still a root commit with no parent.) + +This does not seem avoidable for the initial import, even if the remote was +populated by an earlier export, that starts a new, disconnected history for +reasons explained in the import tree design. + +Subsequent imports though could fix it, by setting the parent of the new +import to the previous import. + +For versioned imports, it reuses commits from the old imported history, +and adds more commits on top of those, so this is mostly not a problem +there. If the old and new imported histories are disjoint, a commit or +commits will be made with no parent, but that seems acceptable; it's an +edge case and it's replicating the information from the remote. From 5e1221ad530fbc9bb63cc4f5997485fc5494c6fe Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 21 May 2019 11:32:54 -0400 Subject: [PATCH 4/4] Improve shape of commit tree when importing from unversioned special remotes Make the import have the previous import as a parent, so eg `git log --stat` displays a useful diff. Also a minor optimisation, only calculate the depth of the imported history once. --- Annex/Import.hs | 38 ++++++++++++------- CHANGELOG | 2 + ...ort_from_special_remote_large_git_log.mdwn | 2 + 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/Annex/Import.hs b/Annex/Import.hs index 006bd02999..2e152a0f7a 100644 --- a/Annex/Import.hs +++ b/Annex/Import.hs @@ -163,40 +163,44 @@ buildImportCommit remote importtreeconfig importcommitconfig importable = buildImportCommit' :: ImportCommitConfig -> Maybe Sha -> History Sha -> Annex (Maybe Sha) buildImportCommit' importcommitconfig mtrackingcommit imported@(History ti _) = case mtrackingcommit of - Nothing -> Just <$> mkcommits imported + Nothing -> Just <$> mkcommitsunconnected imported Just trackingcommit -> do -- Get history of tracking branch to at most -- one more level deep than what was imported, -- so we'll have enough history to compare, -- but not spend too much time getting it. - let maxdepth = succ (historyDepth imported) + let maxdepth = succ importeddepth inRepo (getHistoryToDepth maxdepth trackingcommit) >>= go trackingcommit where - go _ Nothing = Just <$> mkcommits imported + go _ Nothing = Just <$> mkcommitsunconnected imported go trackingcommit (Just h) -- If the tracking branch head is a merge commit -- with a tree that matches the head of the history, -- and one side of the merge matches the history, -- nothing new needs to be committed. - | t == ti && any (sametodepth imported) (S.toList s) = return Nothing + | t == ti && any sametodepth (S.toList s) = return Nothing -- If the tracking branch matches the history, -- nothing new needs to be committed. -- (This is unlikely to happen.) - | sametodepth imported h' = return Nothing + | sametodepth h' = return Nothing | otherwise = do importedcommit <- case getRemoteTrackingBranchImportHistory h of - Nothing -> - mkcommits imported - Just oldimported@(History oldhc _) -> do - let oldimportedtrees = mapHistory historyCommitTree oldimported - mknewcommits oldhc oldimportedtrees imported + Nothing -> mkcommitsunconnected imported + Just oldimported@(History oldhc _) + | importeddepth == 1 -> + mkcommitconnected imported oldimported + | otherwise -> do + let oldimportedtrees = mapHistory historyCommitTree oldimported + mknewcommits oldhc oldimportedtrees imported Just <$> makeRemoteTrackingBranchMergeCommit' trackingcommit importedcommit ti where h'@(History t s) = mapHistory historyCommitTree h - sametodepth a b = a == truncateHistoryToDepth (historyDepth a) b + importeddepth = historyDepth imported + + sametodepth b = imported == truncateHistoryToDepth importeddepth b mkcommit parents tree = inRepo $ Git.Branch.commitTree (importCommitMode importcommitconfig) @@ -204,8 +208,16 @@ buildImportCommit' importcommitconfig mtrackingcommit imported@(History ti _) = parents tree - mkcommits (History importedtree hs) = do - parents <- mapM mkcommits (S.toList hs) + -- Start a new history of import commits, not connected to any + -- prior import commits. + mkcommitsunconnected (History importedtree hs) = do + parents <- mapM mkcommitsunconnected (S.toList hs) + mkcommit parents importedtree + + -- Commit the new history connected with the old history. + -- Used when the import is not versioned, so the history depth is 1. + mkcommitconnected (History importedtree _) (History oldhc _) = do + let parents = [historyCommit oldhc] mkcommit parents importedtree -- Reuse the commits from the old imported History when possible. diff --git a/CHANGELOG b/CHANGELOG index dbcab637ee..bb47c69098 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -5,6 +5,8 @@ git-annex (7.20190508) UNRELEASED; urgency=medium * Makefile: Added install-completions to install target. * Added the ability to run one job per CPU (core), by setting annex.jobs=cpus, or using option --jobs=cpus or -Jcpus. + * Improve shape of commit tree when importing from unversioned special + remotes. -- Joey Hess Mon, 06 May 2019 13:52:02 -0400 diff --git a/doc/todo/import_from_special_remote_large_git_log.mdwn b/doc/todo/import_from_special_remote_large_git_log.mdwn index f1b43221fa..5949367156 100644 --- a/doc/todo/import_from_special_remote_large_git_log.mdwn +++ b/doc/todo/import_from_special_remote_large_git_log.mdwn @@ -18,3 +18,5 @@ and adds more commits on top of those, so this is mostly not a problem there. If the old and new imported histories are disjoint, a commit or commits will be made with no parent, but that seems acceptable; it's an edge case and it's replicating the information from the remote. + +> [[done]] --[[Joey]]