don't copy old date metadata when adding new version of a file

When adding a new version of a file, and annex.genmetadata is enabled,
don't copy the data metadata from the old version of the file, instead use
the mtime of the file. Rationalle being that the user has requested to
generate metadata and so would expect to get the new mtime into metadata.

Also, avoid warning about copying metadata when all the old metadata is
date metadata. Which was rather the harder part.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.

Annex/MetaData.hs

@@ -22,7 +22,6 @@ import Annex.CatFile
 import Utility.Glob
 
 import qualified Data.Set as S
-import qualified Data.Map as M
 import Data.Time.Calendar
 import Data.Time.Clock
 import Data.Time.Clock.POSIX
@@ -41,28 +40,43 @@ genMetaData :: Key -> FilePath -> FileStatus -> Annex ()
 genMetaData key file status = do
 	catKeyFileHEAD file >>= \case
 		Nothing -> noop
-		Just oldkey -> 
-			whenM (copyMetaData oldkey key)
+		Just oldkey ->
+			-- Have to copy first, before adding any
+			-- more metadata, because copyMetaData does not
+			-- preserve any metadata already on key.
+			whenM (copyMetaData oldkey key <&&> (not <$> onlydatemeta oldkey)) $
 				warncopied
 	whenM (annexGenMetaData <$> Annex.getGitConfig) $ do
-		curr <- getCurrentMetaData key
-		addMetaData key (dateMetaData mtime curr)
+		old <- getCurrentMetaData key
+		addMetaData key (dateMetaData mtime old)
   where
 	mtime = posixSecondsToUTCTime $ realToFrac $ modificationTime status
 	warncopied = warning $ 
 		"Copied metadata from old version of " ++ file ++ " to new version. " ++ 
 		"If you don't want this copied metadata, run: git annex metadata --remove-all " ++ file
+	-- If the only fields copied were date metadata, and they'll
+	-- be overwritten with the current mtime, no need to warn about
+	-- copying.
+	onlydatemeta oldkey = ifM (annexGenMetaData <$> Annex.getGitConfig)
+		( null . filter (not . isDateMetaField . fst) . fromMetaData 
+			<$> getCurrentMetaData oldkey
+		, return False
+		)
 
 {- Generates metadata for a file's date stamp.
- - Does not overwrite any existing metadata values. -}
+ -
+ - Any date fields in the old metadata will be overwritten.
+ - 
+ - Note that the returned MetaData does not contain all the input MetaData,
+ - only changes to add the date fields. -}
 dateMetaData :: UTCTime -> MetaData -> MetaData
-dateMetaData mtime old = MetaData $ M.fromList $ filter isnew
-	[ (yearMetaField, S.singleton $ toMetaValue $ show y)
-	, (monthMetaField, S.singleton $ toMetaValue $ show m)
-	, (dayMetaField, S.singleton $ toMetaValue $ show d)
-	]
+dateMetaData mtime old = modMeta old $
+	(SetMeta yearMetaField $ S.singleton $ toMetaValue $ show y)
+		`ComposeModMeta`
+	(SetMeta monthMetaField $ S.singleton $ toMetaValue $ show m)
+		`ComposeModMeta`
+	(SetMeta dayMetaField $ S.singleton $ toMetaValue $ show d)
   where
-	isnew (f, _) = S.null (currentMetaDataValues f old)
 	(y, m, d) = toGregorian $ utctDay mtime
 
 {- Parses field=value, field+=value, field-=value, field?=value -}

Annex/MetaData/StandardFields.hs

@@ -10,6 +10,7 @@ module Annex.MetaData.StandardFields (
 	yearMetaField,
 	monthMetaField,
 	dayMetaField,
+	isDateMetaField,
 	lastChangedField,
 	mkLastChangedField,
 	isLastChangedField
@@ -31,6 +32,12 @@ monthMetaField = mkMetaFieldUnchecked "month"
 dayMetaField :: MetaField
 dayMetaField = mkMetaFieldUnchecked "day"
 
+isDateMetaField :: MetaField -> Bool
+isDateMetaField yearMetaField = True
+isDateMetaField monthMetaField = True
+isDateMetaField datMetaField = True
+isDateMetaField _ = False
+
 lastChangedField :: MetaField
 lastChangedField = mkMetaFieldUnchecked lastchanged
 

CHANGELOG

@@ -8,6 +8,9 @@ git-annex (6.20180317) UNRELEASED; urgency=medium
     to configure transfer retries.
   * Also do forward retrying in cases where no exception is thrown,
     but the transfer failed.
+  * When adding a new version of a file, and annex.genmetadata is enabled,
+    don't copy the data metadata from the old version of the file,
+    instead use the mtime of the file.
 
  -- Joey Hess <id@joeyh.name>  Mon, 19 Mar 2018 23:13:59 -0400
 

Types/MetaData.hs

@@ -252,13 +252,15 @@ data ModMeta
 	= AddMeta MetaField MetaValue
 	| DelMeta MetaField (Maybe MetaValue)
 	-- ^ delete value of a field. With Just, only that specific value
-	-- is deleted; with Nothing, all current values are deleted.a
+	-- is deleted; with Nothing, all current values are deleted.
 	| DelAllMeta
 	-- ^ delete all currently set metadata
 	| SetMeta MetaField (S.Set MetaValue)
 	-- ^ removes any existing values
 	| MaybeSetMeta MetaField MetaValue
 	-- ^ set when field has no existing value
+	| ComposeModMeta ModMeta ModMeta
+	-- ^ composing multiple modifications
 	deriving (Show)
 
 {- Applies a ModMeta, generating the new MetaData.
@@ -279,6 +281,7 @@ modMeta m (SetMeta f s) = updateMetaData' f s $
 modMeta m (MaybeSetMeta f v)
 	| S.null (currentMetaDataValues f m) = updateMetaData f v emptyMetaData
 	| otherwise = emptyMetaData
+modMeta m (ComposeModMeta a b) = unionMetaData (modMeta m a) (modMeta m b)
 
 {- Avoid putting too many fields in the map; extremely large maps make
  - the seriaization test slow due to the sheer amount of data.

doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add.mdwn

@@ -20,3 +20,4 @@ output: `Copied metadata from old version of bar to new version. If you don't wa
 ### What version of git-annex are you using? On what operating system?
 6.20180227, various Linuxes.
 
+> [[fixed|done]] --[[Joey]]

doc/bugs/if_annex.genmetadata_is_true__44___modification_metadata_is_imported_from_older_file_versions_after_unlock+add/comment_1_b215c8ce0f2094a116672572951b19fc._comment

@@ -0,0 +1,25 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2018-04-04T16:51:36Z"
+ content="""
+This happens in Annex.MetaData.genMetaData. First it copies
+metadata from the oldkey to the new key. Then it
+calls addMetaData on the dateMetaData of the file's mtime.
+
+In dateMetaData, there's a `filter isnew`, which makes
+it filter out any of the date fields that already exist
+in the metadata of the new key.
+
+This was done intentionally, see 
+[[!commit 8d5158fa3151be4c7fc698b96ed887b43ac48769]]
+But that's lacking an explanation of why it was done.
+
+Note that dateMetaData is also used in Command.ImportFeed
+to convert a itempubdate into year and month metadata.
+But changing its behavior to override old dates 
+would not change that code path.
+
+So, I don't see a problem with making this change, and have gone ahead and
+done it.
+"""]]