git-annex/doc/bugs/softlink_mtime.mdwn

When adding files to git annex, softlinks are created with current atime (and ctime, etc). Instead, the atime of the added file should be used and added to the meta-data, restoring it everywhere an annex is cloned to. -- RichiH

Optionally, editing the meta-data should change the times in all annexes.

> Thing is, git does not preserve file timestamps much at all. 
> It's not uncommon for a `git checkout` to or `git update` to
> mess up timestamps. This is why things like metastore exist (and
> metastore should work ok with git annexed files too). Trying to 
> make annexed file symlinks have better timestamp handling than regular
> files in git seems pointless. --[[Joey]]

> > Improving an area where git is (not yet?) good at still makes sense, imo. Photos and the like need absolute timestamps more than source code which is fine with relative timestamps (local builds & updates). Maintaining global timestamps for source code could even cause a lot of unwanted effects. As it is, this issue is the only, but a major, blocker for me before I can start adapting git-annex. As I have three different use cases for it, this is a shame. Unfortunately, I don't speak any Haskell so scratching my own itch isn't do-able (without major effort and not soon, at least). Is there a realistic chance that you will tackle this nonetheless or is this WONTFIX? -- RichiH

>>> Not quite WONTFIX. git-annex should at least, when adding new files,
>>> preserve their timestamp in the symlink it creates.
>>>
>>> Since it doesn't have anything to do with maintaining the symlinks
>>> during an update, or a clone, etc, maintaining the permissions of them 
>>> is also out of scope, and it's best to just use metastore if you need
>>> it. Otherwise, git-annex would have to reimplement metastore, and is
>>> unlikely to do it better.

>>>> OK, thanks for the clarification. Would it be acceptable for you to put the timestamps into the metastore with vanilla git? If such an option existed, everyone would be able to benefit and not just me. -- RichiH

>>>>> I've now committed to git changes to make git-annex add make
>>>>> symlinks that reflect the original file's mtime. (It's not possible
>>>>> to set the ctime of a symlink; nor would you want to as messing with
>>>>> ctimes can break backup software ... and atime doesn't much matter.)
>>>>> 
>>>>> So all you have to do is make the pre-commit hook call
>>>>> [metastore](http://david.hardeman.nu/software.php). The hook
>>>>> would look like this: ---[[Joey]]

	#!/bin/sh
	git annex pre-commit .
	metastore --save
	git add .metadata

>>>>>> Thanks a lot. Doing this in a new git-annex repo from the start should at least ensure local consistency and I assume I can simply add a post-pull hook to restore the mtimes on all all other repositories? -- RichiH

>>>>>>> This is even better:

    #!/bin/sh
    if ! type metastore >/dev/null; then echo "$0: metastore is not installed; exiting"; exit 1; fi
    git annex pre-commit .
    metastore --save
    git add .metadata

>>>>>>> -- RichiH

>>>>>>>> After getting to actually play with this from different machines with a bare git as central instance for several distributed repos, the metastore trick does not work. The .metadata is causing merge conflicts for every pull. I removed the "done" tag from this issue. -- RichiH

>>>>>>>>> softbox sounds _really_ nice. File systems need to preserve mtimes. Oviously, it would be nice if git-annex exposed this to the upper layer instead of relying on this FUSE implementation, or the next, or the other totally cool thing around the corner to implement it again and again. 
>>>>>>>>> I talked to the author of metastore; he is aware that the format is merge-unfriendly but never needed merges for himself. He is aware that this is not ideal for something like git. He does not have the time to implement a text storage instead of binary and I lack the skills to do it. If metastore is used, all it would need to do is introduce a new version of the store (it's versioned, apparently) and save metadata in text, one file per line. xattr would need to be ASCII-armoured, the rest could be plain text. I still think storing this directly in git-annex would make the most sense. Introducing a metadata storage file per storage object in .git/annex and using the object file's name as index is impossible because several softlinks might point to one object so it would need to be done per-softlink :/ -- RichiH