git-annex/Annex
Joey Hess eb59da9dd2
Lower precision of timestamps in git-annex branch
This can reduce the size of the branch by up to 8%. My test was
running git-annex add 1000 times on one file each.
Lots of different high-resolution timestamps were recorded before
and eliminating those, after packing, the git repo was 8% smaller.

Due to the use of vector clocks, high resolution timestamps are
not necessary to make clear which information is most recent when
eg, a value is changed repeatedly in the same second. In such a
case, the vector clock will be advanced to the next second after
the last modification. For example, running
git-annex numcopies 1; git-annex numcopies 2
The first will record the current second, while the next records
the second after that even if it runs in the same second.

As for conflicting information written to two different clones of the
repository, this will make git-annex sometimes pick information that
was written earlier in a second over information written later in the
same second. Usually git-annex does not write conflicting information,
but there are some cases where it could. Eg, storing an object on a remote
can update the remote state log with some state. If two repos both store the
same object, and end up storing different remote state for some reason,
this can result in one that ran a tiny bit later winning. Such a situation
seems unlikely to be user visible. And a small amount of clock skew could
already result in such things.

The only case I can think of where this might be a user visible change
is if a configuration command like git-annex numcopies is being run
in 2 clones of a repository on the same machine at very
close to the same time. Then the user will know which they ran last,
and git-annex won't.

If that did become a problem, this could be dialed back to eg log
milliseconds with still some space saving.
2023-12-11 15:04:06 -04:00
..
AdjustedBranch fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Branch newtype MapLog 2023-11-13 14:37:22 -04:00
Concurrent differentiate between concurrency enabled at command line and by git config 2020-09-16 11:47:12 -04:00
Content migrate: support adding size back to URL keys 2023-12-08 16:22:14 -04:00
Debug implement fastDebug 2021-04-06 15:24:28 -04:00
LockPool avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
MetaData sqlite datbase for importfeed 2023-10-23 16:46:22 -04:00
SpecialRemote httpalso: Support being used with special remotes that use chunking. 2023-06-20 13:35:28 -04:00
VectorClock Lower precision of timestamps in git-annex branch 2023-12-11 15:04:06 -04:00
View annex.maxextensionlength for view 2023-03-24 14:01:38 -04:00
Action.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
AdjustedBranch.hs propigateAdjustedCommits: avoid overwriting diverged original branch 2023-07-05 17:09:49 -04:00
AutoMerge.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
BloomFilter.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Branch.hs started migrate --update 2023-12-07 15:50:52 -04:00
BranchState.hs disable journalIgnorable in enableInteractiveBranchAccess 2022-07-15 13:48:41 -04:00
CatFile.hs read a consistent amount from pointer file 2022-02-23 12:52:34 -04:00
ChangedRefs.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
CheckAttr.hs config: Added the --show-origin and --for-file options 2023-06-12 16:24:31 -04:00
CheckIgnore.hs move several readonly values to AnnexRead 2022-06-28 15:40:19 -04:00
Common.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
Concurrent.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Content.hs migrate: support adding size back to URL keys 2023-12-08 16:22:14 -04:00
CopyFile.hs Copy with a reflink when exporting a tree to a directory special remote 2023-03-28 13:09:14 -04:00
CurrentBranch.hs refactor getCurrentBranch 2018-10-19 17:29:18 -04:00
Debug.hs display explanations in --debug too 2023-07-31 13:06:40 -04:00
Difference.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
DirHashes.hs Added http special remote, which is useful for accessing other remotes that publish content stored in them via http/https. 2020-09-01 15:16:35 -04:00
Drop.hs prevent numcopies or mincopies being configured to 0 2022-03-28 15:20:34 -04:00
Environment.hs improve comments 2023-04-04 15:23:39 -04:00
Export.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
ExternalAddonProcess.hs use fastDebug everywhere it can be used 2021-04-06 15:41:24 -04:00
FileMatcher.hs onlyingroup 2023-07-31 14:43:58 -04:00
Fixup.hs fix a bug that prevented git-annex init from working in a submodule 2021-01-21 15:33:15 -04:00
GitOverlay.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
HashObject.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Hook.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Import.hs --explain for preferred/required content matching 2023-07-26 14:50:04 -04:00
Ingest.hs log migration trees to git-annex branch 2023-12-06 15:40:03 -04:00
Init.hs Fix breakage when git is configured with safe.bareRepository = explicit 2023-07-05 14:43:14 -04:00
InodeSentinal.hs fix perms for core.sharedRepository 2023-04-26 16:29:11 -04:00
Journal.hs create journal directory in withJournalHandle 2023-06-21 15:23:59 -04:00
Link.hs fix hang when built with unix-2.8 2023-08-01 20:22:28 -04:00
Locations.hs started migrate --update 2023-12-07 15:50:52 -04:00
LockFile.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
LockPool.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Magic.hs Serialize use of C magic library, which is not thread safe. 2020-09-17 17:27:42 -04:00
MetaData.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Multicast.hs use programPath consistently, not readProgramFile 2020-03-30 16:06:27 -04:00
Notification.hs fix build when dbus is enabled 2022-07-05 13:06:45 -04:00
NumCopies.hs improve message about 1 copy 2023-12-04 11:12:54 -04:00
Path.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Perms.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
PidLock.hs fix windows build 2022-09-26 12:08:04 -04:00
Queue.hs add restage log 2022-09-23 15:47:24 -04:00
RemoteTrackingBranch.hs refactor 2019-11-11 19:10:52 -04:00
ReplaceFile.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
SpecialRemote.hs init: Avoid autoenabling special remotes that have control characters in their names 2023-04-12 12:37:12 -04:00
Ssh.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
StallDetection.hs bwlimit 2021-09-21 16:58:10 -04:00
TaggedPush.hs simplify base64 to only use ByteString 2023-10-26 13:10:05 -04:00
Tmp.hs fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Transfer.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
TransferrerPool.hs avoid build warning on windows 2023-03-27 12:19:26 -04:00
UntrustedFilePath.hs fix mojibake reversion in display of utf8 2023-04-12 13:53:30 -04:00
UpdateInstead.hs v7 for all repositories 2019-08-30 14:09:14 -04:00
Url.hs avoid unncessary use of curl when conduit will do 2023-08-22 10:25:53 -04:00
UUID.hs eliminate s2w8 and w82s 2023-10-26 13:12:57 -04:00
VariantFile.hs more RawFilePath 2019-12-18 17:10:28 -04:00
VectorClock.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Verify.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Version.hs v8 repositories automatically upgrade to v9 2022-07-25 16:20:04 -04:00
View.hs annex.maxextensionlength for view 2023-03-24 14:01:38 -04:00
Wanted.hs new matching options --want-get-by and --want-drop-by 2022-07-28 13:26:03 -04:00
WorkerPool.hs start splitting out readonly values from AnnexState 2021-04-02 15:51:44 -04:00
WorkTree.hs use lookupKeyStaged in --batch code paths 2022-10-26 14:43:06 -04:00
YoutubeDl.hs nub list of files 2023-07-09 14:18:25 -04:00