mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/doc
History
Joey Hess 1acdd18ea8
deal better with clock skew situations, using vector clocks 
* Deal with clock skew, both forwards and backwards, when logging
  information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
  rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.

When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.

When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.

When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.

(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)

Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.

A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.

Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.

Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.

Sponsored-by: Jochen Bartl on Patreon
2021-08-04 12:33:46 -04:00
..
android
Android remove old android comments/bug reports 2021-06-21 13:21:27 -04:00
assistant
automatic_conflict_resolution
backends
bare_repositories
bugs add: When adding a dotfile, avoid treating its name as an extension. 2021-08-03 12:22:58 -04:00
chunking
coding_style
contribute update 2021-03-29 13:58:27 -04:00
copies
design deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
devblog remove old closed bugs and todo items to speed up wiki updates and reduce size 2021-06-21 13:10:13 -04:00
direct_mode
download
encryption Added a comment 2021-04-18 15:40:55 +00:00
footer
forum Added a comment 2021-08-03 15:14:53 +00:00
future_proofing
git-annex-add
git-annex-addurl
git-annex-adjust
git-annex-checkpresentkey
git-annex-config
git-annex-copy
git-annex-direct
git-annex-drop
git-annex-enableremote
git-annex-find
git-annex-import
git-annex-importfeed
git-annex-lookupkey
git-annex-metadata
git-annex-move
git-annex-p2p
git-annex-preferred-content
git-annex-reinject
git-annex-rmurl
git-annex-setpresentkey
git-annex-sync
git-annex-test
git-annex-unannex comment 2021-05-27 12:23:57 -04:00
git-annex-undo
git-annex-uninit
git-annex-unlock Added a comment: re: clarifying unlocked files 2021-04-21 16:34:50 +00:00
git-annex-unused fix sequence 2021-03-11 12:30:02 -04:00
git-annex-whereis
how_it_works
install link to new catalina build 2021-07-14 12:26:13 -04:00
internals Added annex.freezecontent-command and annex.thawcontent-command configs 2021-06-21 14:40:52 -04:00
license
links
metadata
news add news item for git-annex 8.20210803 2021-08-03 12:21:10 -04:00
not
polls
preferred_content
profiling
projects
required_content
scalability
security
special_remotes removed 2021-06-08 18:26:30 +00:00
stickers
submodules
sync
templates
testimonials
thanks update 2021-07-05 09:37:26 -04:00
tips 2021-07-06 19:50:36 +00:00
todo update 2021-08-03 14:03:25 -04:00
trust
tuning
upgrades
use_case
users removed 2021-05-18 17:16:12 +00:00
videos
walkthrough
workflow
Android.mdwn
assistant.mdwn
automatic_conflict_resolution.mdwn
backends.mdwn use GIT keys for export of non-annexed files 2021-03-05 14:12:11 -04:00
bare_repositories.mdwn
bugs.mdwn fix name 2021-07-15 12:56:44 -04:00
builds.mdwn update for autobuilder changes 2021-07-02 13:03:19 -04:00
chunking.mdwn
coding_style.mdwn
comments.mdwn
contact.mdwn
contribute.mdwn
copies.mdwn
design.mdwn
devblog.mdwn
direct_mode.mdwn
distributed_version_control.mdwn
download.mdwn
encryption.mdwn
favicon.ico
forum.mdwn rename tag page 2021-07-15 12:55:05 -04:00
future_proofing.mdwn
git-annex-add.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-addunused.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-addurl.mdwn addurl, importfeed: Added --no-raw option 2021-06-27 11:14:51 -04:00
git-annex-adjust.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-assistant.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-benchmark.mdwn
git-annex-calckey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-checkpresentkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-common-options.mdwn --size-limit exit 101 2021-06-04 16:43:47 -04:00
git-annex-config.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-contentlocation.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-copy.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-dead.mdwn fix innacuracy 2021-06-25 14:48:36 -04:00
git-annex-describe.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-diffdriver.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-direct.mdwn
git-annex-drop.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-dropkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-dropunused.mdwn dropping unused marks as dead 2021-06-25 15:22:26 -04:00
git-annex-edit.mdwn
git-annex-enable-tor.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-enableremote.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-examinekey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-expire.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-export.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-filter-branch.mdwn decided not to include export/import trees 2021-05-17 14:12:15 -04:00
git-annex-find.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-findref.mdwn
git-annex-fix.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-forget.mdwn git-annex-filter-branch man page 2021-05-13 16:17:45 -04:00
git-annex-fromkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-fsck.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-fuzztest.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-get.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-group.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-groupwanted.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-import.mdwn sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
git-annex-importfeed.mdwn addurl, importfeed: Added --no-raw option 2021-06-27 11:14:51 -04:00
git-annex-indirect.mdwn
git-annex-info.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-init.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-initremote.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-inprogress.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-list.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-lock.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-log.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-lookupkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-map.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-matchexpression.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-matching-expression.mdwn
git-annex-matching-options.mdwn New matching options --excludesamecontent and --includesamecontent 2021-05-25 13:08:18 -04:00
git-annex-merge.mdwn sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
git-annex-metadata.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-migrate.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-mincopies.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-mirror.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-move.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-multicast.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-numcopies.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-p2p.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-post-receive.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-pre-commit.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-preferred-content.mdwn
git-annex-proxy.mdwn
git-annex-readpresentkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-registerurl.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-reinit.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-reinject.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-rekey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-remotedaemon.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-renameremote.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-repair.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-required.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-resolvemerge.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-rmurl.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-schedule.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-semitrust.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-setkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-setpresentkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-shell.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-smudge.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-status.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-sync.mdwn sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
git-annex-test.mdwn
git-annex-testremote.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-transferkey.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-transferkeys.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-transferrer.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-trust.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-unannex.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-undo.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-ungroup.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-uninit.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-unlock.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-unregisterurl.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-untrust.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-unused.mdwn link 2021-07-14 16:47:21 -04:00
git-annex-upgrade.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-vadd.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-vcycle.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-version.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-vfilter.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-vicfg.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-view.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-vpop.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-wanted.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-watch.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-webapp.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-whereis.mdwn split out common options to its own page and mention it on each subcommand page 2021-05-10 15:00:13 -04:00
git-annex-whereused.mdwn wording 2021-07-14 17:08:38 -04:00
git-annex.mdwn deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
git-remote-tor-annex.mdwn
git-union-merge.mdwn
how_it_works.mdwn
index.mdwn
install.mdwn git-annex is in openbsd ports 2021-03-22 14:37:06 -04:00
internals.mdwn
license.mdwn
location_tracking.mdwn
logo-old-bw.svg
logo-old.png
logo-old.svg
logo-old_small.png
logo.mdwn
logo.svg
logo_16x16.png
logo_32x32.png
logo_small.png
meta.mdwn
metadata.mdwn
news.mdwn
not.mdwn
polls.mdwn
preferred_content.mdwn
privacy.mdwn
profiling.mdwn
projects.mdwn
publicrepos.mdwn
related_software.mdwn annex-review-unused -> git annex reviewunused 2021-02-28 20:20:03 +00:00
repomap.png
required_content.mdwn
scalability.mdwn
security.mdwn
shortcuts.mdwn
sidebar.mdwn
sitemap.mdwn
special_remotes.mdwn
stickers.mdwn
submodules.mdwn
summary.mdwn
sync.mdwn
testimonials.mdwn
thanks.mdwn
tips.mdwn
todo.mdwn
transferring_data.mdwn
trust.mdwn
tuning.mdwn
upgrades.mdwn
users.mdwn
videos.mdwn
walkthrough.mdwn
workflow.mdwn