2017-08-14 17:55:38 +00:00
|
|
|
{- git-annex vector clocks
|
|
|
|
-
|
deal better with clock skew situations, using vector clocks
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.
When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.
When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.
(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)
Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.
A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.
Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.
Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.
Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
|
|
|
- These are basically a timestamp. However, when logging a new
|
|
|
|
- value, if the old value has a vector clock that is the same or greater
|
|
|
|
- than the current vector clock, the old vector clock is incremented.
|
|
|
|
- This way, clock skew does not cause confusion.
|
2017-08-14 17:55:38 +00:00
|
|
|
-
|
deal better with clock skew situations, using vector clocks
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.
When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.
When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.
(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)
Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.
A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.
Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.
Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.
Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
|
|
|
- Copyright 2017-2021 Joey Hess <id@joeyh.name>
|
2017-08-14 17:55:38 +00:00
|
|
|
-
|
2019-03-13 19:48:14 +00:00
|
|
|
- Licensed under the GNU AGPL version 3 or higher.
|
2017-08-14 17:55:38 +00:00
|
|
|
-}
|
|
|
|
|
2020-12-23 19:21:33 +00:00
|
|
|
module Annex.VectorClock (
|
|
|
|
module Annex.VectorClock,
|
|
|
|
module Types.VectorClock,
|
|
|
|
) where
|
2017-08-14 17:55:38 +00:00
|
|
|
|
2020-12-23 19:21:33 +00:00
|
|
|
import Types.VectorClock
|
|
|
|
import Annex.Common
|
|
|
|
import qualified Annex
|
2018-10-30 03:13:36 +00:00
|
|
|
import Utility.TimeStamp
|
2020-12-23 19:21:33 +00:00
|
|
|
|
|
|
|
import Data.ByteString.Builder
|
2019-01-03 19:27:29 +00:00
|
|
|
import qualified Data.Attoparsec.ByteString.Lazy as A
|
2017-08-14 17:55:38 +00:00
|
|
|
|
deal better with clock skew situations, using vector clocks
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.
When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.
When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.
(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)
Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.
A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.
Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.
Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.
Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
|
|
|
currentVectorClock :: Annex CandidateVectorClock
|
2020-12-23 19:21:33 +00:00
|
|
|
currentVectorClock = liftIO =<< Annex.getState Annex.getvectorclock
|
2017-08-14 17:55:38 +00:00
|
|
|
|
deal better with clock skew situations, using vector clocks
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.
When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.
When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.
(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)
Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.
A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.
Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.
Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.
Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
|
|
|
-- Runs the action and uses the same vector clock throughout,
|
|
|
|
-- except when it's necessary to use a newer one due to a past value having
|
|
|
|
-- a newer vector clock.
|
2020-12-23 19:21:33 +00:00
|
|
|
--
|
|
|
|
-- When the action modifies several files in the git-annex branch,
|
|
|
|
-- this can cause less space to be used, since the same vector clock
|
|
|
|
-- value is used, which can compress better.
|
|
|
|
--
|
|
|
|
-- However, this should not be used when running a long-duration action,
|
|
|
|
-- because the vector clock is based on the start of the action, and not on
|
|
|
|
-- the later points where it writes changes. For example, if this were
|
|
|
|
-- used across downloads of several files, the location log information
|
|
|
|
-- would have an earlier vector clock than necessary, which might cause it
|
|
|
|
-- to be disregarded in favor of other information that was collected
|
|
|
|
-- at an earlier point in time than when the transfers completted and the
|
|
|
|
-- log was written.
|
|
|
|
reuseVectorClockWhile :: Annex a -> Annex a
|
|
|
|
reuseVectorClockWhile = bracket setup cleanup . const
|
|
|
|
where
|
|
|
|
setup = do
|
|
|
|
origget <- Annex.getState Annex.getvectorclock
|
|
|
|
vc <- liftIO origget
|
|
|
|
use (pure vc)
|
|
|
|
return origget
|
2017-08-14 17:55:38 +00:00
|
|
|
|
2020-12-23 19:21:33 +00:00
|
|
|
cleanup origget = use origget
|
2017-08-14 17:55:38 +00:00
|
|
|
|
2020-12-23 19:21:33 +00:00
|
|
|
use vc = Annex.changeState $ \s ->
|
|
|
|
s { Annex.getvectorclock = vc }
|
2017-08-14 18:43:56 +00:00
|
|
|
|
deal better with clock skew situations, using vector clocks
* Deal with clock skew, both forwards and backwards, when logging
information to the git-annex branch.
* GIT_ANNEX_VECTOR_CLOCK can now be set to a fixed value (eg 1)
rather than needing to be advanced each time a new change is made.
* Misuse of GIT_ANNEX_VECTOR_CLOCK will no longer confuse git-annex.
When changing a file in the git-annex branch, the vector clock to use is now
determined by first looking at the current time (or GIT_ANNEX_VECTOR_CLOCK
when set), and comparing it to the newest vector clock already in use in
that file. If a newer time stamp was already in use, advance it forward by
a second instead.
When the clock is set to a time in the past, this avoids logging with
an old timestamp, which would risk that log line later being ignored in favor
of "newer" line that is really not newer.
When a log entry has been made with a clock that was set far ahead in the
future, this avoids newer information being logged with an older timestamp
and so being ignored in favor of that future-timestamped information.
Once all clocks get fixed, this will result in the vector clocks being
incremented, until finally enough time has passed that time gets back ahead
of the vector clock value, and then it will return to usual operation.
(This latter situation is not ideal, but it seems the best that can be done.
The issue with it is, since all writers will be incrementing the last
vector clock they saw, there's no way to tell when one writer made a write
significantly later in time than another, so the earlier write might
arbitrarily be picked when merging. This problem is why git-annex uses
timestamps in the first place, rather than pure vector clocks.)
Advancing forward by 1 second is somewhat arbitrary. setDead
advances a timestamp by just 1 picosecond, and the vector clock could
too. But then it would interfere with setDead, which wants to be
overrulled by any change. So it could use 2 picoseconds or something,
but that seems weird. It could just as well advance it forward by a
minute or whatever, but then it would be harder for real time to catch
up with the vector clock when forward clock slew had happened.
A complication is that many log files contain several different peices of
information, and it may be best to only use vector clocks for the same peice
of information. For example, a key's location log file contains
InfoPresent/InfoMissing for each UUID, and it only looks at the vector
clocks for the UUID that is being changed, and not other UUIDs.
Although exactly where the dividing line is can be hard to determine.
Consider metadata logs, where a field "tag" can have multiple values set
at different times. Should it advance forward past the last tag?
Probably. What about when a different field is set, should it look at
the clocks of other fields? Perhaps not, but currently it does, and
this does not seems like it will cause any problems.
Another one I'm not entirely sure about is the export log, which is
keyed by (fromuuid, touuid). So if multiple repos are exporting to the
same remote, different vector clocks can be used for that remote.
It looks like that's probably ok, because it does not try to determine
what order things occurred when there was an export conflict.
Sponsored-by: Jochen Bartl on Patreon
2021-08-03 20:45:20 +00:00
|
|
|
-- Convert a candidate vector clock in to the final one to use,
|
|
|
|
-- advancing it if necessary when necessary to get ahead of a previously
|
|
|
|
-- used vector clock.
|
|
|
|
advanceVectorClock :: CandidateVectorClock -> [VectorClock] -> VectorClock
|
|
|
|
advanceVectorClock (CandidateVectorClock c) [] = VectorClock c
|
|
|
|
advanceVectorClock (CandidateVectorClock c) prevs
|
|
|
|
| prev >= VectorClock c = case prev of
|
|
|
|
VectorClock v -> VectorClock (v + 1)
|
|
|
|
Unknown -> VectorClock c
|
|
|
|
| otherwise = VectorClock c
|
|
|
|
where
|
|
|
|
prev = maximum prevs
|
|
|
|
|
2017-08-14 18:43:56 +00:00
|
|
|
formatVectorClock :: VectorClock -> String
|
2019-01-09 17:06:37 +00:00
|
|
|
formatVectorClock Unknown = "0"
|
2017-08-14 18:43:56 +00:00
|
|
|
formatVectorClock (VectorClock t) = show t
|
|
|
|
|
2019-01-09 17:06:37 +00:00
|
|
|
buildVectorClock :: VectorClock -> Builder
|
2019-01-09 18:17:00 +00:00
|
|
|
buildVectorClock = string7 . formatVectorClock
|
2019-01-09 17:06:37 +00:00
|
|
|
|
2017-08-14 18:43:56 +00:00
|
|
|
parseVectorClock :: String -> Maybe VectorClock
|
|
|
|
parseVectorClock t = VectorClock <$> parsePOSIXTime t
|
2019-01-03 19:27:29 +00:00
|
|
|
|
|
|
|
vectorClockParser :: A.Parser VectorClock
|
|
|
|
vectorClockParser = VectorClock <$> parserPOSIXTime
|