git-annex/Logs
Joey Hess db89e39df6
partially fix concurrency issue in updating the rollingtotal
It's possible for two processes or threads to both be doing the same
operation at the same time. Eg, both dropping the same key. If one
finishes and updates the rollingtotal, then the other one needs to be
prevented from later updating the rollingtotal as well. And they could
finish at the same time, or with some time in between.

Addressed this by making updateRepoSize be called with the journal
locked, and only once it's been determined that there is an actual
location change to record in the log. updateRepoSize waits for the
database to be updated.

When there is a redundant operation, updateRepoSize won't be called,
and the redundant LiveUpdate will be removed from the database on
garbage collection.

But: There will be a window where the redundant LiveUpdate is still
visible in the db, and processes can see it, combine it with the
rollingtotal, and arrive at the wrong size. This is a small window, but
it still ought to be addressed. Unsure if it would always be safe to
remove the redundant LiveUpdate? Consider the case where two drops and a
get are all running concurrently somehow, and the order they finish is
[drop, get, drop]. The second drop seems redundant to the first, but
it would not be safe to remove it. While this seems unlikely, it's hard
to rule out that a get and drop at different stages can both be running
at the same time.
2024-08-26 09:43:32 -04:00
..
Chunk
Cluster don't sync with cluster nodes by default 2024-06-25 10:24:38 -04:00
ContentIdentifier simplify base64 to only use ByteString 2023-10-26 13:10:05 -04:00
Difference
Export Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
MetaData
PreferredContent start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
Presence migrate --apply 2023-12-08 13:23:46 -04:00
Remote Fix build with attoparsec-0.14 2021-03-24 12:11:50 -04:00
SingleValue deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Trust newtype MapLog 2023-11-13 14:37:22 -04:00
Activity.hs Future proof activity log parsing 2021-06-14 14:18:19 -04:00
AdjustedBranchUpdate.hs sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
Chunk.hs newtype MapLog 2023-11-13 14:37:22 -04:00
Cluster.hs Avoid loading cluster log at startup 2024-07-31 15:54:14 -04:00
Config.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
ContentIdentifier.hs partially fix concurrency issue in updating the rollingtotal 2024-08-26 09:43:32 -04:00
Difference.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
EquivilantKeys.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
Export.hs Merge branch 'master' into proxy 2024-06-10 14:26:18 -04:00
File.hs Windows: Fix CRLF handling in some log files 2023-10-30 14:23:23 -04:00
FsckResults.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Group.hs use hmac in balanced preferred content 2024-08-10 16:32:54 -04:00
Import.hs implement importChanges optimisaton (not used yet) 2023-05-31 16:01:34 -04:00
Line.hs Fix build with attoparsec-0.14 2021-03-24 12:11:50 -04:00
Location.hs partially fix concurrency issue in updating the rollingtotal 2024-08-26 09:43:32 -04:00
MapLog.hs newtype MapLog 2023-11-13 14:37:22 -04:00
MaxSize.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
MetaData.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Migrate.hs migrate --apply 2023-12-08 13:23:46 -04:00
Multicast.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
NumCopies.hs cache negative lookups of global numcopies and mincopies 2023-06-06 14:43:25 -04:00
PreferredContent.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
Presence.hs partially fix concurrency issue in updating the rollingtotal 2024-08-26 09:43:32 -04:00
Proxy.hs add git-annex updatecluster command 2024-06-14 15:02:01 -04:00
Remote.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
RemoteState.hs newtype MapLog 2023-11-13 14:37:22 -04:00
Restage.hs fix deadlock in restagePointerFiles 2022-12-08 14:36:11 -04:00
Schedule.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
SingleValue.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Smudge.hs add a finalizer to streamLogFile 2022-09-23 13:49:01 -04:00
Transfer.hs fix build on windows 2024-03-26 13:12:58 -04:00
Transitions.hs fix obviously wrong attoparsec parser 2022-02-07 14:15:17 -04:00
Trust.hs info: Added --autoenable option 2022-06-01 14:20:38 -04:00
Unused.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Upgrade.hs add annex.dbdir (WIP) 2022-08-11 16:58:53 -04:00
UUID.hs one-way escaping of newlines in uuid.log 2023-03-13 14:19:32 -04:00
UUIDBased.hs newtype MapLog 2023-11-13 14:37:22 -04:00
View.hs understand adjusted view branch names 2023-02-27 14:57:58 -04:00
Web.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00