mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/Annex
History
Joey Hess db89e39df6
partially fix concurrency issue in updating the rollingtotal 
It's possible for two processes or threads to both be doing the same
operation at the same time. Eg, both dropping the same key. If one
finishes and updates the rollingtotal, then the other one needs to be
prevented from later updating the rollingtotal as well. And they could
finish at the same time, or with some time in between.

Addressed this by making updateRepoSize be called with the journal
locked, and only once it's been determined that there is an actual
location change to record in the log. updateRepoSize waits for the
database to be updated.

When there is a redundant operation, updateRepoSize won't be called,
and the redundant LiveUpdate will be removed from the database on
garbage collection.

But: There will be a window where the redundant LiveUpdate is still
visible in the db, and processes can see it, combine it with the
rollingtotal, and arrive at the wrong size. This is a small window, but
it still ought to be addressed. Unsure if it would always be safe to
remove the redundant LiveUpdate? Consider the case where two drops and a
get are all running concurrently somehow, and the order they finish is
[drop, get, drop]. The second drop seems redundant to the first, but
it would not be safe to remove it. While this seems unlikely, it's hard
to rule out that a get and drop at different stages can both be running
at the same time.
2024-08-26 09:43:32 -04:00
..
AdjustedBranch multiple -m second try 2024-04-09 12:56:47 -04:00
Branch newtype MapLog 2023-11-13 14:37:22 -04:00
Concurrent differentiate between concurrency enabled at command line and by git config 2020-09-16 11:47:12 -04:00
Content disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
Debug implement fastDebug 2021-04-06 15:24:28 -04:00
LockPool avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
MetaData sqlite datbase for importfeed 2023-10-23 16:46:22 -04:00
RepoSize partially fix concurrency issue in updating the rollingtotal 2024-08-26 09:43:32 -04:00
SpecialRemote proxied exporttree=yes versionedexport=yes remotes are not untrusted 2024-08-08 15:24:19 -04:00
VectorClock Lower precision of timestamps in git-annex branch 2023-12-11 15:04:06 -04:00
View annex.maxextensions configuration 2024-04-18 14:23:38 -04:00
Action.hs remove dead nodes when loading the cluster log 2024-06-16 14:39:44 -04:00
AdjustedBranch.hs adjust unlocked execute bit handling 2024-05-28 12:39:42 -04:00
AutoMerge.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
Balanced.hs implement fullbalanced=group:N 2024-08-20 13:51:02 -04:00
BloomFilter.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Branch.hs partially fix concurrency issue in updating the rollingtotal 2024-08-26 09:43:32 -04:00
BranchState.hs smarter BranchState cache invalidation 2024-07-28 12:33:32 -04:00
CatFile.hs read a consistent amount from pointer file 2022-02-23 12:52:34 -04:00
ChangedRefs.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
CheckAttr.hs config: Added the --show-origin and --for-file options 2023-06-12 16:24:31 -04:00
CheckIgnore.hs move several readonly values to AnnexRead 2022-06-28 15:40:19 -04:00
Cluster.hs LiveUpdate for clusters 2024-08-24 10:20:12 -04:00
Common.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
Concurrent.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Content.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
CopyFile.hs RawFilePath conversion 2024-01-19 14:26:21 -04:00
CurrentBranch.hs refactor getCurrentBranch 2018-10-19 17:29:18 -04:00
Debug.hs display explanations in --debug too 2023-07-31 13:06:40 -04:00
Difference.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
DirHashes.hs Added http special remote, which is useful for accessing other remotes that publish content stored in them via http/https. 2020-09-01 15:16:35 -04:00
Drop.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
Environment.hs improve comments 2023-04-04 15:23:39 -04:00
Export.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
ExternalAddonProcess.hs startExternalAddonProcess add parameters 2024-04-17 13:09:10 -04:00
FileMatcher.hs improve live update starting 2024-08-24 13:07:05 -04:00
Fixup.hs fix a bug that prevented git-annex init from working in a submodule 2021-01-21 15:33:15 -04:00
GitOverlay.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
HashObject.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Hook.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Import.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
Ingest.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
Init.hs remove dead nodes when loading the cluster log 2024-06-16 14:39:44 -04:00
InodeSentinal.hs fix perms for core.sharedRepository 2023-04-26 16:29:11 -04:00
Journal.hs share single BranchState amoung all threads 2024-07-28 12:30:27 -04:00
Link.hs fix hang when built with unix-2.8 2023-08-01 20:22:28 -04:00
Locations.hs finalize RepoSize database 2024-08-15 12:29:34 -04:00
LockFile.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
LockPool.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Magic.hs Serialize use of C magic library, which is not thread safe. 2020-09-17 17:27:42 -04:00
MetaData.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Multicast.hs use programPath consistently, not readProgramFile 2020-03-30 16:06:27 -04:00
Notification.hs fix build when dbus is enabled 2022-07-05 13:06:45 -04:00
NumCopies.hs toward SafeDropProof expiry checking 2024-07-04 12:39:06 -04:00
Path.hs Propagate --force to git-annex transferrer 2024-07-23 21:16:56 -04:00
Perms.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
PidLock.hs fix windows build 2022-09-26 12:08:04 -04:00
Proxy.hs LiveUpdate for clusters 2024-08-24 10:20:12 -04:00
Queue.hs add restage log 2022-09-23 15:47:24 -04:00
RemoteTrackingBranch.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
ReplaceFile.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
RepoSize.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
SafeDropProof.hs use REMOVE-BEFORE in P2P protocol 2024-07-04 13:47:38 -04:00
SpecialRemote.hs init: Avoid autoenabling special remotes that have control characters in their names 2023-04-12 12:37:12 -04:00
Ssh.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
StallDetection.hs add directional stalldetection and bwlimit configs 2024-01-19 15:27:53 -04:00
Startup.hs Avoid loading cluster log at startup 2024-07-31 15:54:14 -04:00
TaggedPush.hs simplify base64 to only use ByteString 2023-10-26 13:10:05 -04:00
Tmp.hs fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Transfer.hs add optional object file location to storeKey 2024-07-01 10:42:27 -04:00
TransferrerPool.hs avoid build warning on windows 2023-03-27 12:19:26 -04:00
UntrustedFilePath.hs fix mojibake reversion in display of utf8 2023-04-12 13:53:30 -04:00
UpdateInstead.hs v7 for all repositories 2019-08-30 14:09:14 -04:00
Url.hs avoid unncessary use of curl when conduit will do 2023-08-22 10:25:53 -04:00
UUID.hs proxying GET now working 2024-06-11 15:09:43 -04:00
VariantFile.hs more RawFilePath 2019-12-18 17:10:28 -04:00
VectorClock.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Verify.hs When proxying an upload to a special remote, verify the hash. 2024-07-29 13:40:51 -04:00
Version.hs v8 repositories automatically upgrade to v9 2022-07-25 16:20:04 -04:00
View.hs annex.maxextensions configuration 2024-04-18 14:23:38 -04:00
Wanted.hs plumb in LiveUpdate (WIP) 2024-08-23 16:35:12 -04:00
WorkerPool.hs add Annex worker pool to P2PHttp 2024-07-10 12:19:47 -04:00
WorkTree.hs use lookupKeyStaged in --batch code paths 2022-10-26 14:43:06 -04:00
YoutubeDl.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00