mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/Utility
History
Joey Hess ed0afbc36b
avoid concurrent threads trying to take pid lock at same time 
Seem there are several races that happen when 2 threads run PidLock.tryLock
at the same time. One involves checkSaneLock of the side lock file, which may
be deleted by another process that is dropping the lock, causing checkSaneLock
to fail. And even with the deletion disabled, it can still fail, Probably due
to linkToLock failing when a second thread overwrites the lock file.

The same can happen when 2 processes do, but then one process just fails
to take the lock, which is fine. But with 2 threads, some actions where failing
even though the process as a whole had the pid lock held.

Utility.LockPool.PidLock already maintains a STM lock, and since it uses
LockShared, 2 threads can hold the pidlock at the same time, and when
the first thread drops the lock, it will remain held by the second
thread, and so the pid lock file should not get deleted until the last
thread to hold it drops the lock. Which is the right behavior, and why a
LockShared STM lock is used in the first place.

The problem is that each time it takes the STM lock, it then also calls
PidLock.tryLock. So that was getting called repeatedly and concurrently.

Fixed by noticing when the shared lock is already held, and stop calling
PidLock.tryLock again, just use the pid lock that already exists then.

Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed
to take the lock, which was entirely wrong. It should only drop the side
lock.

Sponsored-by: Dartmouth College's Datalad project
2021-12-01 17:14:39 -04:00
..
Directory remove unused imports 2020-11-02 15:36:11 -04:00
DirWatcher fix build on openbsd 2021-02-01 11:53:31 -04:00
Env fix a annex.pidlock issue 2020-06-17 15:30:59 -04:00
LockFile avoid concurrent threads trying to take pid lock at same time 2021-12-01 17:14:39 -04:00
LockPool avoid concurrent threads trying to take pid lock at same time 2021-12-01 17:14:39 -04:00
Path Fix using lookupkey inside a subdirectory 2021-10-26 15:00:45 -04:00
Process fix missing new lines in processTranscript 2021-08-02 13:42:27 -04:00
Scheduled explict export lists 2019-11-23 11:24:10 -04:00
Tmp explict export lists 2019-11-23 11:24:10 -04:00
Aeson.hs switch MetaValue to ByteString and MetaField to Text 2019-01-07 14:18:24 -04:00
Android.hs explict export lists 2019-11-21 16:08:37 -04:00
Applicative.hs squelch tab warnings 2019-11-22 12:49:41 -04:00
Attoparsec.hs fix git ls-tree parser 2019-12-06 14:05:48 -04:00
AuthToken.hs comment typo 2019-01-03 00:22:05 -04:00
Base64.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
Batch.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Bloom.hs remove many old version ifdefs 2019-07-05 15:09:37 -04:00
CoProcess.hs Revert "async exception safety for coprocesses" 2020-07-06 15:11:28 -04:00
CopyFile.hs note about coreutils 9.0 supporting CoW by default 2021-09-30 14:12:58 -04:00
Daemon.hs set cwd rarher than changing current process directory 2021-05-12 17:44:22 -04:00
Data.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
DataUnits.hs
DBus.hs explict export lists 2019-11-21 16:08:37 -04:00
Debug.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
DebugLocks.hs explict export lists 2019-11-21 16:08:37 -04:00
Directory.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
DirWatcher.hs explict export lists 2019-11-21 16:08:37 -04:00
DiskFree.hs
Dot.hs explict export lists 2019-11-21 16:08:37 -04:00
DottedVersion.hs more RawFilePath conversion 2020-10-29 12:03:50 -04:00
Env.hs explict export lists 2019-11-21 16:08:37 -04:00
Exception.hs comment typo 2021-01-21 14:13:55 -04:00
FileMode.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
FileSize.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
FileSystemEncoding.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Format.hs export encode_c' 2020-12-09 15:28:45 -04:00
FreeDesktop.hs
GitLFS.hs git-lfs gitlab interoperability fix 2021-11-10 13:51:11 -04:00
Glob.hs Windows: include= and exclude= containing '/' will also match filenames that are written using '\' 2020-12-15 12:39:34 -04:00
Gpg.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Hash.hs factor out IncrementalHasher from IncrementalVerifier 2021-11-09 12:33:22 -04:00
HtmlDetect.hs addurl: Avoid crashing when used on beegfs. 2021-07-05 13:02:40 -04:00
HttpManagerRestricted.hs Build with the http-client-restricted library when available 2020-06-22 11:31:31 -04:00
HumanNumber.hs Fix a rounding bug in display of data sizes 2021-07-30 09:56:04 -04:00
HumanTime.hs avoid parseDuration succeeding on empty string 2020-12-08 12:51:56 -04:00
InodeCache.hs fix build failure on windows 2021-10-20 15:12:12 -04:00
IPAddress.hs annex.security.allowed-ip-addresses ports syntax 2020-02-25 15:45:52 -04:00
libkqueue.c
libkqueue.h
LinuxMkLibs.hs remove obsolete libgcc1 2021-10-21 03:02:16 -04:00
LockFile.hs
LockPool.hs
LogFile.hs explict export lists 2019-11-21 16:08:37 -04:00
Lsof.hs convert to withCreateProcess for async exception safety 2020-06-04 12:44:09 -04:00
MagicWormhole.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Matcher.hs fix handling of implicit and before parens 2021-01-28 13:51:07 -04:00
MD5.hs use filepath-bytestring for annex object manipulations 2019-12-11 15:25:07 -04:00
Metered.hs avoid cursor jitter when updating progress display 2021-10-07 11:16:41 -04:00
Misc.hs ByteString Ref continued 2020-04-07 11:54:27 -04:00
Monad.hs explict export lists 2019-11-21 16:08:37 -04:00
Mounts.hs
MoveFile.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
Network.hs explict export lists 2019-11-21 16:08:37 -04:00
NotificationBroadcaster.hs
OptParse.hs explict export lists 2019-11-21 16:08:37 -04:00
OSX.hs explict export lists 2019-11-21 16:08:37 -04:00
PartialPrelude.hs explict export lists 2019-11-21 16:08:37 -04:00
Path.hs Fix using lookupkey inside a subdirectory 2021-10-26 15:00:45 -04:00
Percentage.hs
PID.hs explict export lists 2019-11-21 16:08:37 -04:00
Process.hs switch from hslogger to purpose-built Utility.Debug 2021-04-05 13:40:31 -04:00
QuickCheck.hs fix build warning 2021-01-13 14:48:41 -04:00
RawFilePath.hs squash build warnings on windows 2020-11-23 14:00:17 -04:00
ResourcePool.hs check-attr resource pool 2020-04-21 11:05:57 -04:00
Rsync.hs Parse youtube-dl progress output 2020-09-29 17:53:48 -04:00
SafeCommand.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
Scheduled.hs add dropFromEnd 2018-11-23 11:24:05 -04:00
Shell.hs add searchPathContents 2021-02-02 19:06:15 -04:00
ShellEscape.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
SimpleProtocol.hs minor improvements 2020-12-09 15:28:11 -04:00
Split.hs reword comment 2019-11-27 16:38:18 -04:00
SshConfig.hs more RawFilePath conversion 2020-11-05 18:45:37 -04:00
SshHost.hs
Su.hs add searchPathContents 2021-02-02 19:06:15 -04:00
SystemDirectory.hs
Tense.hs explict export lists 2019-11-23 11:24:10 -04:00
ThreadLock.hs explict export lists 2019-11-23 11:24:10 -04:00
ThreadScheduler.hs avoid displaying the scanning annexed files message when repo is not large 2021-06-04 13:16:48 -04:00
TimeStamp.hs explict export lists 2019-11-23 11:24:10 -04:00
TList.hs further change to support dlist-1.0 2020-08-05 10:37:14 -04:00
Tmp.hs decorate openTempFile errors with the template name 2021-08-30 13:05:02 -04:00
Tor.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Touch.hs fix build on windows 2020-11-16 09:31:45 -04:00
Tuple.hs explict export lists 2019-11-23 11:24:10 -04:00
Url.hs factor out IncrementalHasher from IncrementalVerifier 2021-11-09 12:33:22 -04:00
UserInfo.hs
Verifiable.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
WebApp.hs squash remaining build warnings on windows 2020-11-24 12:35:09 -04:00
Yesod.hs remove many old version ifdefs 2019-07-05 15:09:37 -04:00