git-annex/Utility/LockPool/PidLock.hs

160 lines
5 KiB
Haskell
Raw Normal View History

2015-11-12 20:31:34 +00:00
{- Pid locks, using lock pools.
-
avoid concurrent threads trying to take pid lock at same time Seem there are several races that happen when 2 threads run PidLock.tryLock at the same time. One involves checkSaneLock of the side lock file, which may be deleted by another process that is dropping the lock, causing checkSaneLock to fail. And even with the deletion disabled, it can still fail, Probably due to linkToLock failing when a second thread overwrites the lock file. The same can happen when 2 processes do, but then one process just fails to take the lock, which is fine. But with 2 threads, some actions where failing even though the process as a whole had the pid lock held. Utility.LockPool.PidLock already maintains a STM lock, and since it uses LockShared, 2 threads can hold the pidlock at the same time, and when the first thread drops the lock, it will remain held by the second thread, and so the pid lock file should not get deleted until the last thread to hold it drops the lock. Which is the right behavior, and why a LockShared STM lock is used in the first place. The problem is that each time it takes the STM lock, it then also calls PidLock.tryLock. So that was getting called repeatedly and concurrently. Fixed by noticing when the shared lock is already held, and stop calling PidLock.tryLock again, just use the pid lock that already exists then. Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed to take the lock, which was entirely wrong. It should only drop the side lock. Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
- Copyright 2015-2021 Joey Hess <id@joeyh.name>
2015-11-12 20:31:34 +00:00
-
- License: BSD-2-clause
-}
module Utility.LockPool.PidLock (
P.LockFile,
LockHandle,
waitLock,
tryLock,
tryLock',
2015-11-12 20:31:34 +00:00
checkLocked,
getLockStatus,
LockStatus(..),
dropLock,
checkSaneLock,
) where
import qualified Utility.LockFile.PidLock as F
import Utility.LockFile.LockStatus
import qualified Utility.LockPool.STM as P
import Utility.LockPool.STM (LockFile, LockMode(..))
import Utility.LockPool.LockHandle
2015-11-12 21:12:54 +00:00
import Utility.ThreadScheduler
2015-11-12 20:31:34 +00:00
import System.IO
import System.Posix
avoid concurrent threads trying to take pid lock at same time Seem there are several races that happen when 2 threads run PidLock.tryLock at the same time. One involves checkSaneLock of the side lock file, which may be deleted by another process that is dropping the lock, causing checkSaneLock to fail. And even with the deletion disabled, it can still fail, Probably due to linkToLock failing when a second thread overwrites the lock file. The same can happen when 2 processes do, but then one process just fails to take the lock, which is fine. But with 2 threads, some actions where failing even though the process as a whole had the pid lock held. Utility.LockPool.PidLock already maintains a STM lock, and since it uses LockShared, 2 threads can hold the pidlock at the same time, and when the first thread drops the lock, it will remain held by the second thread, and so the pid lock file should not get deleted until the last thread to hold it drops the lock. Which is the right behavior, and why a LockShared STM lock is used in the first place. The problem is that each time it takes the STM lock, it then also calls PidLock.tryLock. So that was getting called repeatedly and concurrently. Fixed by noticing when the shared lock is already held, and stop calling PidLock.tryLock again, just use the pid lock that already exists then. Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed to take the lock, which was entirely wrong. It should only drop the side lock. Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
import Control.Concurrent.STM
2015-11-12 20:31:34 +00:00
import Data.Maybe
avoid concurrent threads trying to take pid lock at same time Seem there are several races that happen when 2 threads run PidLock.tryLock at the same time. One involves checkSaneLock of the side lock file, which may be deleted by another process that is dropping the lock, causing checkSaneLock to fail. And even with the deletion disabled, it can still fail, Probably due to linkToLock failing when a second thread overwrites the lock file. The same can happen when 2 processes do, but then one process just fails to take the lock, which is fine. But with 2 threads, some actions where failing even though the process as a whole had the pid lock held. Utility.LockPool.PidLock already maintains a STM lock, and since it uses LockShared, 2 threads can hold the pidlock at the same time, and when the first thread drops the lock, it will remain held by the second thread, and so the pid lock file should not get deleted until the last thread to hold it drops the lock. Which is the right behavior, and why a LockShared STM lock is used in the first place. The problem is that each time it takes the STM lock, it then also calls PidLock.tryLock. So that was getting called repeatedly and concurrently. Fixed by noticing when the shared lock is already held, and stop calling PidLock.tryLock again, just use the pid lock that already exists then. Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed to take the lock, which was entirely wrong. It should only drop the side lock. Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
import Control.Monad
import Control.Monad.Catch
import Control.Monad.IO.Class
2015-11-12 20:31:34 +00:00
import Control.Applicative
import Prelude
-- Does locking using a pid lock, blocking until the lock is available
-- or the Seconds timeout if the pid lock is held by another process.
--
-- There are two levels of locks. A STM lock is used to handle
-- fine-grained locking amoung threads, locking a specific lockfile,
-- but only in memory. The pid lock handles locking between processes.
--
-- The pid lock is only taken once, and LockShared is used for it,
-- so multiple threads can have it locked. Only the first thread
-- will create the pid lock, and it remains until all threads drop
-- their locks.
waitLock
:: (MonadIO m, MonadMask m)
=> LockFile
-> LockMode
-> Seconds
-> F.PidLockFile
-> (String -> m ())
-> m LockHandle
waitLock finelockfile lockmode timeout pidlockfile displaymessage = do
fl <- takefinelock
pl <- takepidlock
`onException` liftIO (dropLock fl)
registerPostRelease fl pl
return fl
where
takefinelock = fst <$> makeLockHandle P.lockPool finelockfile
(\p f -> P.waitTakeLock p f lockmode)
(\_ _ -> pure (stmonlyflo, ()))
-- A shared STM lock is taken for each use of the pid lock,
-- but only the first thread to take it actually creates the pid
-- lock file.
takepidlock = makeLockHandle P.lockPool pidlockfile
(\p f -> P.waitTakeLock p f LockShared)
(\f (P.FirstLock firstlock firstlocksem) -> if firstlock
then waitlock f firstlocksem
else liftIO (atomically $ readTMVar firstlocksem) >>= \case
P.FirstLockSemWaited True -> alreadylocked f
P.FirstLockSemTried True -> alreadylocked f
P.FirstLockSemWaited False -> F.waitedLock timeout f displaymessage
P.FirstLockSemTried False -> waitlock f firstlocksem
)
waitlock f firstlocksem = do
h <- F.waitLock timeout f displaymessage $
void . atomically . tryPutTMVar firstlocksem . P.FirstLockSemWaited
return (mkflo h, Just h)
alreadylocked f = do
lh <- F.alreadyLocked f
return (mkflo lh, Nothing)
registerPostRelease :: MonadIO m => LockHandle -> (LockHandle, Maybe F.LockHandle) -> m ()
registerPostRelease (LockHandle flh _) (pl@(LockHandle plh _), mpidlock) = do
-- After the fine-grained lock gets dropped (and any shared locks
-- of it are also dropped), drop the associated pid lock.
liftIO $ atomically $
P.registerPostReleaseLock flh (dropLock pl)
-- When the last thread to use the pid lock has dropped it,
-- close the pid lock file itself.
case mpidlock of
Just pidlock -> liftIO $ atomically $
P.registerPostReleaseLock plh (F.dropLock pidlock)
Nothing -> return ()
2015-11-12 20:31:34 +00:00
-- Tries to take a pid lock, but does not block.
tryLock :: LockFile -> LockMode -> F.PidLockFile -> IO (Maybe LockHandle)
tryLock finelockfile lockmode pidlockfile = takefinelock >>= \case
Just fl -> tryLock' pidlockfile >>= \case
Just pl -> do
registerPostRelease fl pl
return (Just fl)
Nothing -> do
dropLock fl
return Nothing
Nothing -> return Nothing
where
takefinelock = fmap fst <$> tryMakeLockHandle P.lockPool finelockfile
(\p f -> P.tryTakeLock p f lockmode)
(\_ _ -> pure (Just (stmonlyflo, ())))
tryLock' :: F.PidLockFile -> IO (Maybe (LockHandle, Maybe F.LockHandle))
tryLock' pidlockfile = tryMakeLockHandle P.lockPool pidlockfile
(\p f -> P.tryTakeLock p f LockShared)
(\f (P.FirstLock firstlock firstlocksem) -> if firstlock
then do
mlh <- F.tryLock f
void $ atomically $ tryPutTMVar firstlocksem
(P.FirstLockSemTried (isJust mlh))
case mlh of
Just lh -> return (Just (mkflo lh, Just lh))
Nothing -> return Nothing
else liftIO (atomically $ readTMVar firstlocksem) >>= \case
P.FirstLockSemWaited True -> alreadylocked f
P.FirstLockSemTried True -> alreadylocked f
P.FirstLockSemWaited False -> return Nothing
P.FirstLockSemTried False -> return Nothing
avoid concurrent threads trying to take pid lock at same time Seem there are several races that happen when 2 threads run PidLock.tryLock at the same time. One involves checkSaneLock of the side lock file, which may be deleted by another process that is dropping the lock, causing checkSaneLock to fail. And even with the deletion disabled, it can still fail, Probably due to linkToLock failing when a second thread overwrites the lock file. The same can happen when 2 processes do, but then one process just fails to take the lock, which is fine. But with 2 threads, some actions where failing even though the process as a whole had the pid lock held. Utility.LockPool.PidLock already maintains a STM lock, and since it uses LockShared, 2 threads can hold the pidlock at the same time, and when the first thread drops the lock, it will remain held by the second thread, and so the pid lock file should not get deleted until the last thread to hold it drops the lock. Which is the right behavior, and why a LockShared STM lock is used in the first place. The problem is that each time it takes the STM lock, it then also calls PidLock.tryLock. So that was getting called repeatedly and concurrently. Fixed by noticing when the shared lock is already held, and stop calling PidLock.tryLock again, just use the pid lock that already exists then. Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed to take the lock, which was entirely wrong. It should only drop the side lock. Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
)
where
alreadylocked f = do
lh <- F.alreadyLocked f
return (Just (mkflo lh, Nothing))
2015-11-12 20:31:34 +00:00
checkLocked :: LockFile -> IO (Maybe Bool)
checkLocked file = P.getLockStatus P.lockPool file
(pure (Just True))
(F.checkLocked file)
getLockStatus :: LockFile -> IO LockStatus
getLockStatus file = P.getLockStatus P.lockPool file
(StatusLockedBy <$> getProcessID)
(F.getLockStatus file)
mkflo :: F.LockHandle -> FileLockOps
mkflo h = FileLockOps
{ fDropLock = return ()
2015-11-12 20:31:34 +00:00
, fCheckSaneLock = \f -> F.checkSaneLock f h
}
stmonlyflo :: FileLockOps
stmonlyflo = FileLockOps
{ fDropLock = return ()
, fCheckSaneLock = const (return True)
}