2015-11-12 20:31:34 +00:00
|
|
|
{- Pid locks, using lock pools.
|
|
|
|
-
|
avoid concurrent threads trying to take pid lock at same time
Seem there are several races that happen when 2 threads run PidLock.tryLock
at the same time. One involves checkSaneLock of the side lock file, which may
be deleted by another process that is dropping the lock, causing checkSaneLock
to fail. And even with the deletion disabled, it can still fail, Probably due
to linkToLock failing when a second thread overwrites the lock file.
The same can happen when 2 processes do, but then one process just fails
to take the lock, which is fine. But with 2 threads, some actions where failing
even though the process as a whole had the pid lock held.
Utility.LockPool.PidLock already maintains a STM lock, and since it uses
LockShared, 2 threads can hold the pidlock at the same time, and when
the first thread drops the lock, it will remain held by the second
thread, and so the pid lock file should not get deleted until the last
thread to hold it drops the lock. Which is the right behavior, and why a
LockShared STM lock is used in the first place.
The problem is that each time it takes the STM lock, it then also calls
PidLock.tryLock. So that was getting called repeatedly and concurrently.
Fixed by noticing when the shared lock is already held, and stop calling
PidLock.tryLock again, just use the pid lock that already exists then.
Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed
to take the lock, which was entirely wrong. It should only drop the side
lock.
Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
|
|
|
- Copyright 2015-2021 Joey Hess <id@joeyh.name>
|
2015-11-12 20:31:34 +00:00
|
|
|
-
|
|
|
|
- License: BSD-2-clause
|
|
|
|
-}
|
|
|
|
|
|
|
|
module Utility.LockPool.PidLock (
|
|
|
|
P.LockFile,
|
|
|
|
LockHandle,
|
|
|
|
waitLock,
|
|
|
|
tryLock,
|
2021-12-03 21:20:21 +00:00
|
|
|
tryLock',
|
2015-11-12 20:31:34 +00:00
|
|
|
checkLocked,
|
|
|
|
getLockStatus,
|
|
|
|
LockStatus(..),
|
|
|
|
dropLock,
|
|
|
|
checkSaneLock,
|
|
|
|
) where
|
|
|
|
|
|
|
|
import qualified Utility.LockFile.PidLock as F
|
|
|
|
import Utility.LockFile.LockStatus
|
|
|
|
import qualified Utility.LockPool.STM as P
|
|
|
|
import Utility.LockPool.STM (LockFile, LockMode(..))
|
|
|
|
import Utility.LockPool.LockHandle
|
2015-11-12 21:12:54 +00:00
|
|
|
import Utility.ThreadScheduler
|
2015-11-12 20:31:34 +00:00
|
|
|
|
|
|
|
import System.IO
|
|
|
|
import System.Posix
|
avoid concurrent threads trying to take pid lock at same time
Seem there are several races that happen when 2 threads run PidLock.tryLock
at the same time. One involves checkSaneLock of the side lock file, which may
be deleted by another process that is dropping the lock, causing checkSaneLock
to fail. And even with the deletion disabled, it can still fail, Probably due
to linkToLock failing when a second thread overwrites the lock file.
The same can happen when 2 processes do, but then one process just fails
to take the lock, which is fine. But with 2 threads, some actions where failing
even though the process as a whole had the pid lock held.
Utility.LockPool.PidLock already maintains a STM lock, and since it uses
LockShared, 2 threads can hold the pidlock at the same time, and when
the first thread drops the lock, it will remain held by the second
thread, and so the pid lock file should not get deleted until the last
thread to hold it drops the lock. Which is the right behavior, and why a
LockShared STM lock is used in the first place.
The problem is that each time it takes the STM lock, it then also calls
PidLock.tryLock. So that was getting called repeatedly and concurrently.
Fixed by noticing when the shared lock is already held, and stop calling
PidLock.tryLock again, just use the pid lock that already exists then.
Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed
to take the lock, which was entirely wrong. It should only drop the side
lock.
Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
|
|
|
import Control.Concurrent.STM
|
2015-11-12 20:31:34 +00:00
|
|
|
import Data.Maybe
|
avoid concurrent threads trying to take pid lock at same time
Seem there are several races that happen when 2 threads run PidLock.tryLock
at the same time. One involves checkSaneLock of the side lock file, which may
be deleted by another process that is dropping the lock, causing checkSaneLock
to fail. And even with the deletion disabled, it can still fail, Probably due
to linkToLock failing when a second thread overwrites the lock file.
The same can happen when 2 processes do, but then one process just fails
to take the lock, which is fine. But with 2 threads, some actions where failing
even though the process as a whole had the pid lock held.
Utility.LockPool.PidLock already maintains a STM lock, and since it uses
LockShared, 2 threads can hold the pidlock at the same time, and when
the first thread drops the lock, it will remain held by the second
thread, and so the pid lock file should not get deleted until the last
thread to hold it drops the lock. Which is the right behavior, and why a
LockShared STM lock is used in the first place.
The problem is that each time it takes the STM lock, it then also calls
PidLock.tryLock. So that was getting called repeatedly and concurrently.
Fixed by noticing when the shared lock is already held, and stop calling
PidLock.tryLock again, just use the pid lock that already exists then.
Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed
to take the lock, which was entirely wrong. It should only drop the side
lock.
Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
|
|
|
import Control.Monad
|
2020-08-26 17:05:34 +00:00
|
|
|
import Control.Monad.Catch
|
|
|
|
import Control.Monad.IO.Class
|
2015-11-12 20:31:34 +00:00
|
|
|
import Control.Applicative
|
|
|
|
import Prelude
|
|
|
|
|
2021-12-03 21:20:21 +00:00
|
|
|
-- Does locking using a pid lock, blocking until the lock is available
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
-- or the Seconds timeout if the pid lock is held by another process.
|
2021-12-03 21:20:21 +00:00
|
|
|
--
|
|
|
|
-- There are two levels of locks. A STM lock is used to handle
|
|
|
|
-- fine-grained locking amoung threads, locking a specific lockfile,
|
|
|
|
-- but only in memory. The pid lock handles locking between processes.
|
|
|
|
--
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
-- The pid lock is only taken once, and LockShared is used for it,
|
|
|
|
-- so multiple threads can have it locked. Only the first thread
|
|
|
|
-- will create the pid lock, and it remains until all threads drop
|
|
|
|
-- their locks.
|
2020-08-26 17:05:34 +00:00
|
|
|
waitLock
|
|
|
|
:: (MonadIO m, MonadMask m)
|
2021-12-03 21:20:21 +00:00
|
|
|
=> LockFile
|
|
|
|
-> LockMode
|
|
|
|
-> Seconds
|
|
|
|
-> F.PidLockFile
|
2020-08-26 17:05:34 +00:00
|
|
|
-> (String -> m ())
|
|
|
|
-> m LockHandle
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
waitLock finelockfile lockmode timeout pidlockfile displaymessage = do
|
|
|
|
fl <- takefinelock
|
2021-12-03 21:20:21 +00:00
|
|
|
pl <- takepidlock
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
`onException` liftIO (dropLock fl)
|
|
|
|
registerPostRelease fl pl
|
|
|
|
return fl
|
2021-12-03 21:20:21 +00:00
|
|
|
where
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
takefinelock = fst <$> makeLockHandle P.lockPool finelockfile
|
2021-12-03 21:20:21 +00:00
|
|
|
(\p f -> P.waitTakeLock p f lockmode)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
(\_ _ -> pure (stmonlyflo, ()))
|
|
|
|
-- A shared STM lock is taken for each use of the pid lock,
|
|
|
|
-- but only the first thread to take it actually creates the pid
|
|
|
|
-- lock file.
|
2021-12-03 21:20:21 +00:00
|
|
|
takepidlock = makeLockHandle P.lockPool pidlockfile
|
|
|
|
(\p f -> P.waitTakeLock p f LockShared)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
(\f (P.FirstLock firstlock firstlocksem) -> if firstlock
|
|
|
|
then waitlock f firstlocksem
|
|
|
|
else liftIO (atomically $ readTMVar firstlocksem) >>= \case
|
|
|
|
P.FirstLockSemWaited True -> alreadylocked f
|
|
|
|
P.FirstLockSemTried True -> alreadylocked f
|
|
|
|
P.FirstLockSemWaited False -> F.waitedLock timeout f displaymessage
|
|
|
|
P.FirstLockSemTried False -> waitlock f firstlocksem
|
2021-12-03 21:20:21 +00:00
|
|
|
)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
waitlock f firstlocksem = do
|
|
|
|
h <- F.waitLock timeout f displaymessage $
|
|
|
|
void . atomically . tryPutTMVar firstlocksem . P.FirstLockSemWaited
|
|
|
|
return (mkflo h, Just h)
|
|
|
|
alreadylocked f = do
|
|
|
|
lh <- F.alreadyLocked f
|
|
|
|
return (mkflo lh, Nothing)
|
|
|
|
|
|
|
|
registerPostRelease :: MonadIO m => LockHandle -> (LockHandle, Maybe F.LockHandle) -> m ()
|
|
|
|
registerPostRelease (LockHandle flh _) (pl@(LockHandle plh _), mpidlock) = do
|
|
|
|
-- After the fine-grained lock gets dropped (and any shared locks
|
|
|
|
-- of it are also dropped), drop the associated pid lock.
|
|
|
|
liftIO $ atomically $
|
|
|
|
P.registerPostReleaseLock flh (dropLock pl)
|
|
|
|
-- When the last thread to use the pid lock has dropped it,
|
|
|
|
-- close the pid lock file itself.
|
|
|
|
case mpidlock of
|
|
|
|
Just pidlock -> liftIO $ atomically $
|
|
|
|
P.registerPostReleaseLock plh (F.dropLock pidlock)
|
|
|
|
Nothing -> return ()
|
2015-11-12 20:31:34 +00:00
|
|
|
|
|
|
|
-- Tries to take a pid lock, but does not block.
|
2021-12-03 21:20:21 +00:00
|
|
|
tryLock :: LockFile -> LockMode -> F.PidLockFile -> IO (Maybe LockHandle)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
tryLock finelockfile lockmode pidlockfile = takefinelock >>= \case
|
|
|
|
Just fl -> tryLock' pidlockfile >>= \case
|
2021-12-03 21:20:21 +00:00
|
|
|
Just pl -> do
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
registerPostRelease fl pl
|
|
|
|
return (Just fl)
|
2021-12-03 21:20:21 +00:00
|
|
|
Nothing -> do
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
dropLock fl
|
2021-12-03 21:20:21 +00:00
|
|
|
return Nothing
|
|
|
|
Nothing -> return Nothing
|
|
|
|
where
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
takefinelock = fmap fst <$> tryMakeLockHandle P.lockPool finelockfile
|
2021-12-03 21:20:21 +00:00
|
|
|
(\p f -> P.tryTakeLock p f lockmode)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
(\_ _ -> pure (Just (stmonlyflo, ())))
|
2021-12-03 21:20:21 +00:00
|
|
|
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
tryLock' :: F.PidLockFile -> IO (Maybe (LockHandle, Maybe F.LockHandle))
|
2021-12-03 21:20:21 +00:00
|
|
|
tryLock' pidlockfile = tryMakeLockHandle P.lockPool pidlockfile
|
Fix shared lock file FD leak.
This fixes behavior in this situation:
l1 <- lockShared Nothing "lck"
l2 <- lockShared Nothing "lck"
dropLock l1
dropLock l2
Before, the lock was dropped upon the second dropLock call, but the fd
remained open, and would never be closed while the program was running.
Fixed by a rather round-about method, but it should work well enough.
It would have been simpler to open open the shared lock once, and not open
it again in the second call to lockShared. But, that's difficult to do
atomically.
This also affects Windows and PID locks, not just posix locks.
In the case of pid locks, multiple calls to waitLock within the same
process are allowed because the side lock is locked using a posix lock,
and so multiple exclusive locks can be taken in the same process. So,
this change fixes a similar problem with pid locks.
l1 <- waitLock (Seconds 1) "lck"
l2 <- waitLock (Seconds 1) "lck"
dropLock l1
dropLock l2
Here the l2 side lock fd remained open but not locked,
although the pid lock file was removed. After this change, the second
dropLock will close both fds to the side lock, and delete the pidlock.
2016-03-01 19:31:39 +00:00
|
|
|
(\p f -> P.tryTakeLock p f LockShared)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
(\f (P.FirstLock firstlock firstlocksem) -> if firstlock
|
|
|
|
then do
|
|
|
|
mlh <- F.tryLock f
|
|
|
|
void $ atomically $ tryPutTMVar firstlocksem
|
|
|
|
(P.FirstLockSemTried (isJust mlh))
|
|
|
|
case mlh of
|
|
|
|
Just lh -> return (Just (mkflo lh, Just lh))
|
|
|
|
Nothing -> return Nothing
|
|
|
|
else liftIO (atomically $ readTMVar firstlocksem) >>= \case
|
|
|
|
P.FirstLockSemWaited True -> alreadylocked f
|
|
|
|
P.FirstLockSemTried True -> alreadylocked f
|
|
|
|
P.FirstLockSemWaited False -> return Nothing
|
|
|
|
P.FirstLockSemTried False -> return Nothing
|
avoid concurrent threads trying to take pid lock at same time
Seem there are several races that happen when 2 threads run PidLock.tryLock
at the same time. One involves checkSaneLock of the side lock file, which may
be deleted by another process that is dropping the lock, causing checkSaneLock
to fail. And even with the deletion disabled, it can still fail, Probably due
to linkToLock failing when a second thread overwrites the lock file.
The same can happen when 2 processes do, but then one process just fails
to take the lock, which is fine. But with 2 threads, some actions where failing
even though the process as a whole had the pid lock held.
Utility.LockPool.PidLock already maintains a STM lock, and since it uses
LockShared, 2 threads can hold the pidlock at the same time, and when
the first thread drops the lock, it will remain held by the second
thread, and so the pid lock file should not get deleted until the last
thread to hold it drops the lock. Which is the right behavior, and why a
LockShared STM lock is used in the first place.
The problem is that each time it takes the STM lock, it then also calls
PidLock.tryLock. So that was getting called repeatedly and concurrently.
Fixed by noticing when the shared lock is already held, and stop calling
PidLock.tryLock again, just use the pid lock that already exists then.
Also, LockFile.PidLock.tryLock was deleting the pid lock when it failed
to take the lock, which was entirely wrong. It should only drop the side
lock.
Sponsored-by: Dartmouth College's Datalad project
2021-12-01 19:22:31 +00:00
|
|
|
)
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
where
|
|
|
|
alreadylocked f = do
|
|
|
|
lh <- F.alreadyLocked f
|
|
|
|
return (Just (mkflo lh, Nothing))
|
2015-11-12 20:31:34 +00:00
|
|
|
|
|
|
|
checkLocked :: LockFile -> IO (Maybe Bool)
|
|
|
|
checkLocked file = P.getLockStatus P.lockPool file
|
|
|
|
(pure (Just True))
|
|
|
|
(F.checkLocked file)
|
|
|
|
|
|
|
|
getLockStatus :: LockFile -> IO LockStatus
|
|
|
|
getLockStatus file = P.getLockStatus P.lockPool file
|
|
|
|
(StatusLockedBy <$> getProcessID)
|
|
|
|
(F.getLockStatus file)
|
|
|
|
|
2021-12-03 21:20:21 +00:00
|
|
|
mkflo :: F.LockHandle -> FileLockOps
|
|
|
|
mkflo h = FileLockOps
|
close pid lock only once no threads use it
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.
The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.
That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.
In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.
registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.
Sponsored-by: Dartmouth College's Datalad project
2021-12-06 19:01:39 +00:00
|
|
|
{ fDropLock = return ()
|
2015-11-12 20:31:34 +00:00
|
|
|
, fCheckSaneLock = \f -> F.checkSaneLock f h
|
|
|
|
}
|
2021-12-03 21:20:21 +00:00
|
|
|
|
|
|
|
stmonlyflo :: FileLockOps
|
|
|
|
stmonlyflo = FileLockOps
|
|
|
|
{ fDropLock = return ()
|
|
|
|
, fCheckSaneLock = const (return True)
|
|
|
|
}
|