mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Utility
History
Joey Hess ef3ab0769e
close pid lock only once no threads use it 
This fixes a FD leak when annex.pidlock is set and -J is used. Also, it
fixes bugs where the pid lock file got deleted because one thread was
done with it, while another thread was still holding it open.

The LockPool now has two distinct types of resources,
one is per-LockHandle and is used for file Handles, which get closed
when the associated LockHandle is closed. The other one is per lock
file, and gets closed when no more LockHandles use that lock file,
including other shared locks of the same file.

That latter kind is used for the pid lock file, so it's opened by the
first thread to use a lock, and closed when the last thread closes a lock.

In practice, this means that eg git-annex get of several files opens and
closes the pidlock file a few times per file. While with -J5 it will open
the pidlock file, process a number of files, until all the threads happen to
finish together, at which point the pidlock file gets closed, and then
that repeats. So in either case, another process still gets a chance to
take the pidlock.

registerPostRelease has a rather intricate dance, there are fine-grained
STM locks, a STM lock of the pidfile itself, and the actual pidlock file
on disk that are all resolved in stages by it.

Sponsored-by: Dartmouth College's Datalad project
2021-12-06 15:01:39 -04:00
..
Directory remove unused imports 2020-11-02 15:36:11 -04:00
DirWatcher fix build on openbsd 2021-02-01 11:53:31 -04:00
Env fix a annex.pidlock issue 2020-06-17 15:30:59 -04:00
LockFile close pid lock only once no threads use it 2021-12-06 15:01:39 -04:00
LockPool close pid lock only once no threads use it 2021-12-06 15:01:39 -04:00
Path Fix using lookupkey inside a subdirectory 2021-10-26 15:00:45 -04:00
Process fix missing new lines in processTranscript 2021-08-02 13:42:27 -04:00
Scheduled explict export lists 2019-11-23 11:24:10 -04:00
Tmp explict export lists 2019-11-23 11:24:10 -04:00
Aeson.hs switch MetaValue to ByteString and MetaField to Text 2019-01-07 14:18:24 -04:00
Android.hs explict export lists 2019-11-21 16:08:37 -04:00
Applicative.hs squelch tab warnings 2019-11-22 12:49:41 -04:00
Attoparsec.hs fix git ls-tree parser 2019-12-06 14:05:48 -04:00
AuthToken.hs comment typo 2019-01-03 00:22:05 -04:00
Base64.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
Batch.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Bloom.hs remove many old version ifdefs 2019-07-05 15:09:37 -04:00
CoProcess.hs Revert "async exception safety for coprocesses" 2020-07-06 15:11:28 -04:00
CopyFile.hs note about coreutils 9.0 supporting CoW by default 2021-09-30 14:12:58 -04:00
Daemon.hs set cwd rarher than changing current process directory 2021-05-12 17:44:22 -04:00
Data.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
DataUnits.hs
DBus.hs explict export lists 2019-11-21 16:08:37 -04:00
Debug.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
DebugLocks.hs explict export lists 2019-11-21 16:08:37 -04:00
Directory.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
DirWatcher.hs explict export lists 2019-11-21 16:08:37 -04:00
DiskFree.hs removed the old Android app 2018-10-13 01:41:11 -04:00
Dot.hs explict export lists 2019-11-21 16:08:37 -04:00
DottedVersion.hs more RawFilePath conversion 2020-10-29 12:03:50 -04:00
Env.hs explict export lists 2019-11-21 16:08:37 -04:00
Exception.hs comment typo 2021-01-21 14:13:55 -04:00
FileMode.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
FileSize.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
FileSystemEncoding.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Format.hs export encode_c' 2020-12-09 15:28:45 -04:00
FreeDesktop.hs
GitLFS.hs git-lfs gitlab interoperability fix 2021-11-10 13:51:11 -04:00
Glob.hs Windows: include= and exclude= containing '/' will also match filenames that are written using '\' 2020-12-15 12:39:34 -04:00
Gpg.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Hash.hs factor out IncrementalHasher from IncrementalVerifier 2021-11-09 12:33:22 -04:00
HtmlDetect.hs addurl: Avoid crashing when used on beegfs. 2021-07-05 13:02:40 -04:00
HttpManagerRestricted.hs Build with the http-client-restricted library when available 2020-06-22 11:31:31 -04:00
HumanNumber.hs Fix a rounding bug in display of data sizes 2021-07-30 09:56:04 -04:00
HumanTime.hs avoid parseDuration succeeding on empty string 2020-12-08 12:51:56 -04:00
InodeCache.hs fix build failure on windows 2021-10-20 15:12:12 -04:00
IPAddress.hs annex.security.allowed-ip-addresses ports syntax 2020-02-25 15:45:52 -04:00
libkqueue.c
libkqueue.h
LinuxMkLibs.hs remove obsolete libgcc1 2021-10-21 03:02:16 -04:00
LockFile.hs
LockPool.hs
LogFile.hs explict export lists 2019-11-21 16:08:37 -04:00
Lsof.hs convert to withCreateProcess for async exception safety 2020-06-04 12:44:09 -04:00
MagicWormhole.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Matcher.hs fix handling of implicit and before parens 2021-01-28 13:51:07 -04:00
MD5.hs use filepath-bytestring for annex object manipulations 2019-12-11 15:25:07 -04:00
Metered.hs avoid cursor jitter when updating progress display 2021-10-07 11:16:41 -04:00
Misc.hs ByteString Ref continued 2020-04-07 11:54:27 -04:00
Monad.hs explict export lists 2019-11-21 16:08:37 -04:00
Mounts.hs removed the old Android app 2018-10-13 01:41:11 -04:00
MoveFile.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
Network.hs explict export lists 2019-11-21 16:08:37 -04:00
NotificationBroadcaster.hs
OptParse.hs explict export lists 2019-11-21 16:08:37 -04:00
OSX.hs explict export lists 2019-11-21 16:08:37 -04:00
PartialPrelude.hs explict export lists 2019-11-21 16:08:37 -04:00
Path.hs Fix using lookupkey inside a subdirectory 2021-10-26 15:00:45 -04:00
Percentage.hs
PID.hs explict export lists 2019-11-21 16:08:37 -04:00
Process.hs switch from hslogger to purpose-built Utility.Debug 2021-04-05 13:40:31 -04:00
QuickCheck.hs fix build warning 2021-01-13 14:48:41 -04:00
RawFilePath.hs squash build warnings on windows 2020-11-23 14:00:17 -04:00
ResourcePool.hs check-attr resource pool 2020-04-21 11:05:57 -04:00
Rsync.hs Parse youtube-dl progress output 2020-09-29 17:53:48 -04:00
SafeCommand.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
Scheduled.hs add dropFromEnd 2018-11-23 11:24:05 -04:00
Shell.hs add searchPathContents 2021-02-02 19:06:15 -04:00
ShellEscape.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
SimpleProtocol.hs minor improvements 2020-12-09 15:28:11 -04:00
Split.hs reword comment 2019-11-27 16:38:18 -04:00
SshConfig.hs more RawFilePath conversion 2020-11-05 18:45:37 -04:00
SshHost.hs
Su.hs add searchPathContents 2021-02-02 19:06:15 -04:00
SystemDirectory.hs
Tense.hs explict export lists 2019-11-23 11:24:10 -04:00
ThreadLock.hs explict export lists 2019-11-23 11:24:10 -04:00
ThreadScheduler.hs avoid displaying the scanning annexed files message when repo is not large 2021-06-04 13:16:48 -04:00
TimeStamp.hs explict export lists 2019-11-23 11:24:10 -04:00
TList.hs further change to support dlist-1.0 2020-08-05 10:37:14 -04:00
Tmp.hs decorate openTempFile errors with the template name 2021-08-30 13:05:02 -04:00
Tor.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Touch.hs fix build on windows 2020-11-16 09:31:45 -04:00
Tuple.hs explict export lists 2019-11-23 11:24:10 -04:00
Url.hs factor out IncrementalHasher from IncrementalVerifier 2021-11-09 12:33:22 -04:00
UserInfo.hs removed the old Android app 2018-10-13 01:41:11 -04:00
Verifiable.hs add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
WebApp.hs squash remaining build warnings on windows 2020-11-24 12:35:09 -04:00
Yesod.hs remove many old version ifdefs 2019-07-05 15:09:37 -04:00