45 lines
2.1 KiB
Text
45 lines
2.1 KiB
Text
|
There's something rotten in POSIX fctnl locking. It's not composable,
|
||
|
|
or thread-safe.
|
||
|
|
|
||
|
|
The most obvious problem with it is that if you have 2 threads, and they
|
||
|
|
both try to take an exclusive lock of the same file (each opening it
|
||
|
|
separately) ... They'll both succeed. Unlike 2 separate processes,
|
||
|
|
where only one can take the lock.
|
||
|
|
|
||
|
|
Then the really crazy bit: If a process has a lock file open and fcntl
|
||
|
|
locked, and then the same process opens the lock file again, for any
|
||
|
|
reason, closing the new FD will release the lock that was set
|
||
|
|
using the other FD.
|
||
|
|
|
||
|
|
So, that's a massive gotcha if you're writing complex multithreaded code.
|
||
|
|
Or generally for composition of code. Of course, C programmers deal with
|
||
|
|
this kind of thing all the time, but in the clean world of Haskell, this is
|
||
|
|
a glaring problem. We don't expect to need to worry about this kind of
|
||
|
|
unrelated side effect that breaks composition and thread safety.
|
||
|
|
|
||
|
|
After noticing this problem affected git-anenx in at least one place,
|
||
|
|
I have to assume there could be more. And I don't want to need to worry
|
||
|
|
about this problem forever. So, I have been working today on a clean fix
|
||
|
|
that I can cleanly switch all my lock-related code to use.
|
||
|
|
|
||
|
|
One reasonable approach would be to avoid fcntl locking, and use flock.
|
||
|
|
But, flock works even less well on NFS than fcntl, and git-annex relies on
|
||
|
|
some fcntl locking features. On Linux, there's an "open file description
|
||
|
|
locks" feature that fixes POSIX fnctl locking to not have this horrible
|
||
|
|
wart, but that's not portable.
|
||
|
|
|
||
|
|
Instead, my approach is to keep track of which files the process has
|
||
|
|
locked. If it tries to do something with a lockfile that it already has
|
||
|
|
locked, it avoids opening the same file again, instead implements its own
|
||
|
|
in-process locking behavior. I use STM to do that in a thread-safe manner.
|
||
|
|
|
||
|
|
I should probably break out git-annex's lock file handling code as a
|
||
|
|
library. Eventually.. This was about as much fun as a root canal, and I'm
|
||
|
|
having a real one tomorrow. :-/
|
||
|
|
|
||
|
|
----
|
||
|
|
|
||
|
|
git-annex is now included in [Stackage](http://www.stackage.org/)!
|
||
|
|
|
||
|
|
Daniel Kahn Gillmor is doing some work on reproducible builds of git-annex.
|