pending adds now retried for kqueue

Rethought how to keep track of pending adds that need to be retried later.
The commit thread already run up every second when there are changes,
so let's keep pending adds queued as changes until they're safe to add.

Also, the committer is now smarter about avoiding empty commits when
all the adds are currently unsafe, or in the rare case that an add event
for a symlink is not received in time. It may avoid them entirely.

This seems to work as before for inotify, and is untested for kqueue.

(Actually commit batching seems to be improved for inotify, although I'm
not sure why. I'm seeing only two commits made during large batch
operations, and the first of those is the non-batch mode commit.)
This commit is contained in:
Joey Hess 2012-06-20 19:04:16 -04:00
parent e0fdfb2e70
commit 33b914bcf1
5 changed files with 135 additions and 107 deletions

View file

@ -7,20 +7,26 @@ module Assistant.Changes where
import Common.Annex
import qualified Annex.Queue
import Types.KeySource
import Control.Concurrent.STM
import Data.Time.Clock
data ChangeType = PendingAddChange | LinkChange | RmChange | RmDirChange
data ChangeType = AddChange | LinkChange | RmChange | RmDirChange
deriving (Show, Eq)
type ChangeChan = TChan Change
data Change = Change
{ changeTime :: UTCTime
, changeFile :: FilePath
, changeType :: ChangeType
}
data Change
= Change
{ changeTime :: UTCTime
, changeFile :: FilePath
, changeType :: ChangeType
}
| PendingAddChange
{ changeTime ::UTCTime
, keySource :: KeySource
}
deriving (Show)
runChangeChan :: STM a -> IO a
@ -33,13 +39,29 @@ newChangeChan = atomically newTChan
madeChange :: FilePath -> ChangeType -> Annex (Maybe Change)
madeChange f t = do
-- Just in case the commit thread is not flushing the queue fast enough.
when (t /= PendingAddChange) $
Annex.Queue.flushWhenFull
Annex.Queue.flushWhenFull
liftIO $ Just <$> (Change <$> getCurrentTime <*> pure f <*> pure t)
noChange :: Annex (Maybe Change)
noChange = return Nothing
{- Indicates an add is in progress. -}
pendingAddChange :: KeySource -> Annex (Maybe Change)
pendingAddChange ks =
liftIO $ Just <$> (PendingAddChange <$> getCurrentTime <*> pure ks)
isPendingAddChange :: Change -> Bool
isPendingAddChange (PendingAddChange {}) = True
isPendingAddChange _ = False
finishedChange :: Change -> Change
finishedChange c@(PendingAddChange { keySource = ks }) = Change
{ changeTime = changeTime c
, changeFile = keyFilename ks
, changeType = AddChange
}
finishedChange c = c
{- Gets all unhandled changes.
- Blocks until at least one change is made. -}
getChanges :: ChangeChan -> IO [Change]