Detect when a preferred content expression contains "not present", which
would lead to repeatedly getting and then dropping files, and make it never
match. This also applies to "not balanced" and "not sizebalanced".
--explain will tell the user when this happens
Note that getMatcher calls matchMrun' and does not check for unstable
negated limits. While there is no --present anyway, if there was,
it would not make sense for --not --present to complain about
instability and fail to match.
This fixes the build on windows.
Changed it to use lock pools, which will behave better if two threads
call getLiveRepoSizes at the same time.
Also this should make it work when annex.pidlock is set. In that case,
once the current process locks this file, or anything, any other process
will have to wait on the pid lock. So checkStaleSizeChanges will
correctly identify any other live changes in the database as stale,
since there can only be one git-annex process running.
The test suite flagged that git-annex info in a readonly repository was
no longer working.
.git/annex/journal.lck: openFd: permission denied
This fixes it, however, in a case where .git/annex/reposize/ is
writable, but .git/annex/journal/ is not, there will still be a
permission denied error. The solution would just be to use consistent
permissions I suppose.
When a live update is removing a key, it might fail. So only count those
once they have succeeded. When a live update is adding a key, count it
immediately to avoid over-filling a repo.
This also makes the 1 minute delay between stale live changes checks
more defensible, because a stale live change can only cause us to err
more on the side of caution.
Reorganized the reposize database directory, and split up a column.
checkStaleSizeChanges needs to run before needLiveUpdate,
otherwise the process won't be holding a lock on its pid file, and
another process could go in and expire the live update it records. It
just so happens that they do get called in the correct order, since
checking balanced preferred content calls getLiveRepoSizes before
needLiveUpdate.
The 1 minute delay between checks is arbitrary, but will avoid excess
work. The downside of it is that, if a process is dropping a file and
gets interrupted, for 1 minute another process can expect a repository
will soon be smaller than it is. And so a process might send data to a
repository when a file is not really going to be dropped from it. But
note that can already happen if a drop takes some time in eg locking and
then fails. So it seems possible that live updates should only be
allowed to increase, rather than decrease the size of a repository.
Only when the preferred content expression being matched uses balanced
preferred content is this overhead needed.
It might be possible to eliminate the locking entirely. Eg, check the
live changes before and after the action and re-run if they are not
stable. For now, this is good enough, it avoids existing preferred
content getting slow. If balanced preferred content turns out to be too
slow to check, that could be tried later.
