git-annex/doc/todo/lockContent_for_special_remotes.mdwn

Support lockContent in more special remotes. This allows dropping
content from a special remote when the only copies are in other special
remotes, without needing to set mincopies to 0 and risk losing content.

(All the easy ones, eg read-only special remotes, are implemented already.)

* It might be possible for an external remote to lock content, but it would
  be a tricky extension to the protocol, since lockContent needs to keep it
  locked while running another action. There would need to be separate
  actions for locking and unlocking.

  If this were implemented in git-annex, and some special remote program
  didn't used to support it, and implemented REMOVE w/o checking a lock,
  then making that program support lockContent would run the risk
  of a mixture of the old and new version being in use for the same remote,
  which could result in data loss.

  To avoid that, the author of the special remote would need to either
  a) always do lock checking from the beginning or
  b) wait long enough or document well enough to be sure that situation
  never happens.

  > Maybe this second part is not a problem actually? Analysis follows:
  > 
  > There are 2 git-annex
  > instances A and B, and 2 special remotes X and Y. A is using the old
  > program that does not support locking, B uses the new program that does.
  > Both X and Y have a copy of an object.
  > 
  > If A is dropping from X, it will not trust Y to satisfy mincopies,
  > since A cannot lockContent Y. So it will not drop from X.
  > 
  > If B is dropping from Y, it can lockContent X, and so the drop
  > succeeds.
  > 
  > It seems it's ok for B to trust lockContent X, even though A does
  > not check if X is locked when dropping from it. Because A will not
  > drop from X unless it's able to satisfy mincopies by locking the
  > content somewhere else. --[[Joey]]

* directory could use fcntl locking

  This would need a transition, because dropping from directory first needs
  to support checking for locks. Only after we're sure that any git-annex
  binaries that might be in use on a machine do that is it safe for
  lockContent to take the lock. So git-annex would need to be released with
  the first change, then wait several months or years before making the
  second change.

  > Update: See analysis above, should also apply here.

  Also, the directory might be on a filesystem that does not support
  locking, with various failure modes. And unlike a git-annex repo,
  there's nowhere in a directory special remote to record information about
  locking problems with it. Getting this right seems hard..

  > If lockContent throws an exception, that remote is treated the same
  > as if it did not support lockContent, so if these problems can be
  > differentiated from a failure to lock at all, can just throw an
  > exception. But need to avoid lockContent succeeding in any case
  > where removeKey will not be prevented from removing the content.

* S3 has an object lock feature, I don't know if it would be usable for
  this.

  It would need a transition, with dropKey first failing when the object
  lock was in place, and then once that git-annex was in use everywhere,
  lockContent setting the object lock.
  
  > Update: See analysis above, should also apply here.
  
  (S3 with versioning=yes already supports lockContent.)

* adb could use some shell trick perhaps.. But it would depend on doing
  locking in /sdcard, which seems likely to be a massive ball of POSIX
  incompliance and pain.