update
This commit is contained in:
parent
d60a33fd13
commit
6660984442
1 changed files with 23 additions and 16 deletions
|
@ -79,7 +79,7 @@ Planned schedule of work:
|
||||||
|
|
||||||
Make checking the balanced preferred content limit record a
|
Make checking the balanced preferred content limit record a
|
||||||
live update in the table and use other live updates in making its
|
live update in the table and use other live updates in making its
|
||||||
decision. With locking as necessary.
|
decision. (done)
|
||||||
|
|
||||||
Note: This will only work when preferred content is being checked.
|
Note: This will only work when preferred content is being checked.
|
||||||
If a git-annex copy without --auto is run, for example, it won't
|
If a git-annex copy without --auto is run, for example, it won't
|
||||||
|
@ -87,6 +87,10 @@ Planned schedule of work:
|
||||||
That seems ok though, because if the user is running a command like
|
That seems ok though, because if the user is running a command like
|
||||||
that, they are ok with a remote filling up.
|
that, they are ok with a remote filling up.
|
||||||
|
|
||||||
|
Make sure that two threads don't check balanced preferred content at the
|
||||||
|
same time, so each thread always sees a consistent picture of what is
|
||||||
|
happening. Use locking as necessary.
|
||||||
|
|
||||||
In the unlikely event that one thread of a process is storing a key and
|
In the unlikely event that one thread of a process is storing a key and
|
||||||
another thread is dropping the same key from the same uuid, at the same
|
another thread is dropping the same key from the same uuid, at the same
|
||||||
time, reconcile somehow. How? Or is this perhaps something that cannot
|
time, reconcile somehow. How? Or is this perhaps something that cannot
|
||||||
|
@ -98,23 +102,14 @@ Planned schedule of work:
|
||||||
|
|
||||||
Make updating location log for a key that is in the in-memory cache
|
Make updating location log for a key that is in the in-memory cache
|
||||||
of the live update table update the db, removing it from that table,
|
of the live update table update the db, removing it from that table,
|
||||||
and updating the in-memory reposizes. This needs to have
|
and updating the in-memory reposizes. (done)
|
||||||
locking to make sure redundant information is never visible:
|
|
||||||
|
Make updading location log have locking to make sure redundant
|
||||||
|
information is never visible:
|
||||||
Take lock, journal update, remove from live update table.
|
Take lock, journal update, remove from live update table.
|
||||||
|
|
||||||
Somehow detect when an upload (or drop) fails, and remove from the live
|
Detect when an upload (or drop) fails, and remove from the live
|
||||||
update table and in-memory cache. How? Possibly have a thread that
|
update table and in-memory cache. (done)
|
||||||
waits on an empty MVar. Thread MVar through somehow to location log
|
|
||||||
update. (Seems this would need checking preferred content to return
|
|
||||||
the MVar? Or alternatively, the MVar could be passed into it, which
|
|
||||||
seems better..) Fill MVar on location log update. If MVar gets
|
|
||||||
GCed without being filled, the thread will get an exception and can
|
|
||||||
remove from table and cache then. This does rely on GC behavior, but if
|
|
||||||
the GC takes some time, it will just cause a failed upload to take
|
|
||||||
longer to get removed from the table and cache, which will just prevent
|
|
||||||
another upload of a different key from running immediately.
|
|
||||||
(Need to check if MVar GC behavior operates like this.
|
|
||||||
See https://stackoverflow.com/questions/10871303/killing-a-thread-when-mvar-is-garbage-collected )
|
|
||||||
|
|
||||||
Have a counter in the reposizes table that is updated on write. This
|
Have a counter in the reposizes table that is updated on write. This
|
||||||
can be used to quickly determine if it has changed. On every check of
|
can be used to quickly determine if it has changed. On every check of
|
||||||
|
@ -135,6 +130,18 @@ Planned schedule of work:
|
||||||
But then, how to check if a PID is git-annex or not? /proc of course,
|
But then, how to check if a PID is git-annex or not? /proc of course,
|
||||||
but what about other OS's? Windows?
|
but what about other OS's? Windows?
|
||||||
|
|
||||||
|
How? Possibly have a thread that
|
||||||
|
waits on an empty MVar. Thread MVar through somehow to location log
|
||||||
|
update. (Seems this would need checking preferred content to return
|
||||||
|
the MVar? Or alternatively, the MVar could be passed into it, which
|
||||||
|
seems better..) Fill MVar on location log update. If MVar gets
|
||||||
|
GCed without being filled, the thread will get an exception and can
|
||||||
|
remove from table and cache then. This does rely on GC behavior, but if
|
||||||
|
the GC takes some time, it will just cause a failed upload to take
|
||||||
|
longer to get removed from the table and cache, which will just prevent
|
||||||
|
another upload of a different key from running immediately.
|
||||||
|
(Need to check if MVar GC behavior operates like this.
|
||||||
|
See https://stackoverflow.com/questions/10871303/killing-a-thread-when-mvar-is-garbage-collected )
|
||||||
Perhaps stale entries can be found in a different way. Require the live
|
Perhaps stale entries can be found in a different way. Require the live
|
||||||
update table to be updated with a timestamp every 5 minutes. The thread
|
update table to be updated with a timestamp every 5 minutes. The thread
|
||||||
that waits on the MVar can do that, as long as the transfer is running. If
|
that waits on the MVar can do that, as long as the transfer is running. If
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue