blog for the day
This commit is contained in:
parent
625330ae78
commit
6873ca0c1b
2 changed files with 50 additions and 3 deletions
|
@ -0,0 +1,36 @@
|
|||
Today, added a thread that deals with recovering when there's been a loss
|
||||
of network connectivity. When the network's down, the normal immediate
|
||||
syncing of changes of course doesn't work. So this thread detects when the
|
||||
network comes back up, and does a pull+push to network remotes, and
|
||||
triggers scanning for file content that needs to be transferred.
|
||||
|
||||
I used dbus again, to detect events generated by both network-manager and
|
||||
wicd when they've sucessfully brought an interface up. Or, if they're not
|
||||
available, it polls every 30 minutes.
|
||||
|
||||
When the network comes up, in addition to the git pull+push, it also
|
||||
currently does a full scan of the repo to find files whose contents
|
||||
need to be transferred to get fully back into sync.
|
||||
|
||||
I think it'll be ok for some git pulls and pushes to happen when
|
||||
moving to a new network, or resuming a laptop (or every 30 minutes when
|
||||
resorting to polling). But the transfer scan is currently really too heavy
|
||||
to be appropriate to do every time in those situations. I have an idea for
|
||||
avoiding that scan when the remote's git-annex branch has not changed. But
|
||||
I need to refine it, to handle cases like this:
|
||||
|
||||
1. a new remote is added
|
||||
2. file contents start being transferred to (or from it)
|
||||
3. the network is taken down
|
||||
4. all the queued transfers fail
|
||||
5. the network comes back up
|
||||
6. the transfer scan needs to know the remote was not all in sync
|
||||
before #3, and so should do a full scan despite the git-annex branch
|
||||
not having changed
|
||||
|
||||
---
|
||||
|
||||
Doubled the ram in my netbook, which I use for all development. Yesod needs
|
||||
rather a lot of ram to compile and link, and this should make me quite a
|
||||
lot more productive. I was struggling with OOM killing bits of chromium
|
||||
during my last week of development.
|
|
@ -3,9 +3,15 @@ all the other git clones, at both the git level and the key/value level.
|
|||
|
||||
## immediate action items
|
||||
|
||||
* At startup, and possibly periodically, or when the network connection
|
||||
changes, or some heuristic suggests that a remote was disconnected from
|
||||
us for a while, queue remotes for processing by the TransferScanner.
|
||||
* Sync with all available remotes on startup.
|
||||
* TransferScanner should avoid unnecessary scanning of remotes.
|
||||
This is paricilarly important for scans queued by the NetWatcher,
|
||||
which can be polling, or could be after a momentary blip in network
|
||||
connectivity. The TransferScanner could check the remote's git-annex
|
||||
branch; if it is not ahead of the local git-annex branch, then
|
||||
there's nothing to transfer. **Except** if the tree was not already
|
||||
up-to-date before the loss of connectivity. So doing this needs
|
||||
tracking of when the tree is not yet fully up-to-date.
|
||||
* Ensure that when a remote receives content, and updates its location log,
|
||||
it syncs that update back out. Prerequisite for:
|
||||
* After git sync, identify new content that we don't have that is now available
|
||||
|
@ -157,3 +163,8 @@ redone to check it.
|
|||
finishes. **done**
|
||||
* Test MountWatcher on KDE, and add whatever dbus events KDE emits when
|
||||
drives are mounted. **done**
|
||||
* Possibly periodically, or when the network connection
|
||||
changes, or some heuristic suggests that a remote was disconnected from
|
||||
us for a while, queue remotes for processing by the TransferScanner.
|
||||
**done**; both network-manager and wicd connection events are supported,
|
||||
and it falls back to polling every 30 minutes when neither is available.
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue