read through all design docs and updated done items
This commit is contained in:
parent
714afb8ede
commit
6d5a84a4fc
6 changed files with 12 additions and 20 deletions
|
@ -2,7 +2,7 @@ The [[syncing]] design assumes the network is connected. But it's often
|
|||
not in these pre-IPV6 days, so the cloud needs to be used to bridge between
|
||||
LANS.
|
||||
|
||||
## The cloud notification problem **done**
|
||||
## The cloud notification problem (**done**)
|
||||
|
||||
Alice and Bob have repos, and there is a cloud remote they both share.
|
||||
Alice adds a file; the assistant transfers it to the cloud remote.
|
||||
|
|
|
@ -8,11 +8,11 @@ The meat of the configuration will be in configuration assistants that walk
|
|||
through setting up common use cases.
|
||||
|
||||
* Create a repository (run when the web app is started without a configured
|
||||
repository too).
|
||||
repository too). **done**
|
||||
* Clone this repo to a USB drive or other removable drive. **done**
|
||||
* Make a bare repo on a remote ssh server **done**
|
||||
* Clone this repo to another host. (Needs [[pairing]]) **done**
|
||||
* Set up Amazon S3.
|
||||
* Set up Amazon S3. **done**
|
||||
* Set up encrypted rsync remote. **done**
|
||||
* Rsync.net special case **done**
|
||||
* Set up gpg encryption key; gpg key distribution.
|
||||
|
|
|
@ -81,4 +81,3 @@ is escaped before going to the browser.
|
|||
|
||||
It should be possible for third parties to tell when pairing is done,
|
||||
but it's actually rather hard since they don't necessarily share the secret.
|
||||
* Pairing over XMPP.
|
||||
|
|
|
@ -18,13 +18,9 @@ all the other git clones, at both the git level and the key/value level.
|
|||
airport somewhere.
|
||||
* Find a way to probe available outgoing bandwidth, to throttle so
|
||||
we don't bufferbloat the network to death.
|
||||
* Investigate the XMPP approach like dvcs-autosync does, or other ways of
|
||||
signaling a change out of band.
|
||||
* Add a hook, so when there's a change to sync, a program can be run
|
||||
and do its own signaling.
|
||||
* --debug will show often unnecessary work being done. Optimise.
|
||||
* This assumes the network is connected. It's often not, so the
|
||||
[[cloud]] needs to be used to bridge between LANs.
|
||||
* Configurablity, including only enabling git syncing but not data transfer;
|
||||
only uploading new files but not downloading, and only downloading
|
||||
files in some directories and not others. See for use cases:
|
||||
|
@ -41,14 +37,11 @@ all the other git clones, at both the git level and the key/value level.
|
|||
working copy and checks each file. That probably needs to be done once,
|
||||
but further calls to the TransferScanner could eg, look at the delta
|
||||
between the last scan and the current one in the git-annex branch.
|
||||
* Ensure that when a remote receives content, and updates its location log,
|
||||
it syncs that update back out. Currently, it does not, unless the master
|
||||
branch also changed. Prerequisite for:
|
||||
* After git sync, identify new content that we don't have that is now available
|
||||
on remotes, and transfer. (Needed when we have a uni-directional connection
|
||||
to a remote, so it won't be uploading content to us.) Note: Does not
|
||||
need to use the TransferScanner, if we get and check a list of the changed
|
||||
files.
|
||||
files. done?
|
||||
* [[use multiple transfer slots|todo/Slow_transfer_for_a_lot_of_small_files.]]
|
||||
* The TransferQueue's list of deferred downloads could theoretically
|
||||
grow without bounds in memory. Limit it to a given number of entries,
|
||||
|
|
|
@ -19,9 +19,6 @@ something smart with such remotes.
|
|||
log is not updated in time, it will fail to drop unwanted content.
|
||||
(There's a 10 second sleep there now to avoid the race, but that's hardly
|
||||
a fix.)
|
||||
* When a file is renamed into an archive directory, it's not immediately
|
||||
transferred to archive remotes. (Next expensive scan does successfully
|
||||
cause the transfer to happen).
|
||||
|
||||
### dropping no longer preferred content
|
||||
|
||||
|
@ -48,7 +45,7 @@ the same content, this gets tricky. Let's assume there are not.)
|
|||
|
||||
That's all! Of these, 1-4 are by far the most important.
|
||||
|
||||
## specifying what data a remote prefers to contain **done**
|
||||
## specifying what data a remote prefers to contain (**done**)
|
||||
|
||||
Imagine a per-remote preferred content setting, that matches things that
|
||||
should be stored on the remote.
|
||||
|
@ -67,7 +64,7 @@ check if preferred content settings rejects the data, and if so, drop it
|
|||
from the repo. So once all three laptops have the data, it is
|
||||
pruned from the transfer drive.
|
||||
|
||||
## repo groups **done**
|
||||
## repo groups (**done**)
|
||||
|
||||
Seems like git-annex needs a way to know the groups of repos. Some
|
||||
groups:
|
||||
|
@ -79,7 +76,7 @@ groups:
|
|||
it does not hold data for long periods of time, and tends to have a
|
||||
limited size.
|
||||
|
||||
Add a group.log that can assign repos to these or other groups. **done**
|
||||
Add a group.log that can assign repos to these or other groups. (**done**)
|
||||
|
||||
Some examples of using groups:
|
||||
|
||||
|
@ -104,7 +101,10 @@ Currently, we have a simple drop down list to select between a few
|
|||
predefined groups with pre-defined preferred content recipes. Is this good
|
||||
enough?
|
||||
|
||||
## the state change problem **done**
|
||||
I think so; useful recipes can be developed on the wiki and included in
|
||||
git-annex.
|
||||
|
||||
## the state change problem (**done**)
|
||||
|
||||
Imagine that a trusted repo has setting like `not copies=trusted:2`
|
||||
This means that `git annex get --auto` should get files not in 2 trusted
|
||||
|
|
|
@ -28,7 +28,7 @@ The webapp is a web server that displays a shiny interface.
|
|||
See: [[todo/wishlist:_an_"assistant"_for_web-browsing_--_tracking_the_sources_of_the_downloads]]
|
||||
* Display the `inotify max_user_watches` exceeded message. **done**
|
||||
* Display something sane when kqueue runs out of file descriptors.
|
||||
* allow removing git remotes
|
||||
* allow removing git remotes **done**
|
||||
* allow disabling syncing to here, which should temporarily disable all
|
||||
local syncing.
|
||||
|
||||
|
|
Loading…
Reference in a new issue