read through all design docs and updated done items
This commit is contained in:
parent
714afb8ede
commit
6d5a84a4fc
6 changed files with 12 additions and 20 deletions
|
@ -2,7 +2,7 @@ The [[syncing]] design assumes the network is connected. But it's often
|
||||||
not in these pre-IPV6 days, so the cloud needs to be used to bridge between
|
not in these pre-IPV6 days, so the cloud needs to be used to bridge between
|
||||||
LANS.
|
LANS.
|
||||||
|
|
||||||
## The cloud notification problem **done**
|
## The cloud notification problem (**done**)
|
||||||
|
|
||||||
Alice and Bob have repos, and there is a cloud remote they both share.
|
Alice and Bob have repos, and there is a cloud remote they both share.
|
||||||
Alice adds a file; the assistant transfers it to the cloud remote.
|
Alice adds a file; the assistant transfers it to the cloud remote.
|
||||||
|
|
|
@ -8,11 +8,11 @@ The meat of the configuration will be in configuration assistants that walk
|
||||||
through setting up common use cases.
|
through setting up common use cases.
|
||||||
|
|
||||||
* Create a repository (run when the web app is started without a configured
|
* Create a repository (run when the web app is started without a configured
|
||||||
repository too).
|
repository too). **done**
|
||||||
* Clone this repo to a USB drive or other removable drive. **done**
|
* Clone this repo to a USB drive or other removable drive. **done**
|
||||||
* Make a bare repo on a remote ssh server **done**
|
* Make a bare repo on a remote ssh server **done**
|
||||||
* Clone this repo to another host. (Needs [[pairing]]) **done**
|
* Clone this repo to another host. (Needs [[pairing]]) **done**
|
||||||
* Set up Amazon S3.
|
* Set up Amazon S3. **done**
|
||||||
* Set up encrypted rsync remote. **done**
|
* Set up encrypted rsync remote. **done**
|
||||||
* Rsync.net special case **done**
|
* Rsync.net special case **done**
|
||||||
* Set up gpg encryption key; gpg key distribution.
|
* Set up gpg encryption key; gpg key distribution.
|
||||||
|
|
|
@ -81,4 +81,3 @@ is escaped before going to the browser.
|
||||||
|
|
||||||
It should be possible for third parties to tell when pairing is done,
|
It should be possible for third parties to tell when pairing is done,
|
||||||
but it's actually rather hard since they don't necessarily share the secret.
|
but it's actually rather hard since they don't necessarily share the secret.
|
||||||
* Pairing over XMPP.
|
|
||||||
|
|
|
@ -18,13 +18,9 @@ all the other git clones, at both the git level and the key/value level.
|
||||||
airport somewhere.
|
airport somewhere.
|
||||||
* Find a way to probe available outgoing bandwidth, to throttle so
|
* Find a way to probe available outgoing bandwidth, to throttle so
|
||||||
we don't bufferbloat the network to death.
|
we don't bufferbloat the network to death.
|
||||||
* Investigate the XMPP approach like dvcs-autosync does, or other ways of
|
|
||||||
signaling a change out of band.
|
|
||||||
* Add a hook, so when there's a change to sync, a program can be run
|
* Add a hook, so when there's a change to sync, a program can be run
|
||||||
and do its own signaling.
|
and do its own signaling.
|
||||||
* --debug will show often unnecessary work being done. Optimise.
|
* --debug will show often unnecessary work being done. Optimise.
|
||||||
* This assumes the network is connected. It's often not, so the
|
|
||||||
[[cloud]] needs to be used to bridge between LANs.
|
|
||||||
* Configurablity, including only enabling git syncing but not data transfer;
|
* Configurablity, including only enabling git syncing but not data transfer;
|
||||||
only uploading new files but not downloading, and only downloading
|
only uploading new files but not downloading, and only downloading
|
||||||
files in some directories and not others. See for use cases:
|
files in some directories and not others. See for use cases:
|
||||||
|
@ -41,14 +37,11 @@ all the other git clones, at both the git level and the key/value level.
|
||||||
working copy and checks each file. That probably needs to be done once,
|
working copy and checks each file. That probably needs to be done once,
|
||||||
but further calls to the TransferScanner could eg, look at the delta
|
but further calls to the TransferScanner could eg, look at the delta
|
||||||
between the last scan and the current one in the git-annex branch.
|
between the last scan and the current one in the git-annex branch.
|
||||||
* Ensure that when a remote receives content, and updates its location log,
|
|
||||||
it syncs that update back out. Currently, it does not, unless the master
|
|
||||||
branch also changed. Prerequisite for:
|
|
||||||
* After git sync, identify new content that we don't have that is now available
|
* After git sync, identify new content that we don't have that is now available
|
||||||
on remotes, and transfer. (Needed when we have a uni-directional connection
|
on remotes, and transfer. (Needed when we have a uni-directional connection
|
||||||
to a remote, so it won't be uploading content to us.) Note: Does not
|
to a remote, so it won't be uploading content to us.) Note: Does not
|
||||||
need to use the TransferScanner, if we get and check a list of the changed
|
need to use the TransferScanner, if we get and check a list of the changed
|
||||||
files.
|
files. done?
|
||||||
* [[use multiple transfer slots|todo/Slow_transfer_for_a_lot_of_small_files.]]
|
* [[use multiple transfer slots|todo/Slow_transfer_for_a_lot_of_small_files.]]
|
||||||
* The TransferQueue's list of deferred downloads could theoretically
|
* The TransferQueue's list of deferred downloads could theoretically
|
||||||
grow without bounds in memory. Limit it to a given number of entries,
|
grow without bounds in memory. Limit it to a given number of entries,
|
||||||
|
|
|
@ -19,9 +19,6 @@ something smart with such remotes.
|
||||||
log is not updated in time, it will fail to drop unwanted content.
|
log is not updated in time, it will fail to drop unwanted content.
|
||||||
(There's a 10 second sleep there now to avoid the race, but that's hardly
|
(There's a 10 second sleep there now to avoid the race, but that's hardly
|
||||||
a fix.)
|
a fix.)
|
||||||
* When a file is renamed into an archive directory, it's not immediately
|
|
||||||
transferred to archive remotes. (Next expensive scan does successfully
|
|
||||||
cause the transfer to happen).
|
|
||||||
|
|
||||||
### dropping no longer preferred content
|
### dropping no longer preferred content
|
||||||
|
|
||||||
|
@ -48,7 +45,7 @@ the same content, this gets tricky. Let's assume there are not.)
|
||||||
|
|
||||||
That's all! Of these, 1-4 are by far the most important.
|
That's all! Of these, 1-4 are by far the most important.
|
||||||
|
|
||||||
## specifying what data a remote prefers to contain **done**
|
## specifying what data a remote prefers to contain (**done**)
|
||||||
|
|
||||||
Imagine a per-remote preferred content setting, that matches things that
|
Imagine a per-remote preferred content setting, that matches things that
|
||||||
should be stored on the remote.
|
should be stored on the remote.
|
||||||
|
@ -67,7 +64,7 @@ check if preferred content settings rejects the data, and if so, drop it
|
||||||
from the repo. So once all three laptops have the data, it is
|
from the repo. So once all three laptops have the data, it is
|
||||||
pruned from the transfer drive.
|
pruned from the transfer drive.
|
||||||
|
|
||||||
## repo groups **done**
|
## repo groups (**done**)
|
||||||
|
|
||||||
Seems like git-annex needs a way to know the groups of repos. Some
|
Seems like git-annex needs a way to know the groups of repos. Some
|
||||||
groups:
|
groups:
|
||||||
|
@ -79,7 +76,7 @@ groups:
|
||||||
it does not hold data for long periods of time, and tends to have a
|
it does not hold data for long periods of time, and tends to have a
|
||||||
limited size.
|
limited size.
|
||||||
|
|
||||||
Add a group.log that can assign repos to these or other groups. **done**
|
Add a group.log that can assign repos to these or other groups. (**done**)
|
||||||
|
|
||||||
Some examples of using groups:
|
Some examples of using groups:
|
||||||
|
|
||||||
|
@ -104,7 +101,10 @@ Currently, we have a simple drop down list to select between a few
|
||||||
predefined groups with pre-defined preferred content recipes. Is this good
|
predefined groups with pre-defined preferred content recipes. Is this good
|
||||||
enough?
|
enough?
|
||||||
|
|
||||||
## the state change problem **done**
|
I think so; useful recipes can be developed on the wiki and included in
|
||||||
|
git-annex.
|
||||||
|
|
||||||
|
## the state change problem (**done**)
|
||||||
|
|
||||||
Imagine that a trusted repo has setting like `not copies=trusted:2`
|
Imagine that a trusted repo has setting like `not copies=trusted:2`
|
||||||
This means that `git annex get --auto` should get files not in 2 trusted
|
This means that `git annex get --auto` should get files not in 2 trusted
|
||||||
|
|
|
@ -28,7 +28,7 @@ The webapp is a web server that displays a shiny interface.
|
||||||
See: [[todo/wishlist:_an_"assistant"_for_web-browsing_--_tracking_the_sources_of_the_downloads]]
|
See: [[todo/wishlist:_an_"assistant"_for_web-browsing_--_tracking_the_sources_of_the_downloads]]
|
||||||
* Display the `inotify max_user_watches` exceeded message. **done**
|
* Display the `inotify max_user_watches` exceeded message. **done**
|
||||||
* Display something sane when kqueue runs out of file descriptors.
|
* Display something sane when kqueue runs out of file descriptors.
|
||||||
* allow removing git remotes
|
* allow removing git remotes **done**
|
||||||
* allow disabling syncing to here, which should temporarily disable all
|
* allow disabling syncing to here, which should temporarily disable all
|
||||||
local syncing.
|
local syncing.
|
||||||
|
|
||||||
|
|
Loading…
Add table
Add a link
Reference in a new issue