mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

implementation plan

Browse source
This commit is contained in:
Joey Hess 2024-06-04 07:51:33 -04:00
parent 6375e3be3b
commit 3df70c5c0c
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 73 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

50
doc/todo/git-annex_proxies.mdwn
View file

@ -19,3 +19,53 @@ Planned schedule of work:
* October: proving behavior of balanced preferred content with proxies
[[!tag projects/openneuro]]
# work notes
For June's work on [[design/passthrough_proxy]], implementation plan:
1. UUID discovery via git-annex branch. Add a log file listing UUIDs
accessible via proxy UUIDs. It also will contain the names
of the remotes that the proxy is a proxy for,
from the perspective of the proxy.
Note that remote names coming from the git-annex branch need to be
limited to what's legal in git remote names.
This will also prevent remote names being a security hazard
via eg escape characters.
1. Add a command that is run on the proxy to update the proxy log file.
This is how the user sets it up as a proxy, and selects the remotes its
proxying for.
2. Remote instantiation for proxies. When a remote "foo" is a proxy,
and has a remote "bar", instantiate a remote "foo-bar" that has the UUID
of bar but is of the same type and configuration of remote "foo".
3. Implement proxying in git-annex-shell so connections with the UUID
of one of the proxy's
4. Let `storeKey` return a list of UUIDs where content was stored,
and make proxies accept uploads directed at them, rather than a specific
instantiated remote, and fan out the upload to whatever nodes behind
the proxy want it. This will need P2P protocol extensions.
5. Make `git-annex copy --from $proxy` pick a node that contains each
file, and use the instantiated remote for getting the file. Same for
similar commands.
6. Make `git-annex drop --from $proxy` drop, when possible, from every
remote accessible by the proxy. Communicate partial drops somehow.
7. Make commands like `git-annex push` not iterate over instantiate
remotes, and instead just send content to the proxy for fanout.
8. Optimise proxy speed. See design for idea.
9. Encryption and chunking. See design for issues.
10. Cycle prevention. See design.
11. indirect uploads (to be considered). See design.