mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

updates

Browse source
This commit is contained in:
Joey Hess 2024-06-12 11:55:18 -04:00
parent 2e76a4744f
commit 67d1e2a459
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 58 additions and 31 deletions
Download patch file Download diff file Expand all files Collapse all files

67
doc/todo/git-annex_proxies.mdwn
View file

@ -26,46 +26,55 @@ In development on the `proxy` branch.
For June's work on [[design/passthrough_proxy]], implementation plan:
1. UUID discovery via git-annex branch. Add a log file listing UUIDs
accessible via proxy UUIDs. It also will contain the names
of the remotes that the proxy is a proxy for,
from the perspective of the proxy. (done)
* UUID discovery via git-annex branch. Add a log file listing UUIDs
accessible via proxy UUIDs. It also will contain the names
of the remotes that the proxy is a proxy for,
from the perspective of the proxy. (done)
1. Add `git-annex updateproxy` command and remote.name.annex-proxy
configuration. (done)
* Add `git-annex updateproxy` command and remote.name.annex-proxy
configuration. (done)
2. Remote instantiation for proxies. (done)
* Remote instantiation for proxies. (done)
3. Implement git-annex-shell proxying to git remotes. (done)
* Implement git-annex-shell proxying to git remotes. (done)
3. Proxy should update location tracking information for proxied remotes,
so it is available to other users who sync with it. (done)
* Proxy should update location tracking information for proxied remotes,
so it is available to other users who sync with it. (done)
4. Let `storeKey` return a list of UUIDs where content was stored,
and make proxies accept uploads directed at them, rather than a specific
instantiated remote, and fan out the upload to whatever nodes behind
the proxy want it. This will need P2P protocol extensions.
* Would it be possible to get instantiated remotes into git remote list?
This would make eg, tab completion of remote names work. Just setting
annex-uuid would suffice, but currently any such config prevents setting
up a remote as an instantiated remote. Perhaps if only annex-uuid and no
other config is set, treat that as an instantiated remote, and overwrite
the annex-uuid config as necessaery? Or, add a config that says this is
an instanatiated remote, and when set, allow overwriting configs.
This seems better, it would let `git push proxy-foo` work, for example.
5. Make `git-annex copy --from $proxy` pick a node that contains each
file, and use the instantiated remote for getting the file. Same for
similar commands.
* Cycle prevention. See design.
6. Make `git-annex drop --from $proxy` drop, when possible, from every
remote accessible by the proxy. Communicate partial drops somehow.
* Make `git-annex copy --from $proxy` pick a node that contains each
file, and use the instantiated remote for getting the file. Same for
similar commands.
7. Make commands like `git-annex push` not iterate over instantiate
remotes, and instead just send content to the proxy for fanout.
* Make `git-annex drop --from $proxy` drop, when possible, from every
remote accessible by the proxy. Communicate partial drops somehow.
8. Optimise proxy speed. See design for idea.
* Let `storeKey` return a list of UUIDs where content was stored,
and make proxies accept uploads directed at them, rather than a specific
instantiated remote, and fan out the upload to whatever nodes behind
the proxy want it. This will need P2P protocol extensions.
8. Use `sendfile()` to avoid data copying overhead when
`receiveBytes` is being fed right into `sendBytes`.
* Make commands like `git-annex push` not iterate over instantiated
remotes, and instead just send content to the proxy for fanout.
9. Encryption and chunking. See design for issues.
* Optimise proxy speed. See design for ideas.
10. Cycle prevention. See design.
* Use `sendfile()` to avoid data copying overhead when
`receiveBytes` is being fed right into `sendBytes`.
11. indirect uploads (to be considered). See design.
* Encryption and chunking. See design for issues.
12. Support using a proxy when its url is a P2P address.
(Eg tor-annex remotes.)
* indirect uploads (to be considered). See design.
* Support using a proxy when its url is a P2P address.
(Eg tor-annex remotes.)