git-annex/doc
Joey Hess a2f4a8eddf
proxying GET now working
Memory use is small and constant; receiveBytes returns a lazy bytestring
and it does stream.

Comparing speed of a get of a 500 mb file over proxy from origin-origin,
vs from the same remote over a direct ssh:

joey@darkstar:~/tmp/bench/client>/usr/bin/time git-annex get bigfile --from origin-origin
get bigfile (from origin-origin...)
ok
(recording state in git...)
1.89user 0.67system 0:10.79elapsed 23%CPU (0avgtext+0avgdata 68716maxresident)k
0inputs+984320outputs (0major+10779minor)pagefaults 0swaps

joey@darkstar:~/tmp/bench/client>/usr/bin/time git-annex get bigfile --from direct-ssh
get bigfile (from direct-ssh...)
ok
1.79user 0.63system 0:10.49elapsed 23%CPU (0avgtext+0avgdata 65776maxresident)k
0inputs+1024312outputs (0major+9773minor)pagefaults 0swaps

So the proxy doesn't add much overhead even when run on the same machine as
the client and remote.

Still, piping receiveBytes into sendBytes like this does suggest that the proxy
could be made to use less CPU resouces by using `sendfile()`.
2024-06-11 15:09:43 -04:00
..
Android
android
assistant
automatic_conflict_resolution
backends
bare_repositories
bugs notes on behavior 2024-06-10 11:07:04 -04:00
chunking
coding_style
contribute
copies
design started implementing git-annex-shell proxy 2024-06-10 18:01:36 -04:00
devblog devblog 2024-05-16 13:23:36 -04:00
direct_mode
download
encryption
footer
forum removed 2024-06-06 13:40:26 +00:00
future_proofing
git-annex-add
git-annex-addurl
git-annex-adjust
git-annex-benchmark
git-annex-checkpresentkey
git-annex-config
git-annex-copy
git-annex-diffdriver
git-annex-direct
git-annex-drop
git-annex-enableremote
git-annex-find
git-annex-get
git-annex-import
git-annex-importfeed
git-annex-lookupkey
git-annex-matching-options
git-annex-metadata
git-annex-move
git-annex-p2p
git-annex-preferred-content
git-annex-reinject
git-annex-repair
git-annex-rmurl
git-annex-setpresentkey
git-annex-sync
git-annex-test
git-annex-unannex
git-annex-undo
git-annex-uninit
git-annex-unlock
git-annex-unused
git-annex-webapp
git-annex-whereis promote comment to todo 2024-04-30 15:13:59 -04:00
git-annex-whereused
how_it_works
install
internals update 2024-05-28 13:36:27 -04:00
license
links
metadata
news add news item for git-annex 10.20240531 2024-05-31 12:32:42 -04:00
not
polls
preferred_content
profiling
projects Fix compatable typo (yet to add to codespell) 2024-05-01 15:46:25 -04:00
required_content
scalability
security
special_remotes Fix compatable typo (yet to add to codespell) 2024-05-01 15:46:25 -04:00
stickers
submodules
sync
templates
testimonials
thanks update 2024-05-06 11:36:43 -04:00
tips tweak 2024-05-30 13:31:49 -04:00
todo proxying GET now working 2024-06-11 15:09:43 -04:00
trust
tuning
upgrades
use_case
users
videos
walkthrough
workflow
Android.mdwn
assistant.mdwn
automatic_conflict_resolution.mdwn
backends.mdwn add Backend.GitRemoteAnnex 2024-05-07 13:54:08 -04:00
bare_repositories.mdwn
bugs.mdwn
builds.mdwn
chunking.mdwn
coding_style.mdwn
comments.mdwn
contact.mdwn
contribute.mdwn
copies.mdwn
design.mdwn
devblog.mdwn
direct_mode.mdwn
distributed_version_control.mdwn
download.mdwn
encryption.mdwn Fix compatable typo (yet to add to codespell) 2024-05-01 15:46:25 -04:00
favicon.ico
forum.mdwn
future_proofing.mdwn preparing to merge git-remote-annex 2024-05-10 15:06:15 -04:00
git-annex-add.mdwn
git-annex-addunused.mdwn
git-annex-addurl.mdwn
git-annex-adjust.mdwn
git-annex-assist.mdwn
git-annex-assistant.mdwn
git-annex-backends.mdwn
git-annex-benchmark.mdwn
git-annex-calckey.mdwn
git-annex-checkpresentkey.mdwn
git-annex-common-options.mdwn
git-annex-config.mdwn
git-annex-configremote.mdwn
git-annex-contentlocation.mdwn
git-annex-copy.mdwn
git-annex-dead.mdwn
git-annex-describe.mdwn
git-annex-diffdriver.mdwn
git-annex-direct.mdwn
git-annex-drop.mdwn
git-annex-dropkey.mdwn
git-annex-dropunused.mdwn
git-annex-edit.mdwn
git-annex-enable-tor.mdwn
git-annex-enableremote.mdwn initremote, enableremote: Added --with-url to enable using git-remote-annex 2024-05-24 14:29:36 -04:00
git-annex-examinekey.mdwn
git-annex-expire.mdwn
git-annex-export.mdwn
git-annex-filter-branch.mdwn
git-annex-filter-process.mdwn
git-annex-find.mdwn
git-annex-findkeys.mdwn
git-annex-findref.mdwn
git-annex-fix.mdwn
git-annex-forget.mdwn
git-annex-fromkey.mdwn
git-annex-fsck.mdwn
git-annex-fuzztest.mdwn
git-annex-get.mdwn
git-annex-group.mdwn group: Added --list option 2024-05-29 13:37:35 -04:00
git-annex-groupwanted.mdwn
git-annex-import.mdwn
git-annex-importfeed.mdwn
git-annex-indirect.mdwn
git-annex-info.mdwn
git-annex-init.mdwn
git-annex-initremote.mdwn initremote, enableremote: Added --with-url to enable using git-remote-annex 2024-05-24 14:29:36 -04:00
git-annex-inprogress.mdwn
git-annex-list.mdwn
git-annex-lock.mdwn
git-annex-log.mdwn
git-annex-lookupkey.mdwn
git-annex-map.mdwn
git-annex-matchexpression.mdwn
git-annex-matching-expression.mdwn
git-annex-matching-options.mdwn
git-annex-merge.mdwn
git-annex-metadata.mdwn
git-annex-migrate.mdwn
git-annex-mincopies.mdwn
git-annex-mirror.mdwn
git-annex-move.mdwn
git-annex-multicast.mdwn
git-annex-numcopies.mdwn
git-annex-oldkeys.mdwn
git-annex-p2p.mdwn
git-annex-post-receive.mdwn
git-annex-pre-commit.mdwn
git-annex-preferred-content.mdwn
git-annex-proxy.mdwn
git-annex-pull.mdwn 2024-05-02 14:32:26 +00:00
git-annex-push.mdwn fix option desc pasted from pull 2024-04-18 10:05:11 -04:00
git-annex-readpresentkey.mdwn
git-annex-registerurl.mdwn
git-annex-reinit.mdwn
git-annex-reinject.mdwn
git-annex-rekey.mdwn
git-annex-remotedaemon.mdwn
git-annex-renameremote.mdwn
git-annex-repair.mdwn
git-annex-required.mdwn
git-annex-reregisterurl.mdwn
git-annex-resolvemerge.mdwn
git-annex-restage.mdwn
git-annex-rmurl.mdwn
git-annex-satisfy.mdwn
git-annex-schedule.mdwn
git-annex-semitrust.mdwn
git-annex-setkey.mdwn
git-annex-setpresentkey.mdwn
git-annex-shell.mdwn git-annex-shell: accept uuid of remote that proxying is enabled for 2024-06-10 12:44:35 -04:00
git-annex-smudge.mdwn
git-annex-status.mdwn
git-annex-sync.mdwn
git-annex-test.mdwn
git-annex-testremote.mdwn
git-annex-transferkey.mdwn
git-annex-transferkeys.mdwn
git-annex-transferrer.mdwn
git-annex-trust.mdwn
git-annex-unannex.mdwn
git-annex-undo.mdwn
git-annex-ungroup.mdwn
git-annex-uninit.mdwn
git-annex-unlock.mdwn
git-annex-unregisterurl.mdwn
git-annex-untrust.mdwn
git-annex-unused.mdwn
git-annex-updateproxy.mdwn Added updateproxy command and remote.name.annex-proxy configuration 2024-06-04 14:52:03 -04:00
git-annex-upgrade.mdwn
git-annex-vadd.mdwn
git-annex-vcycle.mdwn
git-annex-version.mdwn
git-annex-vfilter.mdwn
git-annex-vicfg.mdwn
git-annex-view.mdwn
git-annex-vpop.mdwn
git-annex-wanted.mdwn
git-annex-watch.mdwn
git-annex-webapp.mdwn
git-annex-whereis.mdwn
git-annex-whereused.mdwn
git-annex.mdwn Added updateproxy command and remote.name.annex-proxy configuration 2024-06-04 14:52:03 -04:00
git-remote-annex.mdwn tip 2024-05-30 13:26:34 -04:00
git-remote-tor-annex.mdwn fix name of gitremote-helpers 2024-05-06 12:07:05 -04:00
git-union-merge.mdwn
how_it_works.mdwn
index.mdwn
install.mdwn
internals.mdwn Added updateproxy command and remote.name.annex-proxy configuration 2024-06-04 14:52:03 -04:00
license.mdwn
location_tracking.mdwn
logo-old-bw.svg
logo-old.png
logo-old.svg
logo-old_small.png
logo-with-cli.png
logo-with-cli.svg
logo.mdwn
logo.svg
logo_16x16.png
logo_32x32.png
logo_small.png
meta.mdwn
metadata.mdwn
news.mdwn
not.mdwn
polls.mdwn
preferred_content.mdwn
privacy.mdwn
profiling.mdwn
projects.mdwn
publicrepos.mdwn
related_software.mdwn
repomap.png
required_content.mdwn
scalability.mdwn
security.mdwn
shortcuts.mdwn
sidebar.mdwn
sitemap.mdwn
special_remotes.mdwn update docs for git-remote-annex 2024-05-14 15:31:16 -04:00
stickers.mdwn
submodules.mdwn
summary.mdwn
sync.mdwn
testimonials.mdwn
thanks.mdwn
tips.mdwn
todo.mdwn
transferring_data.mdwn
trust.mdwn
tuning.mdwn
upgrades.mdwn
users.mdwn
videos.mdwn
walkthrough.mdwn
workflow.mdwn