git-annex/doc
Joey Hess 9d36c826c0
use fine-grained WorkerStages when transferring and verifying
This means that Command.Move and Command.Get don't need to
manually set the stage, and is a lot cleaner conceptually.

Also, this makes Command.Sync.syncFile use the worker pool better.
In the scenario where it first downloads content and then uploads it to
some other remotes, it will start in TransferStage, then enter VerifyStage
and then go back to TransferStage for each transfer to the remotes.
Before, it entered CleanupStage after the download, and stayed in it for
the upload, so too many transfer jobs could run at the same time.

Note that, in Remote.Git, it uses runTransfer and also verifyKeyContent
inside onLocal. That has a Annex state for the remote, with no worker pool.
So the resulting calls to enteringStage won't block in there.

While Remote.Git.copyToRemote does do checksum verification, I
realized that should not use a verification slot in the WorkerPool
to do it. Because, it's reading back from eg, a removable disk to checksum.
That will contend with other writes to that disk. It's best to treat
that checksum verification as just part of the transer. So, removed the todo
item about that, as there's nothing needing to be done.
2019-06-19 13:24:20 -04:00
..
Android Added a comment: Android - gitlab repo 2019-05-08 11:35:04 +00:00
android
assistant
automatic_conflict_resolution
backends
bare_repositories
bugs 2019-06-14 04:24:47 +00:00
coding_style
contribute
copies
design clarify 2019-06-04 21:49:53 -04:00
devblog devblog 2019-06-17 15:26:46 -04:00
direct_mode
download
encryption Added a comment: Option to disable filename encyprtion 2018-12-29 01:13:21 +00:00
footer
forum re: status of v7 2019-06-12 15:53:12 +00:00
future_proofing
git-annex-add response 2019-01-22 17:11:21 -04:00
git-annex-addurl Added a comment: Re: Inconsistent idiom 2019-03-28 03:59:04 +00:00
git-annex-adjust
git-annex-checkpresentkey
git-annex-config
git-annex-copy
git-annex-direct
git-annex-drop
git-annex-enableremote
git-annex-find Added a comment: Find (and other operations) over multiple repos 2019-02-25 14:33:27 +00:00
git-annex-importfeed
git-annex-lookupkey comment 2019-04-03 13:01:07 -04:00
git-annex-metadata Added a comment 2019-05-03 15:50:43 +00:00
git-annex-move
git-annex-preferred-content
git-annex-reinject
git-annex-rmurl
git-annex-setpresentkey
git-annex-sync
git-annex-test
git-annex-unannex
git-annex-undo
git-annex-uninit response 2019-01-01 12:31:18 -04:00
git-annex-unlock Added a comment 2019-01-01 18:17:21 +00:00
git-annex-unused
git-annex-whereis
how_it_works
install Added a comment: Warning building with cabal 2019-03-30 00:19:11 +00:00
internals added a hyperlink from key internals page to git-annex-examinekey 2019-04-24 17:18:33 +00:00
license
links
metadata
news add news item for git-annex 7.20190615 2019-06-15 12:39:48 -04:00
not
polls
preferred_content response 2018-12-26 13:04:35 -04:00
profiling new comment (and rename for consistency) 2019-01-14 18:01:02 -04:00
required_content
scalability
security rename annex.security.allowed-http-addresses 2019-05-30 12:43:40 -04:00
special_remotes reorg page 2019-05-28 11:53:49 -04:00
stickers
sync
templates
testimonials
thanks update 2019-06-02 15:26:50 -04:00
tips rename annex.security.allowed-http-addresses 2019-05-30 12:43:40 -04:00
todo use fine-grained WorkerStages when transferring and verifying 2019-06-19 13:24:20 -04:00
trust
tuning
upgrades
use_case
users add kyle 2019-02-08 17:22:43 +00:00
videos
walkthrough this is not all run as root 2019-03-27 11:15:03 -04:00
Android.mdwn adb import 2019-04-09 17:52:41 -04:00
assistant.mdwn
automatic_conflict_resolution.mdwn
backends.mdwn
bare_repositories.mdwn
bugs.mdwn
builds.mdwn remove 2019-05-03 11:06:35 -04:00
chunking.mdwn
coding_style.mdwn
comments.mdwn
contact.mdwn
contribute.mdwn
copies.mdwn
design.mdwn
devblog.mdwn
direct_mode.mdwn link deprecation notice to tips/unlocked_files 2019-01-19 06:41:10 +00:00
distributed_version_control.mdwn
download.mdwn fix link on download page and add a few more to break later 2018-12-10 12:55:13 -04:00
encryption.mdwn annex.cachecreds: New config to allow disabling of credentials caching for special remotes. 2018-12-04 14:16:56 -04:00
favicon.ico
forum.mdwn
future_proofing.mdwn
git-annex-add.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-addunused.mdwn
git-annex-addurl.mdwn rename annex.security.allowed-http-addresses 2019-05-30 12:43:40 -04:00
git-annex-adjust.mdwn mention annex.thin for unlock 2019-01-01 12:23:40 -04:00
git-annex-assistant.mdwn
git-annex-benchmark.mdwn new improved benchmark command that can benchmark anything git-annex does 2019-01-04 13:46:36 -04:00
git-annex-calckey.mdwn shorten some too-long descriptions 2019-01-16 14:16:32 -04:00
git-annex-checkpresentkey.mdwn
git-annex-config.mdwn
git-annex-contentlocation.mdwn
git-annex-copy.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-dead.mdwn added renameremote command 2019-04-15 13:49:03 -04:00
git-annex-describe.mdwn
git-annex-diffdriver.mdwn
git-annex-direct.mdwn
git-annex-drop.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-dropkey.mdwn
git-annex-dropunused.mdwn
git-annex-edit.mdwn
git-annex-enable-tor.mdwn
git-annex-enableremote.mdwn
git-annex-examinekey.mdwn
git-annex-expire.mdwn
git-annex-export.mdwn honor preferred content when importing 2019-05-21 14:38:06 -04:00
git-annex-find.mdwn add --branch option to git-annex find and mildly deprecate findref in favor of it 2018-12-09 14:10:37 -04:00
git-annex-findref.mdwn add --branch option to git-annex find and mildly deprecate findref in favor of it 2018-12-09 14:10:37 -04:00
git-annex-fix.mdwn
git-annex-forget.mdwn
git-annex-fromkey.mdwn fromkey --json 2019-02-05 14:03:29 -04:00
git-annex-fsck.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-fuzztest.mdwn
git-annex-get.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-group.mdwn
git-annex-groupwanted.mdwn
git-annex-import.mdwn honor preferred content when importing 2019-05-21 14:38:06 -04:00
git-annex-importfeed.mdwn rename annex.security.allowed-http-addresses 2019-05-30 12:43:40 -04:00
git-annex-indirect.mdwn
git-annex-info.mdwn improve wording 2019-03-28 13:25:14 -04:00
git-annex-init.mdwn
git-annex-initremote.mdwn split out section on common configuration parameters 2019-04-16 13:09:12 -04:00
git-annex-inprogress.mdwn
git-annex-list.mdwn
git-annex-lock.mdwn
git-annex-log.mdwn
git-annex-lookupkey.mdwn
git-annex-map.mdwn
git-annex-matchexpression.mdwn Added mimeencoding= term to annex.largefiles expressions. 2019-04-30 12:17:22 -04:00
git-annex-matching-options.mdwn
git-annex-merge.mdwn
git-annex-metadata.mdwn response; cleanup 2019-05-03 11:37:41 -04:00
git-annex-migrate.mdwn
git-annex-mirror.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-move.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-multicast.mdwn
git-annex-numcopies.mdwn
git-annex-p2p.mdwn
git-annex-post-receive.mdwn
git-annex-pre-commit.mdwn
git-annex-preferred-content.mdwn honor preferred content when importing 2019-05-21 14:38:06 -04:00
git-annex-proxy.mdwn
git-annex-readpresentkey.mdwn
git-annex-registerurl.mdwn
git-annex-reinit.mdwn
git-annex-reinject.mdwn
git-annex-rekey.mdwn
git-annex-remotedaemon.mdwn
git-annex-renameremote.mdwn forgot to add man page 2019-04-16 12:25:49 -04:00
git-annex-repair.mdwn
git-annex-required.mdwn
git-annex-resolvemerge.mdwn
git-annex-rmurl.mdwn
git-annex-schedule.mdwn
git-annex-semitrust.mdwn
git-annex-setkey.mdwn
git-annex-setpresentkey.mdwn
git-annex-shell.mdwn
git-annex-smudge.mdwn
git-annex-status.mdwn
git-annex-sync.mdwn annex.jobs=cpus etc 2019-05-10 13:27:08 -04:00
git-annex-test.mdwn
git-annex-testremote.mdwn testremote: Support testing readonly remotes with the --test-readonly option 2019-01-17 12:44:52 -04:00
git-annex-transferkey.mdwn
git-annex-transferkeys.mdwn
git-annex-trust.mdwn
git-annex-unannex.mdwn
git-annex-undo.mdwn
git-annex-ungroup.mdwn
git-annex-uninit.mdwn
git-annex-unlock.mdwn
git-annex-untrust.mdwn
git-annex-unused.mdwn doc clarification 2019-04-09 11:38:11 -04:00
git-annex-upgrade.mdwn
git-annex-vadd.mdwn
git-annex-vcycle.mdwn
git-annex-version.mdwn
git-annex-vfilter.mdwn
git-annex-vicfg.mdwn
git-annex-view.mdwn
git-annex-vpop.mdwn
git-annex-wanted.mdwn
git-annex-watch.mdwn
git-annex-webapp.mdwn
git-annex-whereis.mdwn
git-annex.mdwn rename annex.security.allowed-http-addresses 2019-05-30 12:43:40 -04:00
git-remote-tor-annex.mdwn
git-union-merge.mdwn
how_it_works.mdwn
index.mdwn
install.mdwn
internals.mdwn use colon not space to delimit content identifier list 2019-02-21 13:45:16 -04:00
license.mdwn update 2019-02-08 15:26:15 -04:00
location_tracking.mdwn
logo-old-bw.svg
logo-old.png
logo-old.svg
logo-old_small.png
logo.mdwn
logo.svg
logo_16x16.png
logo_32x32.png
logo_small.png
meta.mdwn
metadata.mdwn
news.mdwn
not.mdwn
polls.mdwn
preferred_content.mdwn
privacy.mdwn
profiling.mdwn rename page 2019-01-14 15:57:04 -04:00
publicrepos.mdwn
related_software.mdwn Updated Densho description , formerly "Japanese American Legacy Project", and total collection size. 2019-01-04 17:55:54 +00:00
repomap.png
required_content.mdwn
scalability.mdwn
security.mdwn
shortcuts.mdwn
sidebar.mdwn
sitemap.mdwn
special_remotes.mdwn improve docs 2019-03-22 09:59:56 -04:00
stickers.mdwn
submodules.mdwn
summary.mdwn
sync.mdwn
testimonials.mdwn
thanks.mdwn update logo 2018-12-29 13:44:39 -04:00
tips.mdwn
todo.mdwn don't list subpages of todos 2019-04-15 13:01:53 -04:00
transferring_data.mdwn
trust.mdwn
tuning.mdwn
upgrades.mdwn clarify anarcat's change 2018-12-05 12:22:36 -04:00
users.mdwn
videos.mdwn
Void.mdwn
walkthrough.mdwn
workflow.mdwn