git-annex/Command
Joey Hess 18e00500ce
bwlimit
Added annex.bwlimit and remote.name.annex-bwlimit config that works for git
remotes and many but not all special remotes.

This nearly works, at least for a git remote on the same disk. With it set
to 100kb/1s, the meter displays an actual bandwidth of 128 kb/s, with
occasional spikes to 160 kb/s. So it needs to delay just a bit longer...
I'm unsure why.

However, at the beginning a lot of data flows before it determines the
right bandwidth limit. A granularity of less than 1s would probably improve
that.

And, I don't know yet if it makes sense to have it be 100ks/1s rather than
100kb/s. Is there a situation where the user would want a larger
granularity? Does granulatity need to be configurable at all? I only used that
format for the config really in order to reuse an existing parser.

This can't support for external special remotes, or for ones that
themselves shell out to an external command. (Well, it could, but it
would involve pausing and resuming the child process tree, which seems
very hard to implement and very strange besides.) There could also be some
built-in special remotes that it still doesn't work for, due to them not
having a progress meter whose displays blocks the bandwidth using thread.
But I don't think there are actually any that run a separate thread for
downloads than the thread that displays the progress meter.

Sponsored-by: Graham Spencer on Patreon
2021-09-21 16:58:10 -04:00
..
Add.hs bwlimit 2021-09-21 16:58:10 -04:00
AddUnused.hs more RawFilePath conversion 2020-11-03 10:11:04 -04:00
AddUrl.hs improve url download failure display 2021-09-01 15:33:38 -04:00
Adjust.hs rename to --unlock-present and better reverse adjusting 2020-11-13 14:56:43 -04:00
Assistant.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Benchmark.hs improve benchmark --databases 2019-11-21 17:25:20 -04:00
CalcKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
CheckPresentKey.hs --batch-keys 2021-08-25 14:21:12 -04:00
Commit.hs use Branch.name instead of hard coding the branch name 2020-09-15 15:47:22 -04:00
Config.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
ConfigList.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
ContentLocation.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Copy.hs --batch-keys 2021-08-25 14:21:12 -04:00
Dead.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Describe.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
DiffDriver.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Direct.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Drop.hs --batch-keys 2021-08-25 14:21:12 -04:00
DropKey.hs --batch-keys 2021-08-25 14:21:12 -04:00
DropUnused.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
EnableRemote.hs cache remote.log 2020-09-22 13:52:26 -04:00
EnableTor.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
ExamineKey.hs examinekey: Added a "file" format variable 2020-11-16 09:59:11 -04:00
Expire.hs Future proof activity log parsing 2021-06-14 14:18:19 -04:00
Export.hs add git ls-tree --long parser 2021-03-23 12:47:00 -04:00
FilterBranch.hs decided not to include export/import trees 2021-05-17 14:12:15 -04:00
Find.hs --batch-keys 2021-08-25 14:21:12 -04:00
FindRef.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Fix.hs fix potential race in updating inode cache 2021-07-27 13:08:08 -04:00
Forget.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
FromKey.hs --batch-keys 2021-08-25 14:21:12 -04:00
Fsck.hs add: Detect when xattrs or perhaps ACLs prevent locking down a file's content 2021-08-27 14:33:01 -04:00
FuzzTest.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
GCryptSetup.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Get.hs --batch-keys 2021-08-25 14:21:12 -04:00
Group.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
GroupWanted.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Help.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Import.hs defer write permissions checking in import until after copy to repo 2021-09-02 13:45:21 -04:00
ImportFeed.hs incremental verification for web special remote 2021-08-18 15:02:22 -04:00
InAnnex.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Indirect.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Info.hs --batch-keys 2021-08-25 14:21:12 -04:00
Init.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
InitRemote.hs initremote --private 2021-04-23 14:47:46 -04:00
Inprogress.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
List.hs add "input" field to json output 2020-09-15 16:22:44 -04:00
Lock.hs fix a place where the inode cache could potentially have gotten stale 2021-07-26 14:12:58 -04:00
LockContent.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Log.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
LookupKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Map.hs avoid making absolute git remote path relative 2021-02-08 13:18:01 -04:00
MatchExpression.hs split ProvidedInfo and UserProvidedInfo 2020-09-28 12:12:38 -04:00
Merge.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
MetaData.hs --batch-keys 2021-08-25 14:21:12 -04:00
Migrate.hs include locked files in the keys database associated files 2021-05-21 16:24:37 -04:00
MinCopies.hs mincopies 2021-01-06 14:15:19 -04:00
Mirror.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
Move.hs --batch-keys 2021-08-25 14:21:12 -04:00
Multicast.hs Command/Multicast: use proper hyphen 2021-05-04 05:44:31 +02:00
NotifyChanges.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
NumCopies.hs mincopies 2021-01-06 14:15:19 -04:00
P2P.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
P2PStdIO.hs finish CommandStart transition 2019-06-12 13:24:01 -04:00
PostReceive.hs remove direct mode remnant of merging unrelated histories 2021-07-19 11:41:26 -04:00
PreCommit.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Proxy.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
ReadPresentKey.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
RecvKey.hs make getViaTmpFrom no longer update location log 2020-12-11 11:50:13 -04:00
RegisterUrl.hs --batch-keys 2021-08-25 14:21:12 -04:00
Reinit.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
Reinject.hs split verifyKeyContent 2021-07-29 13:58:40 -04:00
ReKey.hs --batch-keys 2021-08-25 14:21:12 -04:00
RemoteDaemon.hs assistant: Fix a crash on startup by avoiding using forkProcess 2021-05-12 15:08:03 -04:00
RenameRemote.hs cache remote.log 2020-09-22 13:52:26 -04:00
Repair.hs finished this stage of the RawFilePath conversion 2020-11-06 14:10:58 -04:00
Required.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
ResolveMerge.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
RmUrl.hs --batch-keys 2021-08-25 14:21:12 -04:00
Schedule.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Semitrust.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
SendKey.hs use fastDebug everywhere it can be used 2021-04-06 15:41:24 -04:00
SetKey.hs annex.adjustedbranchrefresh 2020-11-16 14:27:28 -04:00
SetPresentKey.hs --batch-keys 2021-08-25 14:21:12 -04:00
Smudge.hs defer write permissions checking in import until after copy to repo 2021-09-02 13:45:21 -04:00
Status.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
Sync.hs sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
Test.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
TestRemote.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
TransferInfo.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
TransferKey.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
TransferKeys.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
Transferrer.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
Trust.hs fix format of message 2021-01-11 13:14:09 -04:00
Unannex.hs remove inode cache in unannex 2021-05-12 11:09:38 -04:00
Undo.hs finished this stage of the RawFilePath conversion 2020-11-04 14:20:37 -04:00
Ungroup.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Uninit.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Unlock.hs fix potential race in updating inode cache 2021-07-27 13:08:08 -04:00
UnregisterUrl.hs --batch-keys 2021-08-25 14:21:12 -04:00
Untrust.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Unused.hs unused: Skip the refs/annex/last-index ref that git-annex recently started creating 2021-08-24 12:58:14 -04:00
Upgrade.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
VAdd.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
VCycle.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Version.hs external backends wip 2020-07-29 15:23:18 -04:00
VFilter.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Vicfg.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
View.hs more RawFilePath conversion 2020-10-30 15:55:59 -04:00
VPop.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Wanted.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Watch.hs remotedaemon: Don't list --stop in help since it's not supported. 2019-09-30 14:40:46 -04:00
WebApp.hs finished this stage of the RawFilePath conversion 2020-11-04 14:20:37 -04:00
Whereis.hs --batch-keys 2021-08-25 14:21:12 -04:00
WhereUsed.hs better key matching with a regexp 2021-07-14 16:31:17 -04:00