git-annex/Command
Joey Hess d154e7022e
incremental verification for web special remote
Except when configuration makes curl be used. It did not seem worth
trying to tail the file when curl is downloading.

But when an interrupted download is resumed, it does not read the whole
existing file to hash it. Same reason discussed in
commit 7eb3742e4b76d1d7a487c2c53bf25cda4ee5df43; that could take a long
time with no progress being displayed. And also there's an open http
request, which needs to be consumed; taking a long time to hash the file
might cause it to time out.

Also in passing implemented it for git and external special remotes when
downloading from the web. Several others like S3 are within striking
distance now as well.

Sponsored-by: Dartmouth College's DANDI project
2021-08-18 15:02:22 -04:00
..
Add.hs remove inode cache in unannex 2021-05-12 11:09:38 -04:00
AddUnused.hs more RawFilePath conversion 2020-11-03 10:11:04 -04:00
AddUrl.hs incremental verification for web special remote 2021-08-18 15:02:22 -04:00
Adjust.hs rename to --unlock-present and better reverse adjusting 2020-11-13 14:56:43 -04:00
Assistant.hs add searchPathContents 2021-02-02 19:06:15 -04:00
Benchmark.hs improve benchmark --databases 2019-11-21 17:25:20 -04:00
CalcKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
CheckPresentKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Commit.hs use Branch.name instead of hard coding the branch name 2020-09-15 15:47:22 -04:00
Config.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
ConfigList.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
ContentLocation.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Copy.hs more RawFilePath conversion 2020-11-03 10:11:04 -04:00
Dead.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Describe.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
DiffDriver.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Direct.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Drop.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
DropKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
DropUnused.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
EnableRemote.hs cache remote.log 2020-09-22 13:52:26 -04:00
EnableTor.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
ExamineKey.hs examinekey: Added a "file" format variable 2020-11-16 09:59:11 -04:00
Expire.hs Future proof activity log parsing 2021-06-14 14:18:19 -04:00
Export.hs add git ls-tree --long parser 2021-03-23 12:47:00 -04:00
FilterBranch.hs decided not to include export/import trees 2021-05-17 14:12:15 -04:00
Find.hs examinekey: Added a "file" format variable 2020-11-16 09:59:11 -04:00
FindRef.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Fix.hs fix potential race in updating inode cache 2021-07-27 13:08:08 -04:00
Forget.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
FromKey.hs fromkey: create directory for pointer files too 2021-05-07 11:10:06 -04:00
Fsck.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
FuzzTest.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
GCryptSetup.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Get.hs get: Improve output when failing to get a file fails 2021-01-29 15:11:19 -04:00
Group.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
GroupWanted.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Help.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Import.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
ImportFeed.hs incremental verification for web special remote 2021-08-18 15:02:22 -04:00
InAnnex.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Indirect.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Info.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Init.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
InitRemote.hs initremote --private 2021-04-23 14:47:46 -04:00
Inprogress.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
List.hs add "input" field to json output 2020-09-15 16:22:44 -04:00
Lock.hs fix a place where the inode cache could potentially have gotten stale 2021-07-26 14:12:58 -04:00
LockContent.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Log.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
LookupKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Map.hs avoid making absolute git remote path relative 2021-02-08 13:18:01 -04:00
MatchExpression.hs split ProvidedInfo and UserProvidedInfo 2020-09-28 12:12:38 -04:00
Merge.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
MetaData.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Migrate.hs include locked files in the keys database associated files 2021-05-21 16:24:37 -04:00
MinCopies.hs mincopies 2021-01-06 14:15:19 -04:00
Mirror.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
Move.hs dropping unused marks as dead 2021-06-25 15:22:26 -04:00
Multicast.hs Command/Multicast: use proper hyphen 2021-05-04 05:44:31 +02:00
NotifyChanges.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
NumCopies.hs mincopies 2021-01-06 14:15:19 -04:00
P2P.hs finish fixing removeLink on windows 2020-11-24 13:20:44 -04:00
P2PStdIO.hs finish CommandStart transition 2019-06-12 13:24:01 -04:00
PostReceive.hs remove direct mode remnant of merging unrelated histories 2021-07-19 11:41:26 -04:00
PreCommit.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Proxy.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
ReadPresentKey.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
RecvKey.hs make getViaTmpFrom no longer update location log 2020-12-11 11:50:13 -04:00
RegisterUrl.hs unregisterurl: New command 2021-03-01 14:28:24 -04:00
Reinit.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
Reinject.hs split verifyKeyContent 2021-07-29 13:58:40 -04:00
ReKey.hs include locked files in the keys database associated files 2021-05-21 16:24:37 -04:00
RemoteDaemon.hs assistant: Fix a crash on startup by avoiding using forkProcess 2021-05-12 15:08:03 -04:00
RenameRemote.hs cache remote.log 2020-09-22 13:52:26 -04:00
Repair.hs finished this stage of the RawFilePath conversion 2020-11-06 14:10:58 -04:00
Required.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
ResolveMerge.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
RmUrl.hs rmurl: remove all forms of an url, no matter what the downloader is set to 2021-03-22 12:09:15 -04:00
Schedule.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Semitrust.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
SendKey.hs use fastDebug everywhere it can be used 2021-04-06 15:41:24 -04:00
SetKey.hs annex.adjustedbranchrefresh 2020-11-16 14:27:28 -04:00
SetPresentKey.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Smudge.hs remove second pass in scanAnnexedFiles 2021-07-30 17:46:11 -04:00
Status.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
Sync.hs sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
Test.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
TestRemote.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
TransferInfo.hs more RawFilePath conversion 2020-11-02 16:31:28 -04:00
TransferKey.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
TransferKeys.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
Transferrer.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
Trust.hs fix format of message 2021-01-11 13:14:09 -04:00
Unannex.hs remove inode cache in unannex 2021-05-12 11:09:38 -04:00
Undo.hs finished this stage of the RawFilePath conversion 2020-11-04 14:20:37 -04:00
Ungroup.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Uninit.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Unlock.hs fix potential race in updating inode cache 2021-07-27 13:08:08 -04:00
UnregisterUrl.hs unregisterurl: remove all forms of an url, no matter what the downloader is set to 2021-03-22 12:17:17 -04:00
Untrust.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Unused.hs change message to suggest using whereused --historical 2021-07-14 16:08:47 -04:00
Upgrade.hs work around strange auto-init bug 2021-07-30 18:36:03 -04:00
VAdd.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
VCycle.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Version.hs external backends wip 2020-07-29 15:23:18 -04:00
VFilter.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Vicfg.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
View.hs more RawFilePath conversion 2020-10-30 15:55:59 -04:00
VPop.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Wanted.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Watch.hs remotedaemon: Don't list --stop in help since it's not supported. 2019-09-30 14:40:46 -04:00
WebApp.hs finished this stage of the RawFilePath conversion 2020-11-04 14:20:37 -04:00
Whereis.hs whereis: Don't include yt: prefix when showing url to content retrieved with youtube-dl 2021-03-22 19:56:24 -04:00
WhereUsed.hs better key matching with a regexp 2021-07-14 16:31:17 -04:00