git-annex/Annex
Joey Hess c2634e7df2
automatically adjust stall detection period
Improve annex.stalldetection to handle remotes that update progress less
frequently than the configured time period.

In particular, this makes remotes that don't report progress but are
chunked work when transferring a single chunk takes longer than the
specified time period.

Any remotes that just have very low update granulatity would also be
handled by this.

The change to Remote.Helper.Chunked avoids an extra progress update when
resuming an interrupted upload. In that case, the code saw first Nothing
and then Just the already transferred number of bytes, which defeated this
new heuristic. This change will mean that, when resuming an interrupted
upload to a chunked remote that does not do its own progress reporting, the
progress display does not start out displaying the amount sent so far,
until after the first chunk is sent. This behavior change does not seem
like a major problem.

About the scalefudgefactor, it seems reasonable to expect subsequent chunks
to take no more than 1.5 times as long as the first chunk to transfer.
Could set it to 1, but then any chunk taking a little longer would be
treated as a stall. 2 also seems a likely value. Even 10 might be fine?

Sponsored-by: Dartmouth College's DANDI project
2024-01-18 17:12:10 -04:00
..
AdjustedBranch fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Branch newtype MapLog 2023-11-13 14:37:22 -04:00
Concurrent differentiate between concurrency enabled at command line and by git config 2020-09-16 11:47:12 -04:00
Content disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
Debug implement fastDebug 2021-04-06 15:24:28 -04:00
LockPool avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
MetaData sqlite datbase for importfeed 2023-10-23 16:46:22 -04:00
SpecialRemote httpalso: Support being used with special remotes that use chunking. 2023-06-20 13:35:28 -04:00
VectorClock Lower precision of timestamps in git-annex branch 2023-12-11 15:04:06 -04:00
View annex.maxextensionlength for view 2023-03-24 14:01:38 -04:00
Action.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
AdjustedBranch.hs propigateAdjustedCommits: avoid overwriting diverged original branch 2023-07-05 17:09:49 -04:00
AutoMerge.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
BloomFilter.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Branch.hs started migrate --update 2023-12-07 15:50:52 -04:00
BranchState.hs disable journalIgnorable in enableInteractiveBranchAccess 2022-07-15 13:48:41 -04:00
CatFile.hs read a consistent amount from pointer file 2022-02-23 12:52:34 -04:00
ChangedRefs.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
CheckAttr.hs config: Added the --show-origin and --for-file options 2023-06-12 16:24:31 -04:00
CheckIgnore.hs move several readonly values to AnnexRead 2022-06-28 15:40:19 -04:00
Common.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
Concurrent.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Content.hs disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
CopyFile.hs Copy with a reflink when exporting a tree to a directory special remote 2023-03-28 13:09:14 -04:00
CurrentBranch.hs refactor getCurrentBranch 2018-10-19 17:29:18 -04:00
Debug.hs display explanations in --debug too 2023-07-31 13:06:40 -04:00
Difference.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
DirHashes.hs Added http special remote, which is useful for accessing other remotes that publish content stored in them via http/https. 2020-09-01 15:16:35 -04:00
Drop.hs prevent numcopies or mincopies being configured to 0 2022-03-28 15:20:34 -04:00
Environment.hs improve comments 2023-04-04 15:23:39 -04:00
Export.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
ExternalAddonProcess.hs use fastDebug everywhere it can be used 2021-04-06 15:41:24 -04:00
FileMatcher.hs importtree: support preferred content expressions needing keys 2023-12-18 16:27:59 -04:00
Fixup.hs fix a bug that prevented git-annex init from working in a submodule 2021-01-21 15:33:15 -04:00
GitOverlay.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
HashObject.hs use ResourcePool for hash-object handles 2022-07-25 17:32:39 -04:00
Hook.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Import.hs disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
Ingest.hs log migration trees to git-annex branch 2023-12-06 15:40:03 -04:00
Init.hs Fix breakage when git is configured with safe.bareRepository = explicit 2023-07-05 14:43:14 -04:00
InodeSentinal.hs fix perms for core.sharedRepository 2023-04-26 16:29:11 -04:00
Journal.hs create journal directory in withJournalHandle 2023-06-21 15:23:59 -04:00
Link.hs fix hang when built with unix-2.8 2023-08-01 20:22:28 -04:00
Locations.hs started migrate --update 2023-12-07 15:50:52 -04:00
LockFile.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
LockPool.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Magic.hs Serialize use of C magic library, which is not thread safe. 2020-09-17 17:27:42 -04:00
MetaData.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Multicast.hs use programPath consistently, not readProgramFile 2020-03-30 16:06:27 -04:00
Notification.hs fix build when dbus is enabled 2022-07-05 13:06:45 -04:00
NumCopies.hs improve message about 1 copy 2023-12-04 11:12:54 -04:00
Path.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Perms.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
PidLock.hs fix windows build 2022-09-26 12:08:04 -04:00
Queue.hs add restage log 2022-09-23 15:47:24 -04:00
RemoteTrackingBranch.hs refactor 2019-11-11 19:10:52 -04:00
ReplaceFile.hs push RawFilePath down into Annex.ReplaceFile 2023-10-26 13:36:49 -04:00
SpecialRemote.hs init: Avoid autoenabling special remotes that have control characters in their names 2023-04-12 12:37:12 -04:00
Ssh.hs avoid annexFileMode special case 2023-04-27 15:58:37 -04:00
StallDetection.hs automatically adjust stall detection period 2024-01-18 17:12:10 -04:00
TaggedPush.hs simplify base64 to only use ByteString 2023-10-26 13:10:05 -04:00
Tmp.hs fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Transfer.hs disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
TransferrerPool.hs avoid build warning on windows 2023-03-27 12:19:26 -04:00
UntrustedFilePath.hs fix mojibake reversion in display of utf8 2023-04-12 13:53:30 -04:00
UpdateInstead.hs v7 for all repositories 2019-08-30 14:09:14 -04:00
Url.hs avoid unncessary use of curl when conduit will do 2023-08-22 10:25:53 -04:00
UUID.hs eliminate s2w8 and w82s 2023-10-26 13:12:57 -04:00
VariantFile.hs more RawFilePath 2019-12-18 17:10:28 -04:00
VectorClock.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Verify.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Version.hs v8 repositories automatically upgrade to v9 2022-07-25 16:20:04 -04:00
View.hs annex.maxextensionlength for view 2023-03-24 14:01:38 -04:00
Wanted.hs new matching options --want-get-by and --want-drop-by 2022-07-28 13:26:03 -04:00
WorkerPool.hs start splitting out readonly values from AnnexState 2021-04-02 15:51:44 -04:00
WorkTree.hs use lookupKeyStaged in --batch code paths 2022-10-26 14:43:06 -04:00
YoutubeDl.hs nub list of files 2023-07-09 14:18:25 -04:00