579d9b60c1
Use separate stages for download and upload. In the common case where it downloads the file from one remote and then uploads to the other, those are by far the most expensive operations, and there's a decent chance the two remotes bottleneck on different resources. Suppose it's being run with -J2 and a bunch of 10 mb files. Two threads will be started both downloading from the src remote. They will probably finish at the same time. Then two threads will be started uploading to the dst remote. They will probably take the same time as well. Before this change, it would alternate back and forth, bottlenecking on src and dst. With this change, as soon as the two threads start uploading to dst, two more threads are able to start, downloading from src. So bandwidth to both remotes is saturated more often. Other commands that use transferStages only send in one direction at a time. So the worker threads for the other direction will sit idle, and there will be no change in their behavior. Sponsored-by: Dartmouth College's DANDI project
25 lines
588 B
Haskell
25 lines
588 B
Haskell
{- git-annex transfer direction types
|
|
-
|
|
- Copyright 2012 Joey Hess <id@joeyh.name>
|
|
-
|
|
- Licensed under the GNU AGPL version 3 or higher.
|
|
-}
|
|
|
|
{-# LANGUAGE OverloadedStrings #-}
|
|
|
|
module Types.Direction where
|
|
|
|
import qualified Data.ByteString as B
|
|
|
|
data Direction = Upload | Download
|
|
deriving (Eq, Ord, Show, Read)
|
|
|
|
formatDirection :: Direction -> B.ByteString
|
|
formatDirection Upload = "upload"
|
|
formatDirection Download = "download"
|
|
|
|
parseDirection :: String -> Maybe Direction
|
|
parseDirection "upload" = Just Upload
|
|
parseDirection "download" = Just Download
|
|
parseDirection _ = Nothing
|
|
|