mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

convert Key to ShortByteString

Browse source 
This adds the overhead of a copy when serializing and deserializing keys.
I have not benchmarked much, but runtimes seem barely changed at all by that.

When a lot of keys are in memory, it improves memory use.

And, it prevents keys sometimes getting PINNED in memory and failing to GC,
which is a problem ByteString has sometimes. In particular, git-annex sync
from a borg special remote had that problem and this improved its memory
use by a large amount.

Sponsored-by: Shae Erisson on Patreon
This commit is contained in:
Joey Hess 2021-10-05 20:20:08 -04:00
parent 012b71e471
commit 19e78816f0
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
15 changed files with 65 additions and 36 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Remote/External/Types.hs vendored
View file

@ -60,6 +60,7 @@ import Control.Concurrent.STM
import Network.URI
import Data.Char
import Text.Read
import qualified Data.ByteString.Short as S (fromShort)
data External = External
{ externalType :: ExternalType
@ -138,7 +139,7 @@ newtype SafeKey = SafeKey Key
mkSafeKey :: Key -> Either String SafeKey
mkSafeKey k
| any isSpace (decodeBS $ fromKey keyName k) = Left $ concat
| any isSpace (decodeBS $ S.fromShort $ fromKey keyName k) = Left $ concat
[ "Sorry, this file cannot be stored on an external special remote because its key's name contains a space. "
, "To avoid this problem, you can run: git-annex migrate --backend="
, decodeBS (formatKeyVariety (fromKey keyVariety k))