8bde6101e3
importfeed: Use caching database to avoid needing to list urls on every run, and avoid using too much memory. Benchmarking in my podcasts repo, importfeed got 1.42 seconds faster, and memory use dropped from 203000k to 59408k. Database.ImportFeed is Database.ContentIdentifier with the serial number filed off. There is a bit of code duplication I would like to avoid, particularly recordAnnexBranchTree, and getAnnexBranchTree. But these use the persistent sqlite tables, so despite the code being the same, they cannot be factored out. Since this database includes the contentidentifier metadata, it will be slightly redundant if a sqlite database is ever added for metadata. I did consider making such a generic database and using it for this. But, that would then need importfeed to update both the url database and the metadata database, which is twice as much work diffing the git-annex branch trees. Or would entagle updating two databases in a complex way. So instead it seems better to optimise the database that importfeed needs, and if the metadata database is used by another command, use a little more disk space and do a little bit of redundant work to update it. Sponsored-by: unqueued on Patreon
67 lines
1.5 KiB
Haskell
67 lines
1.5 KiB
Haskell
{- git-annex metadata, standard fields
|
|
-
|
|
- Copyright 2014 Joey Hess <id@joeyh.name>
|
|
-
|
|
- Licensed under the GNU AGPL version 3 or higher.
|
|
-}
|
|
|
|
{-# LANGUAGE OverloadedStrings #-}
|
|
|
|
module Annex.MetaData.StandardFields (
|
|
tagMetaField,
|
|
yearMetaField,
|
|
monthMetaField,
|
|
dayMetaField,
|
|
isDateMetaField,
|
|
lastChangedField,
|
|
mkLastChangedField,
|
|
isLastChangedField,
|
|
itemIdField
|
|
) where
|
|
|
|
import Types.MetaData
|
|
|
|
import qualified Data.Text as T
|
|
import Data.Monoid
|
|
import Prelude
|
|
|
|
tagMetaField :: MetaField
|
|
tagMetaField = mkMetaFieldUnchecked "tag"
|
|
|
|
yearMetaField :: MetaField
|
|
yearMetaField = mkMetaFieldUnchecked "year"
|
|
|
|
monthMetaField :: MetaField
|
|
monthMetaField = mkMetaFieldUnchecked "month"
|
|
|
|
dayMetaField :: MetaField
|
|
dayMetaField = mkMetaFieldUnchecked "day"
|
|
|
|
isDateMetaField :: MetaField -> Bool
|
|
isDateMetaField f
|
|
| f == yearMetaField = True
|
|
| f == monthMetaField = True
|
|
| f == dayMetaField = True
|
|
| otherwise = False
|
|
|
|
lastChangedField :: MetaField
|
|
lastChangedField = mkMetaFieldUnchecked lastchanged
|
|
|
|
mkLastChangedField :: MetaField -> MetaField
|
|
mkLastChangedField f = mkMetaFieldUnchecked (fromMetaField f <> lastchangedSuffix)
|
|
|
|
isLastChangedField :: MetaField -> Bool
|
|
isLastChangedField f
|
|
| f == lastChangedField = True
|
|
| otherwise = lastchanged `T.isSuffixOf` s && s /= lastchangedSuffix
|
|
where
|
|
s = fromMetaField f
|
|
|
|
lastchanged :: T.Text
|
|
lastchanged = "lastchanged"
|
|
|
|
lastchangedSuffix :: T.Text
|
|
lastchangedSuffix = "-lastchanged"
|
|
|
|
itemIdField :: MetaField
|
|
itemIdField = mkMetaFieldUnchecked "itemid"
|