mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/Command
History
Joey Hess 8bde6101e3
sqlite datbase for importfeed 
importfeed: Use caching database to avoid needing to list urls on every
run, and avoid using too much memory.

Benchmarking in my podcasts repo, importfeed got 1.42 seconds faster,
and memory use dropped from 203000k to 59408k.

Database.ImportFeed is Database.ContentIdentifier with the serial number
filed off. There is a bit of code duplication I would like to avoid,
particularly recordAnnexBranchTree, and getAnnexBranchTree. But these use
the persistent sqlite tables, so despite the code being the same, they
cannot be factored out.

Since this database includes the contentidentifier metadata, it will be
slightly redundant if a sqlite database is ever added for metadata. I
did consider making such a generic database and using it for this. But,
that would then need importfeed to update both the url database and the
metadata database, which is twice as much work diffing the git-annex
branch trees. Or would entagle updating two databases in a complex way.
So instead it seems better to optimise the database that
importfeed needs, and if the metadata database is used by another command,
use a little more disk space and do a little bit of redundant work to
update it.

Sponsored-by: unqueued on Patreon
2023-10-23 16:46:22 -04:00
..
Add.hs git-annex assist 2023-05-18 14:37:43 -04:00
AddUnused.hs addunused, dropunused: Support --json and --json-error-messages 2023-05-05 14:01:40 -04:00
AddUrl.hs split out Utility.Url.Parse 2023-08-14 12:28:10 -04:00
Adjust.hs rename to --unlock-present and better reverse adjusting 2020-11-13 14:56:43 -04:00
Assist.hs assist: With --jobs, parallelize transferring content to/from remotes 2023-07-05 12:47:30 -04:00
Assistant.hs git-annex assist 2023-05-18 14:37:43 -04:00
Benchmark.hs improve benchmark --databases 2019-11-21 17:25:20 -04:00
CalcKey.hs safe output to terminal for calckey inprogress and lookupkey 2023-04-12 14:03:44 -04:00
CheckPresentKey.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
Config.hs config: Added the --show-origin and --for-file options 2023-06-12 16:24:31 -04:00
ConfigList.hs don't frontload reconcileStaged in git-annex init 2022-11-18 13:58:47 -04:00
ConfigRemote.hs configremote: Support --json and --json-error-messages 2023-05-10 14:21:42 -04:00
ContentLocation.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Copy.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Dead.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
Describe.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
DiffDriver.hs fix some build warnings from ghc 9.4.6 2023-09-21 13:38:36 -04:00
Direct.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Drop.hs display drop from remote more consistently 2023-06-27 19:01:33 -04:00
DropKey.hs remove --backend from global options 2022-06-29 13:33:25 -04:00
DropUnused.hs dropunused: Support --jobs 2023-07-21 14:03:34 -04:00
EnableRemote.hs enableremote: Avoid overwriting existing git remote when passed the uuid of a specialremote that was earlier initialized with the same name 2023-09-22 13:29:48 -04:00
EnableTor.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
ExamineKey.hs fix mojibake reversion in display of utf8 2023-04-12 13:53:30 -04:00
Expire.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
Export.hs --explain for preferred/required content matching 2023-07-26 14:50:04 -04:00
FilterBranch.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
FilterProcess.hs smudge: Warn when encountering a pointer file that has other content appended to it 2022-02-23 15:17:08 -04:00
Find.hs avoid quoting spaces in git-annex find output to terminal 2023-04-26 00:18:30 -04:00
FindKeys.hs fix mojibake reversion in display of utf8 2023-04-12 13:53:30 -04:00
FindRef.hs remove --backend from global options 2022-06-29 13:33:25 -04:00
Fix.hs fix: Support --json and --json-error-messages 2023-05-04 14:28:21 -04:00
Forget.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
FromKey.hs split out Utility.Url.Parse 2023-08-14 12:28:10 -04:00
Fsck.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
FuzzTest.hs eliminate showStart showStartOther 2023-04-10 16:28:58 -04:00
GCryptSetup.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Get.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Group.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
GroupWanted.hs git style quoting for ActionItemOther 2023-04-08 16:30:01 -04:00
Help.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Import.hs use importChanges optimisation 2023-06-01 13:47:00 -04:00
ImportFeed.hs sqlite datbase for importfeed 2023-10-23 16:46:22 -04:00
InAnnex.hs add SeekInput (not yet used) 2020-09-15 15:41:13 -04:00
Indirect.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Info.hs info: Added available to the info displayed for a remote 2023-08-16 14:52:58 -04:00
Init.hs init: Support --json and --json-error-messages 2023-05-08 14:58:08 -04:00
InitRemote.hs remove unused import 2023-05-11 13:24:34 -04:00
Inprogress.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
List.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Lock.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Log.hs log: Support --json and --json-error-messages 2023-05-04 12:36:31 -04:00
LookupKey.hs lookupkey: Added --ref option 2023-09-12 12:49:11 -04:00
Map.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
MatchExpression.hs --explain for preferred/required content matching 2023-07-26 14:50:04 -04:00
Merge.hs sync: When in an adjusted branch, merge changes from the original branch 2023-07-06 12:42:24 -04:00
MetaData.hs factor out maybeAddJSONField 2023-05-08 16:15:41 -04:00
Migrate.hs migrate, undo: Support --json and --json-error-messages 2023-05-04 16:34:35 -04:00
MinCopies.hs git style quoting for ActionItemOther 2023-04-08 16:30:01 -04:00
Mirror.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Move.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Multicast.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
NotifyChanges.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
NumCopies.hs git style quoting for ActionItemOther 2023-04-08 16:30:01 -04:00
OldKeys.hs oldkeys: check associated files by default and add --unchecked 2023-08-23 13:46:41 -04:00
P2P.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
P2PStdIO.hs finish CommandStart transition 2019-06-12 13:24:01 -04:00
PostReceive.hs fix some build warnings from ghc 9.4.6 2023-09-21 13:40:22 -04:00
PreCommit.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
Proxy.hs Automatically convert direct mode repositories to v7 with adjusted unlocked branches 2019-08-26 15:05:25 -04:00
Pull.hs git-annex pull and push 2023-05-16 16:51:07 -04:00
Push.hs git-annex pull and push 2023-05-16 16:51:07 -04:00
ReadPresentKey.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
RecvKey.hs avoid flushing keys db queue after each Annex action 2022-10-12 14:12:23 -04:00
RegisterUrl.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Reinit.hs reinit: Support --json and --json-error-messages 2023-05-08 15:07:40 -04:00
Reinject.hs reinject: Added --guesskeys option 2023-06-26 14:05:31 -04:00
ReKey.hs rekey, setpresentkey: Support --json and --json-error-messages 2023-05-04 16:03:54 -04:00
RemoteDaemon.hs fix build with unix-2.8.0 2023-08-01 18:41:27 -04:00
RenameRemote.hs renameremote: Support --json and --json-error-messages 2023-05-08 16:25:40 -04:00
Repair.hs move several readonly values to AnnexRead 2022-06-28 15:40:19 -04:00
Required.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
ResolveMerge.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
Restage.hs restage: New git-annex command, handles restaging unlocked files 2022-09-23 16:29:59 -04:00
RmUrl.hs rmurl: Support --json and --json-error-messages 2023-05-04 11:28:27 -04:00
Satisfy.hs added git-annex satisfy 2023-06-29 15:34:53 -04:00
Schedule.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
Semitrust.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
SendKey.hs addurl, importfeed: Fix failure when annex.securehashesonly is set 2023-03-27 15:10:46 -04:00
SetKey.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
SetPresentKey.hs rekey, setpresentkey: Support --json and --json-error-messages 2023-05-04 16:03:54 -04:00
Smudge.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Status.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
Sync.hs propigateAdjustedCommits in seekExportContent 2023-09-11 14:54:26 -04:00
Test.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
TestRemote.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
TransferKey.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
TransferKeys.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Transferrer.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
Trust.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
Unannex.hs unannex: Support --json and --json-error-messages 2023-05-03 15:56:20 -04:00
Undo.hs migrate, undo: Support --json and --json-error-messages 2023-05-04 16:34:35 -04:00
Ungroup.hs git style quoting for ActionItemOther 2023-04-08 16:30:01 -04:00
Uninit.hs fix empty tree import when directory does not exist 2023-08-15 12:57:41 -04:00
Unlock.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
UnregisterUrl.hs restore old registerurl location tracking behavior 2023-04-05 17:06:44 -04:00
Untrust.hs expire, trust et al, dead, describe: Support --json and --json-error-messages 2023-05-05 15:33:30 -04:00
Unused.hs remove unused imports 2023-05-16 16:33:02 -04:00
Upgrade.hs upgrade: Support --json and --json-error-messages and --json-progress 2023-05-10 12:54:48 -04:00
VAdd.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
VCycle.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Version.hs version: Avoid error message when entire output is not read 2023-05-19 15:00:57 -04:00
VFilter.hs understand adjusted view branch names 2023-02-27 14:57:58 -04:00
Vicfg.hs vicfg: Include mincopies configuration 2022-09-15 15:11:59 -04:00
View.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
VPop.hs git style quoting for ActionItemOther 2023-04-08 16:30:01 -04:00
Wanted.hs filter out control characters and quote filenames 2023-04-11 14:27:22 -04:00
Watch.hs git-annex assist 2023-05-18 14:37:43 -04:00
WebApp.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Whereis.hs json object for FileNotFound 2023-04-25 19:26:20 -04:00
WhereUsed.hs whereused: Fix display of branch:file when run in a subdirectory 2023-04-12 15:18:04 -04:00