From d5cd1de2803725308eff1778cb053d03b52b5abf Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 12 Oct 2022 15:53:56 -0400 Subject: [PATCH] update and open a todo about something I'm pondering --- ..._d02b310b007b304e0eda23bd5e565851._comment | 16 +++++++++++++++ .../withExclusiveLock_blocking_issue.mdwn | 20 +++++++++++++++++++ 2 files changed, 36 insertions(+) create mode 100644 doc/bugs/get_is_busy_doing_nothing/comment_26_d02b310b007b304e0eda23bd5e565851._comment create mode 100644 doc/todo/withExclusiveLock_blocking_issue.mdwn diff --git a/doc/bugs/get_is_busy_doing_nothing/comment_26_d02b310b007b304e0eda23bd5e565851._comment b/doc/bugs/get_is_busy_doing_nothing/comment_26_d02b310b007b304e0eda23bd5e565851._comment new file mode 100644 index 0000000000..3aafff13ff --- /dev/null +++ b/doc/bugs/get_is_busy_doing_nothing/comment_26_d02b310b007b304e0eda23bd5e565851._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 26""" + date="2022-10-12T19:34:05Z" + content=""" +I've avoided the excessive sqlite database writes. Which doubled the speed +of git-annex in some circumstances, wow! + +@yoh see if it still happens once you upgrade to a git-annex +with [[!commit 6fbd337e34cd62881ed5e21616e9c00d7bf35378]]. + +It would be possible for git-annex to do its own locking around writes +to the sqlite database. That would surely avoid any problem that sqlite might +have that would cause ErrorBusy. I want to think some about +[[todo/withExclusiveLock_blocking_issue]] first. +"""]] diff --git a/doc/todo/withExclusiveLock_blocking_issue.mdwn b/doc/todo/withExclusiveLock_blocking_issue.mdwn new file mode 100644 index 0000000000..6915015da3 --- /dev/null +++ b/doc/todo/withExclusiveLock_blocking_issue.mdwn @@ -0,0 +1,20 @@ +Some parts of git-annex use withExclusiveLock or otherwise wait for an +exclusive lock and hold it while performing an operation. Now consider what +happens if the git-annex process is suspended. Another git-annex process +that is running and that blocks on the same lock will stall forever, until +the git-annex process is resumed. + +These time windows tend to be small, but may not always be. + +Would it be better for the second git-annex process, rather than hanging +indefinitely, to try to take the lock a few times over a few seconds, and +then error out? The risk with doing that is, when 2 concurrent git-annex +processes are running and taking the locks repeatedly, one might get +unlucky, fail to take the lock, and error out, when waiting a little longer +would have succeeded, because the other process is not holding the lock all +the time. + +Is there any better way git-annex could handle this? Is it a significant +problem at all? I don't think I've ever seen it happen, but I rarely ^Z +git-annex either. How do other programs handle this, if at all? +--[[Joey]]