Commit graph

13 commits

Author SHA1 Message Date
Joey Hess
47084b8a1d
enable filter.annex.process in v9
This has tradeoffs, but is generally a win, and users who it causes git add to
slow down unacceptably for can just disable it again.

It needed to happen in an upgrade, since there are git-annex versions
that do not support it, and using such an old version with a v8
repository with filter.annex.process set will cause bad behavior.
By enabling it in v9, it's guaranteed that any git-annex version that
can use the repository does support it. Although, this is not a perfect
protection against problems, since an old git-annex version, if it's
used with a v9 repository, will cause git add to try to run
git-annex filter-process, which will fail. But at least, the user is
unlikely to have an old git-annex in path if they are using a v9
repository, since it won't work in that repository.

Sponsored-by: Dartmouth College's Datalad project
2022-01-21 13:11:18 -04:00
Joey Hess
9121154a75
new todo 2021-11-09 15:52:17 -04:00
Joey Hess
a0758bdd10
dynamically disable filter-process in restagePointerFile when it would be slower
Based on my earlier benchmark, I have a rough cost model for how
expensive it is for git-annex smudge to be run on a file, vs
how expensive it is for a gigabyte of a file's content to be read and
piped through to filter-process.

So, using that cost model, it can decide if using filter-process will
be more or less expensive than running the smudge filter on the files to
be restaged.

It turned out to be *really* annoying to temporarily disable
filter-process. I did find a way, but urk, this is horrible. Notice
that, if it's interrupted with it disabled, it will remain disabled
until the next time restagePointerFile runs. Which could be some time
later. If the user runs `git add` or `git checkout` on a lot of small
files before that, they will see slower than expected performance.

(This commit also deletes where I wrote down the benchmark results
earlier.)

Sponsored-by: Noam Kremen on Patreon
2021-11-08 16:20:34 -04:00
Joey Hess
054c803f8d
benchmarking of filter-process vs smudge/clean
No firm conclusions yet, but it's doing better than I would have
expected.

Sponsored-by: Graham Spencer on Patreon
2021-11-05 13:37:53 -04:00
Joey Hess
099e8fe061
close 2021-11-05 12:46:56 -04:00
Joey Hess
b25a138e22
update for git-annex filter-process 2021-11-04 15:15:26 -04:00
Joey Hess
bf1408f7bf
long-running-smudge branch started 2021-11-03 15:44:05 -04:00
Joey Hess
38ba8cca1b
investigation results
Also, close dup bug.
2021-11-02 15:06:20 -04:00
Joey Hess
3093e8fcc3
thought 2021-08-27 00:59:24 -04:00
Joey Hess
cffa2446e8
tagged the past 2 years of open todos and followed up to a few of them
also moved some that were really bug reports to bugs/ and closed a
couple
2020-01-30 15:22:05 -04:00
Joey Hess
4cf9daf8b9
remove obsolete warning 2018-12-01 14:04:38 -04:00
Joey Hess
6bd67d4d46
cherry-pick too 2018-10-22 16:24:14 -04:00
Joey Hess
4ba9c6608e
add for later 2018-10-22 13:00:42 -04:00