Commit graph

40191 commits

Author SHA1 Message Date
Joey Hess
e147ae07f4
remove supportUnlocked check that is not worth its overhead
moveAnnex only gets to that check if the object file was not present
before. So in the case where dup files are being added repeatedly,
it will only run the first time, and so there's no significant speedup
from doing it; all it avoids is a single sqlite lookup. Since MVar
accesses do have overhead, it's better to optimise for the common case,
where unlocked files are supported.

removeAnnex is less clear cut, but I think mostly is skipped running on
keys when the object has already been dropped, so similar reasoning
applies.
2021-06-15 09:28:56 -04:00
9qf@758d7b174d81a134727acab9db0168c8f0782b3a
85b2dbce32 2021-06-15 12:21:19 +00:00
Joey Hess
6099edbf1c
bloom doesn't work, but this should I hope 2021-06-14 17:53:01 -04:00
Joey Hess
2df4c1cf91
plan 2021-06-14 17:13:37 -04:00
Joey Hess
0e3802c7ee
comment 2021-06-14 15:11:09 -04:00
Joey Hess
643dc36e37
going round and round, boredly 2021-06-14 14:37:06 -04:00
Joey Hess
fa6e8fc660
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-14 14:34:35 -04:00
Joey Hess
711252331e
comment 2021-06-14 14:34:22 -04:00
Joey Hess
398f9decd4
comment 2021-06-14 14:32:38 -04:00
Joey Hess
78da00c7a6
Future proof activity log parsing
When the log has an activity that is not known, eg added by a future
version of git-annex, it used to be treated as no activity at all,
which would make git-annex expire think it should expire the repository,
despite it having some kind of recent activity.

Hopefully there will be no reason to add a new activity until enough
time has passed that this commit is in use everywhere.

Sponsored-by: Jake Vosloo on Patreon
2021-06-14 14:18:19 -04:00
yarikoptic
6043a2c7a0 Added a comment 2021-06-14 17:36:16 +00:00
james@06209b7878fcf3b5c46b8028dacb3cec6609369c
9d34a9d013 2021-06-14 17:19:50 +00:00
Joey Hess
372ace599a
comment 2021-06-14 13:13:46 -04:00
Joey Hess
f0cbaa194c
improve docs based on forum feedback 2021-06-14 13:04:58 -04:00
Joey Hess
fbd2f96b2c
comment 2021-06-14 12:56:29 -04:00
Joey Hess
dcd2c95249
fix windows build 2021-06-14 12:43:26 -04:00
Joey Hess
3ac9363c03
comment 2021-06-14 12:42:11 -04:00
Joey Hess
014dc63a55
avoid sometimes expensive operations when annex.supportunlocked = false
This will mostly just avoid a DB lookup, so things get marginally
faster. But in cases where there are many files using the same key, it
can be a more significant speedup.

Added overhead is one MVar lookup per call, which should be small
enough, since this happens after transferring or ingesting a file,
which is always a lot more work than that. It would be nice, though,
to move getGitConfig to AnnexRead, which there is an open todo about.
2021-06-14 12:40:41 -04:00
Joey Hess
a02b5c2904
response 2021-06-14 12:36:42 -04:00
yarikoptic
51fede57a2 Added a comment 2021-06-14 16:23:41 +00:00
Ilya_Shlyakhter
35afd58a76 Added a comment: git-annex-add slowdown 2021-06-14 16:00:44 +00:00
Joey Hess
c4f1465a81
check symlink before reading file
This is faster because when multiple files are in a directory, it gets
cached.
2021-06-14 11:53:51 -04:00
Joey Hess
4163344ed6
retitle 2021-06-14 11:44:55 -04:00
Joey Hess
0eff5a3f71
reproduced 2021-06-14 11:37:21 -04:00
Joey Hess
26a9ea12d1
handle edge case of symlink to something that is not really a pointer file
That seems very unlikely to happen, but still, it's possible it could.
And with the recent addition of locked files to the keys db, this could
be called by places that did not call it before, so it seems even more
important it's correct.

Adds an extra stat of the file, and is potentially racy, but both
problems are fixed by the unix-2.8.0 path. I have not tested that path
builds because that package is not yet released and it would be difficult
to install it since it's tightly tied to a ghc version.
2021-06-14 11:35:52 -04:00
Joey Hess
673b2feaf3
rename for clarity
Associated files are recorded now also for locked files, but this is
only needed to populate unlocked files.
2021-06-14 10:55:24 -04:00
yarikoptic
8f66f73fea Added a comment 2021-06-09 22:28:06 +00:00
yarikoptic
e30f973323 Added a comment: more "mystery resolved" -- identical (empty) keys 2021-06-09 21:00:34 +00:00
Joey Hess
4b09b93a18
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-09 15:38:58 -04:00
Joey Hess
fad281767a
comment 2021-06-09 15:38:55 -04:00
yarikoptic
714d9f1315 Added a comment 2021-06-08 22:02:34 +00:00
yarikoptic
a8fb61329d Added a comment 2021-06-08 21:58:20 +00:00
yarikoptic
3985ae3224 Added a comment: OSX mystery resolved. add --batch is effective mitigation 2021-06-08 21:56:53 +00:00
Joey Hess
6cb9113ff5
comments 2021-06-08 17:38:56 -04:00
yarikoptic
c3993a2655 Added a comment 2021-06-08 20:23:09 +00:00
yarikoptic
437d9366b7 Added a comment: getting closer... 2021-06-08 19:21:59 +00:00
Ilya_Shlyakhter
be4a029e1b Added a comment 2021-06-08 19:08:01 +00:00
jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421
be173f213d 2021-06-08 18:40:09 +00:00
jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421
e4cf6cc306 removed 2021-06-08 18:26:30 +00:00
Joey Hess
530c957c3e
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-08 12:52:08 -04:00
yarikoptic
697921ecd8 Added a comment: all recent builds/logs are fetched to smaug 2021-06-08 16:50:12 +00:00
Joey Hess
7b6deb1109
display scanning message whenever reconcileStaged has enough files to chew on
Clear visible progress bar first.

Removed showSideActionAfter because it can't be used in reconcileStaged
(import loop). Instead, it counts the number of files it
processes and displays it after it's seen a sufficient to know it's
taking a while.

Sponsored-by: Dartmouth College's Datalad project
2021-06-08 12:48:30 -04:00
Joey Hess
ecbaa52571
clarification 2021-06-08 12:00:01 -04:00
Joey Hess
1a6fa5abc8
add debugging for reconcileStaged calls for benchmarking 2021-06-08 11:57:23 -04:00
Joey Hess
13b9a288d3
scanAnnexedFiles in smudge --update
This makes git checkout and git merge hooks do the work to catch up with
changes that they made to the tree. Rather than doing it at some later
point when the user is not thinking about that past operation.

Sponsored-by: Dartmouth College's Datalad project
2021-06-08 11:37:47 -04:00
Joey Hess
c380687aa3
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-08 11:13:09 -04:00
Joey Hess
7f742589f9
claw back annexed file scan speedup
Following commit c941ab6f5b, this avoids
the second, redundant scan when annex.thin is not set.

The benchmark now runs in 35.5 seconds, down from 40 seconds.

Note that the inode cache of the annex object has to be passed to
addInodeCaches now, because it might not already be in the inode caches,
unlike previously.

Sponsored-by: Dartmouth College's Datalad project
2021-06-08 11:09:15 -04:00
Joey Hess
ec1f2f246b
improve comment
remove obsolete part about a commit preventing it seeing changes
2021-06-08 10:43:48 -04:00
yarikoptic
62758ffb9f Added a comment: slow down is OSX specific 2021-06-08 14:28:18 +00:00
Joey Hess
d12120739d
comment 2021-06-08 10:19:04 -04:00