Commit graph

10626 commits

Author SHA1 Message Date
Joey Hess
0966bd2de4
comment 2021-06-25 13:33:27 -04:00
Joey Hess
df2001aa88
Improve display of errors when transfers fail
Transfers from or to a local git repo could fail without a reason being
given, if the content failed to verify, or if the object file's stat
changed while it was being copied. Now display messages in these cases.

Sponsored-by: Jack Hill on Patreon
2021-06-25 13:17:04 -04:00
Joey Hess
0fe550af75
fix windows build 2021-06-22 09:46:06 -04:00
jwodder
4b93e933db 2021-06-22 13:21:17 +00:00
Joey Hess
a6a6217322
remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before 2020.

Except for ones tagged projects/* since projects like datalad want to keep
around records of old deleted bugs longer.

Command line used:

for f in $(grep -l '|done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2020 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2020 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
for f in $(grep -l '\[\[done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2020 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2020 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
2021-06-21 13:10:13 -04:00
falsifian
95f685f2ca Added a comment 2021-06-19 23:48:21 +00:00
falsifian
a73624b161 Added a comment 2021-06-19 23:41:25 +00:00
yarikoptic
538780ebaf Added a comment 2021-06-16 21:32:58 +00:00
Ilya_Shlyakhter
afb8c5f078 Added a comment: thanks 2021-06-16 19:54:02 +00:00
yarikoptic
9dfda0899a removed 2021-06-15 15:38:47 +00:00
yarikoptic
35371af3da Added a comment 2021-06-15 15:38:28 +00:00
yarikoptic
5ac6583621 Added a comment 2021-06-15 15:38:16 +00:00
Joey Hess
3af4c9a29a
fix exponential blowup when adding lots of identical files
This was an old problem when the files were being added unlocked,
so the changelog mentions that being fixed. However, recently it's also
affected locked files.

The fix for locked files is kind of stupidly simple. moveAnnex already
handles populating unlocked files, and only does it when the object file
was not already present. So remove the redundant populateUnlockedFiles
call. (That call was added all the way back in
cfaac52b88, and has always been
unncessary.)

Sponsored-by: Dartmouth College's Datalad project
2021-06-15 09:45:55 -04:00
Joey Hess
e147ae07f4
remove supportUnlocked check that is not worth its overhead
moveAnnex only gets to that check if the object file was not present
before. So in the case where dup files are being added repeatedly,
it will only run the first time, and so there's no significant speedup
from doing it; all it avoids is a single sqlite lookup. Since MVar
accesses do have overhead, it's better to optimise for the common case,
where unlocked files are supported.

removeAnnex is less clear cut, but I think mostly is skipped running on
keys when the object has already been dropped, so similar reasoning
applies.
2021-06-15 09:28:56 -04:00
Joey Hess
6099edbf1c
bloom doesn't work, but this should I hope 2021-06-14 17:53:01 -04:00
Joey Hess
2df4c1cf91
plan 2021-06-14 17:13:37 -04:00
Joey Hess
643dc36e37
going round and round, boredly 2021-06-14 14:37:06 -04:00
yarikoptic
6043a2c7a0 Added a comment 2021-06-14 17:36:16 +00:00
Joey Hess
372ace599a
comment 2021-06-14 13:13:46 -04:00
Joey Hess
dcd2c95249
fix windows build 2021-06-14 12:43:26 -04:00
Joey Hess
014dc63a55
avoid sometimes expensive operations when annex.supportunlocked = false
This will mostly just avoid a DB lookup, so things get marginally
faster. But in cases where there are many files using the same key, it
can be a more significant speedup.

Added overhead is one MVar lookup per call, which should be small
enough, since this happens after transferring or ingesting a file,
which is always a lot more work than that. It would be nice, though,
to move getGitConfig to AnnexRead, which there is an open todo about.
2021-06-14 12:40:41 -04:00
Joey Hess
a02b5c2904
response 2021-06-14 12:36:42 -04:00
yarikoptic
51fede57a2 Added a comment 2021-06-14 16:23:41 +00:00
Ilya_Shlyakhter
35afd58a76 Added a comment: git-annex-add slowdown 2021-06-14 16:00:44 +00:00
Joey Hess
c4f1465a81
check symlink before reading file
This is faster because when multiple files are in a directory, it gets
cached.
2021-06-14 11:53:51 -04:00
Joey Hess
4163344ed6
retitle 2021-06-14 11:44:55 -04:00
Joey Hess
0eff5a3f71
reproduced 2021-06-14 11:37:21 -04:00
yarikoptic
8f66f73fea Added a comment 2021-06-09 22:28:06 +00:00
yarikoptic
e30f973323 Added a comment: more "mystery resolved" -- identical (empty) keys 2021-06-09 21:00:34 +00:00
Joey Hess
4b09b93a18
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-09 15:38:58 -04:00
Joey Hess
fad281767a
comment 2021-06-09 15:38:55 -04:00
yarikoptic
714d9f1315 Added a comment 2021-06-08 22:02:34 +00:00
yarikoptic
a8fb61329d Added a comment 2021-06-08 21:58:20 +00:00
yarikoptic
3985ae3224 Added a comment: OSX mystery resolved. add --batch is effective mitigation 2021-06-08 21:56:53 +00:00
Joey Hess
6cb9113ff5
comments 2021-06-08 17:38:56 -04:00
yarikoptic
c3993a2655 Added a comment 2021-06-08 20:23:09 +00:00
yarikoptic
437d9366b7 Added a comment: getting closer... 2021-06-08 19:21:59 +00:00
Ilya_Shlyakhter
be4a029e1b Added a comment 2021-06-08 19:08:01 +00:00
yarikoptic
697921ecd8 Added a comment: all recent builds/logs are fetched to smaug 2021-06-08 16:50:12 +00:00
Joey Hess
ecbaa52571
clarification 2021-06-08 12:00:01 -04:00
Joey Hess
1a6fa5abc8
add debugging for reconcileStaged calls for benchmarking 2021-06-08 11:57:23 -04:00
yarikoptic
62758ffb9f Added a comment: slow down is OSX specific 2021-06-08 14:28:18 +00:00
Joey Hess
d12120739d
comment 2021-06-08 10:19:04 -04:00
yarikoptic
57b567ac87 Added a comment 2021-06-07 21:39:05 +00:00
yarikoptic
2ffb9cc01b Added a comment: clarification 2021-06-07 21:20:35 +00:00
Joey Hess
e9a8b48a52
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-07 17:02:15 -04:00
Joey Hess
0f10f208a7
avoid double work in git-annex init
reconcileStaged was doing a redundant scan to scannAnnexedFiles.

It would probably make sense to move the body of scannAnnexedFiles
into reconcileStaged, the separation does not really serve any purpose.

Sponsored-by: Dartmouth College's Datalad project
2021-06-07 16:50:14 -04:00
Ilya_Shlyakhter
bdf3c06401 Added a comment: deferring the scan 2021-06-07 17:41:45 +00:00
Joey Hess
0101363eb8
correctly update keys db in merge conflict
This is quite a subtle edge case, see the bug report for full details.

The second git diff is needed only when there's a merge conflict.
It would be possible to speed it up marginally by using
--diff-filter=Unmerged, but probably not enough to bother with.

Sponsored-by: Graham Spencer on Patreon
2021-06-07 12:52:36 -04:00
Joey Hess
da24034331
comment 2021-06-07 11:53:25 -04:00