Commit graph

10713 commits

Author SHA1 Message Date
Joey Hess
2018cb4a97
comment 2021-07-12 16:00:59 -04:00
yarikoptic
d26c28c5f3 Added a comment 2021-07-12 17:44:58 +00:00
Joey Hess
a6767ca81f
close bug
and mention another aspect of the reversion in changelog
2021-07-12 10:45:57 -04:00
Joey Hess
5faf84c3a7
close as dup 2021-07-12 10:32:19 -04:00
Joey Hess
b2ba8c8aa4
Merge branch 'master' of ssh://git-annex.branchable.com 2021-07-12 10:24:59 -04:00
Joey Hess
0ea263a036
update 2021-07-12 10:24:42 -04:00
yarikoptic
0bb668dcbe Added a comment 2021-07-12 14:17:52 +00:00
Joey Hess
6a581f8b8b
fix init reversion when core.sharedRepository = group
init: Fix misbehavior when core.sharedRepository = group that caused it to
enter an adjusted branch. (Reversion in version 8.20210630)

Commit 4b1b9d7a83 made init call
freezeContent in case there was a hook that could prevent writing in
situations where perms don't. But with the above git config, freezeContent
does not prevent write at all. So init needs to do what freezeContent does
with a non-shared git config.

Or init could check for that config, and skip the probing, since it
won't actually be preventing write to any files. But that would make init
too aware if details of Annex.Perms, and also would break if the git config
were changed after init.

Sponsored-by: Dartmouth College's Datalad project
2021-07-12 10:15:49 -04:00
Joey Hess
215aee105d
commet 2021-07-12 09:57:48 -04:00
Joey Hess
cd37ecb319
followup 2021-07-12 09:45:37 -04:00
Joey Hess
b885007f0e
--debug output goes to stderr again, not stdout
Reversion in version 8.20210428

Sponsored-by: Dartmouth College's Datalad project
2021-07-12 09:40:38 -04:00
yarikoptic
ff5a2e815c initial coarse report on the change in behavior in recent release 2021-07-08 01:42:32 +00:00
yarikoptic
fe1faa17d3 2021-07-07 22:32:08 +00:00
yarikoptic
abe08e3e49 initial report on stdout 2021-07-07 21:57:11 +00:00
Joey Hess
ec809de4b0
comment 2021-07-05 13:19:53 -04:00
Joey Hess
b9db859221
addurl: Avoid crashing when used on beegfs.
Sponsored-by: Dartmouth College's DANDI project
2021-07-05 13:02:40 -04:00
yarikoptic
4c191b2f0f Added a comment 2021-07-04 03:27:20 +00:00
yarikoptic
c381d60f64 allocate to dandi project 2021-07-03 13:44:53 +00:00
Joey Hess
5f0cc72303
comment 2021-07-02 10:42:54 -04:00
yarikoptic
f8f6d692b7 beegfs tests fails 2021-07-01 23:44:25 +00:00
Joey Hess
898a677bce
fix build on windows
Sponsored-by: Dartmouth College's Datalad project
2021-07-01 12:52:33 -04:00
Joey Hess
d9124bb8ce
tag because this is affecting datalad
https://github.com/datalad/datalad/issues/5750
2021-07-01 12:49:32 -04:00
Joey Hess
51335e9aad
comments 2021-07-01 12:45:03 -04:00
Joey Hess
d2c48404a8
assistant: Avoid unncessary git repository repair
In a situation where git fsck gets confused about a commit that is made
while it's running.

Sponsored-by: Graham Spencer on Patreon
2021-06-30 18:00:16 -04:00
Joey Hess
ee2e7c28a6
update 2021-06-30 17:20:15 -04:00
Joey Hess
0ffcfbdca6
bug 2021-06-30 16:44:30 -04:00
Joey Hess
73ccf34763
closing 2021-06-30 11:47:39 -04:00
jwodder
d296fb9693 2021-06-30 13:16:19 +00:00
falsifian
0a9a9c44e7 Added a comment 2021-06-30 00:09:23 +00:00
Atemu
99266d7875 Added a comment 2021-06-29 22:57:39 +00:00
yarikoptic
fe32661cf5 Added a comment 2021-06-29 17:27:35 +00:00
Joey Hess
199391befe
make repair interruption safe
Fixed bug that interrupting git-annex repair (or assistant) while it was
fixing repository corruption would lose objects that were contained in pack
files.

Unpack all pack files and move objects into place *before* deleting the
pack files. The old approach moved the pack files to a temp directory
before unpacking them, which was not interruption safe.

Sponsored-By: Jochen Bartl on Patreon
2021-06-29 13:14:28 -04:00
Joey Hess
a492553eca
some analysis 2021-06-28 15:00:21 -04:00
Joey Hess
26b0895187
close 2021-06-28 13:14:19 -04:00
Joey Hess
34b958144d
close 2021-06-28 13:11:13 -04:00
Joey Hess
d7d9363677
comment 2021-06-28 12:54:41 -04:00
Joey Hess
9cb5ed2446
avoid using cp --reflink=auto on windows
Sponsored-by: Dartmouth College's Datalad project
2021-06-28 12:51:40 -04:00
Joey Hess
0966bd2de4
comment 2021-06-25 13:33:27 -04:00
Joey Hess
df2001aa88
Improve display of errors when transfers fail
Transfers from or to a local git repo could fail without a reason being
given, if the content failed to verify, or if the object file's stat
changed while it was being copied. Now display messages in these cases.

Sponsored-by: Jack Hill on Patreon
2021-06-25 13:17:04 -04:00
Joey Hess
0fe550af75
fix windows build 2021-06-22 09:46:06 -04:00
jwodder
4b93e933db 2021-06-22 13:21:17 +00:00
Joey Hess
a6a6217322
remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before 2020.

Except for ones tagged projects/* since projects like datalad want to keep
around records of old deleted bugs longer.

Command line used:

for f in $(grep -l '|done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2020 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2020 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
for f in $(grep -l '\[\[done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2020 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2020 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
2021-06-21 13:10:13 -04:00
falsifian
95f685f2ca Added a comment 2021-06-19 23:48:21 +00:00
falsifian
a73624b161 Added a comment 2021-06-19 23:41:25 +00:00
yarikoptic
538780ebaf Added a comment 2021-06-16 21:32:58 +00:00
Ilya_Shlyakhter
afb8c5f078 Added a comment: thanks 2021-06-16 19:54:02 +00:00
yarikoptic
9dfda0899a removed 2021-06-15 15:38:47 +00:00
yarikoptic
35371af3da Added a comment 2021-06-15 15:38:28 +00:00
yarikoptic
5ac6583621 Added a comment 2021-06-15 15:38:16 +00:00
Joey Hess
3af4c9a29a
fix exponential blowup when adding lots of identical files
This was an old problem when the files were being added unlocked,
so the changelog mentions that being fixed. However, recently it's also
affected locked files.

The fix for locked files is kind of stupidly simple. moveAnnex already
handles populating unlocked files, and only does it when the object file
was not already present. So remove the redundant populateUnlockedFiles
call. (That call was added all the way back in
cfaac52b88, and has always been
unncessary.)

Sponsored-by: Dartmouth College's Datalad project
2021-06-15 09:45:55 -04:00
Joey Hess
e147ae07f4
remove supportUnlocked check that is not worth its overhead
moveAnnex only gets to that check if the object file was not present
before. So in the case where dup files are being added repeatedly,
it will only run the first time, and so there's no significant speedup
from doing it; all it avoids is a single sqlite lookup. Since MVar
accesses do have overhead, it's better to optimise for the common case,
where unlocked files are supported.

removeAnnex is less clear cut, but I think mostly is skipped running on
keys when the object has already been dropped, so similar reasoning
applies.
2021-06-15 09:28:56 -04:00
Joey Hess
6099edbf1c
bloom doesn't work, but this should I hope 2021-06-14 17:53:01 -04:00
Joey Hess
2df4c1cf91
plan 2021-06-14 17:13:37 -04:00
Joey Hess
643dc36e37
going round and round, boredly 2021-06-14 14:37:06 -04:00
yarikoptic
6043a2c7a0 Added a comment 2021-06-14 17:36:16 +00:00
Joey Hess
372ace599a
comment 2021-06-14 13:13:46 -04:00
Joey Hess
dcd2c95249
fix windows build 2021-06-14 12:43:26 -04:00
Joey Hess
014dc63a55
avoid sometimes expensive operations when annex.supportunlocked = false
This will mostly just avoid a DB lookup, so things get marginally
faster. But in cases where there are many files using the same key, it
can be a more significant speedup.

Added overhead is one MVar lookup per call, which should be small
enough, since this happens after transferring or ingesting a file,
which is always a lot more work than that. It would be nice, though,
to move getGitConfig to AnnexRead, which there is an open todo about.
2021-06-14 12:40:41 -04:00
Joey Hess
a02b5c2904
response 2021-06-14 12:36:42 -04:00
yarikoptic
51fede57a2 Added a comment 2021-06-14 16:23:41 +00:00
Ilya_Shlyakhter
35afd58a76 Added a comment: git-annex-add slowdown 2021-06-14 16:00:44 +00:00
Joey Hess
c4f1465a81
check symlink before reading file
This is faster because when multiple files are in a directory, it gets
cached.
2021-06-14 11:53:51 -04:00
Joey Hess
4163344ed6
retitle 2021-06-14 11:44:55 -04:00
Joey Hess
0eff5a3f71
reproduced 2021-06-14 11:37:21 -04:00
yarikoptic
8f66f73fea Added a comment 2021-06-09 22:28:06 +00:00
yarikoptic
e30f973323 Added a comment: more "mystery resolved" -- identical (empty) keys 2021-06-09 21:00:34 +00:00
Joey Hess
4b09b93a18
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-09 15:38:58 -04:00
Joey Hess
fad281767a
comment 2021-06-09 15:38:55 -04:00
yarikoptic
714d9f1315 Added a comment 2021-06-08 22:02:34 +00:00
yarikoptic
a8fb61329d Added a comment 2021-06-08 21:58:20 +00:00
yarikoptic
3985ae3224 Added a comment: OSX mystery resolved. add --batch is effective mitigation 2021-06-08 21:56:53 +00:00
Joey Hess
6cb9113ff5
comments 2021-06-08 17:38:56 -04:00
yarikoptic
c3993a2655 Added a comment 2021-06-08 20:23:09 +00:00
yarikoptic
437d9366b7 Added a comment: getting closer... 2021-06-08 19:21:59 +00:00
Ilya_Shlyakhter
be4a029e1b Added a comment 2021-06-08 19:08:01 +00:00
yarikoptic
697921ecd8 Added a comment: all recent builds/logs are fetched to smaug 2021-06-08 16:50:12 +00:00
Joey Hess
ecbaa52571
clarification 2021-06-08 12:00:01 -04:00
Joey Hess
1a6fa5abc8
add debugging for reconcileStaged calls for benchmarking 2021-06-08 11:57:23 -04:00
yarikoptic
62758ffb9f Added a comment: slow down is OSX specific 2021-06-08 14:28:18 +00:00
Joey Hess
d12120739d
comment 2021-06-08 10:19:04 -04:00
yarikoptic
57b567ac87 Added a comment 2021-06-07 21:39:05 +00:00
yarikoptic
2ffb9cc01b Added a comment: clarification 2021-06-07 21:20:35 +00:00
Joey Hess
e9a8b48a52
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-07 17:02:15 -04:00
Joey Hess
0f10f208a7
avoid double work in git-annex init
reconcileStaged was doing a redundant scan to scannAnnexedFiles.

It would probably make sense to move the body of scannAnnexedFiles
into reconcileStaged, the separation does not really serve any purpose.

Sponsored-by: Dartmouth College's Datalad project
2021-06-07 16:50:14 -04:00
Ilya_Shlyakhter
bdf3c06401 Added a comment: deferring the scan 2021-06-07 17:41:45 +00:00
Joey Hess
0101363eb8
correctly update keys db in merge conflict
This is quite a subtle edge case, see the bug report for full details.

The second git diff is needed only when there's a merge conflict.
It would be possible to speed it up marginally by using
--diff-filter=Unmerged, but probably not enough to bother with.

Sponsored-by: Graham Spencer on Patreon
2021-06-07 12:52:36 -04:00
Joey Hess
da24034331
comment 2021-06-07 11:53:25 -04:00
Joey Hess
3aabaa1a2b
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-07 11:47:13 -04:00
Joey Hess
254199edc9
comment 2021-06-07 11:46:47 -04:00
jwodder
8822bf0803 2021-06-07 14:32:25 +00:00
Atemu
234d235a03 rename bugs/delayadd_doesn__39__t_work.mdwn to bugs/delayadd_doesn__39__t_work_with_smallfiles.mdwn 2021-06-06 16:50:40 +00:00
falsifian
f58d686ccf Add bug report 2021-06-05 20:31:39 +00:00
yarikoptic
717000f4f8 Added a comment 2021-06-05 13:50:43 +00:00
yarikoptic
c5e62c1968 Initial report on performance regression 2021-06-05 13:23:13 +00:00
Joey Hess
5b7429e73a
avoid removing old associated file when there is a merge conflict
It makes sense to keep the key used by the old version of an
associated file, until the merge conflict is resolved.

Note that, since in this case git diff is being run with --index, it's
not possible to use -1 or -3, which would let the keys
associated with the new versions of the file also be added. That would
be better, because it's possible that the local modification to the file
that caused the merge conflict has not yet gotten its new key recorded
in the db.

Opened a bug about a case this is thus not able to address.

Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-06-01 11:43:00 -04:00
Joey Hess
f00e365f41
comments 2021-05-31 17:54:17 -04:00
Atemu
1b6715b4ca Added a comment 2021-05-31 19:07:31 +00:00
Ilya_Shlyakhter
9dbf93b9c5 Added a comment: clarifying preferred content expressions 2021-05-30 19:40:51 +00:00
Joey Hess
cd73fcc92c
Merge branch 'master' of ssh://git-annex.branchable.com 2021-05-25 11:45:02 -04:00
Joey Hess
483fc4dc6b
Merge branch 'trackassociated' 2021-05-25 11:43:52 -04:00