78a3d44ea0
The remaining callers all did not rely on it checking gitignore, so were easy to convert. They were susceptable to the same overwrite race as add and fix, although less likely to have it and a narrower window than add's race. Command.Rekey in passing got an unncessary call to removeFile deleted. addSymlink handles deleting any existing worktree file.
57 lines
1.9 KiB
Markdown
57 lines
1.9 KiB
Markdown
I was running `git-annex add` on 5 gb of files, and accidentially overwrote
|
|
some of the first ones, which it had already processed, while it was
|
|
running. This caused binary files to get staged in git, rather than the
|
|
annex pointers.
|
|
|
|
Test case:
|
|
|
|
echo hi > 1
|
|
dd if=/dev/urandom of=2 bs=1M count=1000
|
|
(sleep 2s; rm 1; echo bye > 1) &
|
|
git-annex add
|
|
git diff --cached 1
|
|
diff --git a/1 b/1
|
|
new file mode 100644
|
|
index 0000000..b023018
|
|
--- /dev/null
|
|
+++ b/1
|
|
@@ -0,0 +1 @@
|
|
+bye
|
|
|
|
This happens due to ingestAdd using addLink on the symlink,
|
|
which just queues a "git add" of the file for later. In the
|
|
meantime, the symlink is replaced with something else, so git
|
|
adds that.
|
|
|
|
It seems that the solution will be to use update-index rather than git add.
|
|
Note that addLink has a comment about why it uses git add, which seems to mostly
|
|
be that it's faster. It also sometimes relies on git add to check gitignore,
|
|
although sometimes redundandly, some of the callers of it may rely on that
|
|
and have to be changed to check it first themselves.
|
|
|
|
Since adding a file to the annex also involves locking it down and
|
|
detecting modifications made while generating the key, update-index is
|
|
sufficient.
|
|
|
|
> Update: This is fixed.
|
|
|
|
When it's adding a file unlocked, it already stages the pointer file using
|
|
update-index instead so there is no overwrite problem there.
|
|
|
|
But, there's a similar problem when it decides not to annex a file
|
|
and adds it to git. If the file content is overwritten then, it will
|
|
git add the new content. Which may be large enough that it should have been
|
|
added to the annex after all. Test case for this:
|
|
|
|
git config annex.largefiles largerthan=3b
|
|
echo hi > 1
|
|
dd if=/dev/urandom of=2 bs=1M count=1000
|
|
(sleep 2s; rm 1; echo bye > 1) &
|
|
git-annex add
|
|
git diff --cached 1
|
|
|
|
Unsure how to fix this case yet? Maybe it needs to cache the inode,
|
|
hash the file content, then verifiy the inode did not change during
|
|
hashing, and then also use update-index.
|
|
|
|
--[[Joey]]
|