mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2019-02-24 12:45:32 -04:00
commit 413c101850
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 89 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
doc/bugs/data_loss_from_dedup_when_v7_unlocked_file_is_duplicated__47__truncated/comment_1_32748e7bd298d31347c68c9aabd17c77._comment Normal file
View file

@ -0,0 +1,24 @@
[[!comment format=mdwn
username="david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8"
nickname="david.j.buckmaster"
avatar="http://cdn.libravatar.org/avatar/1650cdf23a0999fd0e03650e20c90ee7"
subject="comment 1"
date="2019-02-24T01:20:14Z"
content="""
I guess the issue here isn't specifically the truncation, but rather that annexed content isn't confirmed when committing an rm.
set -x && \
mkdir tmp && cd tmp && \
git init && \
git-annex init --version 7 base && \
git-annex config --set annex.thin true && \
\
echo \"bar\">foo && cat foo && git-annex add foo && ls -l && git-annex sync && git-annex unlock foo && git-annex sync && \
echo \"barbar\">foo && cat foo && rm foo && git-annex sync && \
\
echo \"bar\">ffoo && cat ffoo && git-annex add ffoo && git-annex sync && cat ffoo
This results in ffoo containing the text \"barbar\" rather than \"bar\" (symlink to foo's last committed content), and unlocking ffoo after this leads to a pointer file instead of hardlink (and git-annex info ffoo shows present: false).
I'm new to this, so apologies if I'm doing something stupid.
"""]]

65
doc/bugs/data_loss_from_dedup_when_v7_unlocked_file_is_duplicated__47__truncated/comment_2_5395837d61947644930285e65a4a8529._comment Normal file
View file

@ -0,0 +1,65 @@
[[!comment format=mdwn
username="david.j.buckmaster@984ff2704feacab53415ac5647206517d18f88f8"
nickname="david.j.buckmaster"
avatar="http://cdn.libravatar.org/avatar/1650cdf23a0999fd0e03650e20c90ee7"
subject="comment 2"
date="2019-02-24T07:01:35Z"
content="""
In case this is NOT a bug but rather an incompatibility between borg/git-annex, the bash snippet below seems to sidestep the issue if run AFTER a borg operation but BEFORE any other annex commit/sync. Unsure if I'm the only person in the world that wants a not-permanently-append-only borg repository in an unlocked+thin git-annex...but...after a borg operation that truncates+deletes files, the function uses the status printout to identify all missing, borg-controlled files, and for each
1. Reverts the deletion (git checkout -- ..)
2. Checks if the restored file is a (dead) file pointer (file contents start with /annex/objects). If no, redo the delete and continue loop; if yes, truncate (but do not delete) the file and stage a commit.
Truncations are then committed to force annex to see that contents have changed, and all deletions are repeated. After that, git annex add-ing a new file with old content restores the annex content that borg truncated, rather than ending up with a pointer file and missing contents.
Test case:
set -x && \
mkdir tmp && cd tmp && \
git init && \
git-annex init --version 7 base && \
git-annex config --set annex.thin true && \
\
echo \"bar\">foo && cat foo && git-annex add foo && ls -l && git-annex sync && git-annex unlock foo && git-annex sync && \
echo \"barbar\">foo && cat foo && rm foo && \
\
git checkout -- foo && truncate -s 0 foo && git add foo && git-annex sync && \
rm foo && git-annex sync && \
\
echo \"bar\">ffoo && cat ffoo && git-annex add ffoo && git-annex sync && cat ffoo
Bash function:
handleborgdeletes() {
echo \"(handleborgdeletes\"
local PTRFILES=()
local HAVETRUNCS=
local f
for f in $(IFS=$'\n' $GITANNEX status | grep ^D | grep -E '(index|hints|integrity|data/)'); do
f=${f:2}
$GIT checkout -- \"$f\" #restore
if [ -n \"$(grep -F --text '/annex/objects' $f)\" ]; then #is broken annex pointer file
echo \"$f is a pointer file\"
HAVETRUNCS=1
truncate -s 0 \"$f\" #truncate
PTRFILES+=(\"$f\") #track
$GIT add \"$f\" #stage content change
else
echo \"$f is not a pointer file\"
rm \"$f\"
fi
done
if [ -n \"$HAVETRUNCS\" ]; then
$GIT commit -m \"truncating borg deletions\" #commit content changes
for f in ${PTRFILES[@]}; do
rm \"$f\" #repeat delete to be committed later
done
fi
echo \"handleborgdeletes)\"
}
"""]]