mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions 1

comment

Browse source
This commit is contained in:
Joey Hess 2022-11-09 15:14:28 -04:00
parent 5409b0c5c4
commit 94ee608b2e
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
1 changed files with 50 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

50
doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment Normal file
View file

@ -0,0 +1,50 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2022-11-09T18:16:29Z"
content="""
I tried making a repository with just 2 files, one in git and one in git-annex,
and am unable to reproduce the bug. Here is
what `git-annex export master --to remote --debug` showed when
exporting a tree that deleted file "foo" which was a git object:
[2022-11-09 14:16:09.513485291] (Remote.S3) String to sign: "DELETE\n/foo\n\nhost:t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20221109T181609Z\n\nhost;x-amz-content-sha256;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
[2022-11-09 14:16:09.513608493] (Remote.S3) Host: "t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com"
[2022-11-09 14:16:09.513697445] (Remote.S3) Path: "/foo"
[2022-11-09 14:16:09.513748086] (Remote.S3) Query string: ""
[2022-11-09 14:16:09.513829584] (Remote.S3) Header: (redacted -- JEH)
[2022-11-09 14:16:09.687814925] (Remote.S3) Response status: Status {statusCode = 204, statusMessage = "No Content"}
The S3 console showed that the file was deleted from the bucket.
And as far as the S3 remote implementation is concerned, there should
not be anything different between a git object and a git-annex object.
At the level of the S3 remote both have a git-annex key that it deletes
in the same way.
In your log, the only thing it does with the file is export it, but it does
not later unexport it:
$ grep baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json Versioned_S3_tree_does_not_unexport_git_objects.mdwn
Git files in the 1.0.0 tag are still present in the S3 1.0.1 export. sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json is an example file not present in 1.0.1 that is still present on S3.
export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json
$
If the bug is that it's somehow failing to try to unexport the file,
that should happen independently of the special remote type, so would also
happen with a directory special remote. So I tried that:
$ git clone https://github.com/openneurodatasets/ds001705
$ cd ds001705
$ git-annex get --branch=tags/1.0.0
$ git-annex get --branch=tags/1.0.1
$ mkdir ../d
$ git-annex initremote d type=directory directory=../d encryption=none exporttree=yes
$ git-annex export 1.0.0 --to d
$ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json
../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json
$ git-annex export 1.0.1 --to d
$ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json
../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json
Ok, so, nothing to do with S3 or versioning at all.
"""]]