From 94ee608b2e19b917ce14038032bfb6ccd7864f38 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 9 Nov 2022 15:14:28 -0400 Subject: [PATCH] comment --- ..._70428ebc4027538253edc483dc5cb971._comment | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment diff --git a/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment new file mode 100644 index 0000000000..9a016f3498 --- /dev/null +++ b/doc/bugs/Versioned_S3_tree_does_not_unexport_git_objects/comment_1_70428ebc4027538253edc483dc5cb971._comment @@ -0,0 +1,50 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2022-11-09T18:16:29Z" + content=""" +I tried making a repository with just 2 files, one in git and one in git-annex, +and am unable to reproduce the bug. Here is +what `git-annex export master --to remote --debug` showed when +exporting a tree that deleted file "foo" which was a git object: + + [2022-11-09 14:16:09.513485291] (Remote.S3) String to sign: "DELETE\n/foo\n\nhost:t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com\nx-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nx-amz-date:20221109T181609Z\n\nhost;x-amz-content-sha256;x-amz-date\ne3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + [2022-11-09 14:16:09.513608493] (Remote.S3) Host: "t-a9b9d406-30e5-41cc-a74c-c5d83b2953fb.s3.amazonaws.com" + [2022-11-09 14:16:09.513697445] (Remote.S3) Path: "/foo" + [2022-11-09 14:16:09.513748086] (Remote.S3) Query string: "" + [2022-11-09 14:16:09.513829584] (Remote.S3) Header: (redacted -- JEH) + [2022-11-09 14:16:09.687814925] (Remote.S3) Response status: Status {statusCode = 204, statusMessage = "No Content"} + +The S3 console showed that the file was deleted from the bucket. +And as far as the S3 remote implementation is concerned, there should +not be anything different between a git object and a git-annex object. +At the level of the S3 remote both have a git-annex key that it deletes +in the same way. + +In your log, the only thing it does with the file is export it, but it does +not later unexport it: + + $ grep baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json Versioned_S3_tree_does_not_unexport_git_objects.mdwn + Git files in the 1.0.0 tag are still present in the S3 1.0.1 export. sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json is an example file not present in 1.0.1 that is still present on S3. + export s3-PUBLIC sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json + $ + +If the bug is that it's somehow failing to try to unexport the file, +that should happen independently of the special remote type, so would also +happen with a directory special remote. So I tried that: + + $ git clone https://github.com/openneurodatasets/ds001705 + $ cd ds001705 + $ git-annex get --branch=tags/1.0.0 + $ git-annex get --branch=tags/1.0.1 + $ mkdir ../d + $ git-annex initremote d type=directory directory=../d encryption=none exporttree=yes + $ git-annex export 1.0.0 --to d + $ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json + ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json + $ git-annex export 1.0.1 --to d + $ ls ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json + ../d/sub-000101/ses-baseline/pet/sub-000101_ses-baseline_rec-MLEM_pet.json + +Ok, so, nothing to do with S3 or versioning at all. +"""]]