From 8c542d20a386af4f9debdb3cb98abd7bedcb0591 Mon Sep 17 00:00:00 2001 From: psxvoid Date: Sun, 19 Oct 2025 04:36:37 +0000 Subject: [PATCH 1/4] Added a comment: Re: support for bulk write/read/test remote - joey --- ..._20103729c97cb4392715987dca5408ae._comment | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 doc/design/external_special_remote_protocol/comment_57_20103729c97cb4392715987dca5408ae._comment diff --git a/doc/design/external_special_remote_protocol/comment_57_20103729c97cb4392715987dca5408ae._comment b/doc/design/external_special_remote_protocol/comment_57_20103729c97cb4392715987dca5408ae._comment new file mode 100644 index 0000000000..f74740ca6f --- /dev/null +++ b/doc/design/external_special_remote_protocol/comment_57_20103729c97cb4392715987dca5408ae._comment @@ -0,0 +1,34 @@ +[[!comment format=mdwn + username="psxvoid" + avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b" + subject="Re: support for bulk write/read/test remote - joey" + date="2025-10-19T04:36:37Z" + content=""" +Hi Joey, + +Sorry, for the late response, and thanks for the feedback. + +> \"that's fundamentally different than how git-annex works\" + +Hence the previous comment :) + +> \"And I think you could put it in your special remote.\" + +That's exactly what I was doing around a year ago. I was implementing a special remote to support writing data on BDXL disks. + +> \"So that when git-annex sends a file to your remote, the file is actually stored in the remote, rather than in a temporary location.\" + +Yep, roughly that's how I was implementing it - storing intermediate data in an sqlite database. + +I'd put the project on hold because I started to ask myself the following questions: + +1. OK, I can store transactions in the special remote. It means storing what is where on which disk. Isn't it what git annex supposed to do? +2. If a BDXL disk get's corrupted or lost, how to reflect it in the git annex repo and the special remote? I can mark it as \"lost\" in the remote, then run fsck in git annex remote. +3. Because I have to track location data separately in the special remote, what if it get's corrupted (the sqlite database)? +4. What if I buy 50GB BDXL instead of 100GB which I'm using? Does it means the special remote also should track free space on each disk? +5. Burning a disk - what if it won't be successful? Git annex will think that it was successful, cause it doesn't support bulk operations and numcopies rules will be violated. + +There were many more questions like this. + +And at some point the design started to look more like a blown-up feature-reach archival application/solution. The main point here is that it's definitely possible. I can limit the scope but there are many many issues, and nobody except me will be interested in it. Plus, many responsibilities would be overlapping with git annex. +"""]] From f88942266ffb275f3ec15841bbd1d78da5c8a352 Mon Sep 17 00:00:00 2001 From: psxvoid Date: Sun, 19 Oct 2025 04:58:54 +0000 Subject: [PATCH 2/4] Added a comment: Rr: support for bulk write/read/test remote (PS) --- .../comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment diff --git a/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment b/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment new file mode 100644 index 0000000000..c48cd63c2b --- /dev/null +++ b/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="psxvoid" + avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b" + subject="Rr: support for bulk write/read/test remote (PS)" + date="2025-10-19T04:58:54Z" + content=""" +P.S.: And to make it more clear why I told about dar first and then about writing BDXLs - when I mentioned dar - it was the stage when I experimented with dar to use it as an intermediary to write to BDXLs. But then I started to experiment with plain files because it could be better for long-term archival solution. +"""]] From ade2bfc6083be37b18c785c7a2ff94334f55fa55 Mon Sep 17 00:00:00 2001 From: psxvoid Date: Sun, 19 Oct 2025 04:59:19 +0000 Subject: [PATCH 3/4] removed --- .../comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment diff --git a/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment b/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment deleted file mode 100644 index c48cd63c2b..0000000000 --- a/doc/design/external_special_remote_protocol/comment_58_2bd7eb40046423b1424eaa2aae78ba95._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="psxvoid" - avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b" - subject="Rr: support for bulk write/read/test remote (PS)" - date="2025-10-19T04:58:54Z" - content=""" -P.S.: And to make it more clear why I told about dar first and then about writing BDXLs - when I mentioned dar - it was the stage when I experimented with dar to use it as an intermediary to write to BDXLs. But then I started to experiment with plain files because it could be better for long-term archival solution. -"""]] From c946d96c4f73589357df4556de521d031320d4c4 Mon Sep 17 00:00:00 2001 From: psxvoid Date: Sun, 19 Oct 2025 05:00:34 +0000 Subject: [PATCH 4/4] Added a comment: Re: support for bulk write/read/test remote - ps --- .../comment_58_d2ba09a90544cdfa245e69b951107702._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/design/external_special_remote_protocol/comment_58_d2ba09a90544cdfa245e69b951107702._comment diff --git a/doc/design/external_special_remote_protocol/comment_58_d2ba09a90544cdfa245e69b951107702._comment b/doc/design/external_special_remote_protocol/comment_58_d2ba09a90544cdfa245e69b951107702._comment new file mode 100644 index 0000000000..6df734ae3a --- /dev/null +++ b/doc/design/external_special_remote_protocol/comment_58_d2ba09a90544cdfa245e69b951107702._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="psxvoid" + avatar="http://cdn.libravatar.org/avatar/fde068fbdeabeea31e3be7aa9c55d84b" + subject="Re: support for bulk write/read/test remote - ps" + date="2025-10-19T05:00:34Z" + content=""" +P.S.: And to make it more clear why I told about dar first and then about writing BDXLs - when I mentioned dar - it was the stage when I was experimenting with dar to use it as an intermediary to write to BDXLs. But then I started to experiment with plain files because it could be better for a long-term archival solution. +"""]]