From bd06bcaebe9642122634c0268e34c5e5e998b477 Mon Sep 17 00:00:00 2001 From: "Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082" Date: Wed, 13 Feb 2019 11:21:10 +0000 Subject: [PATCH] --- ...ssion_on_special_remotes_like___34__directory__34__.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__.mdwn diff --git a/doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__.mdwn b/doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__.mdwn new file mode 100644 index 0000000000..cf5f9d1ea4 --- /dev/null +++ b/doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__.mdwn @@ -0,0 +1,6 @@ +Despite the possibility to [compress data on a special remote with encryption](http://git-annex.branchable.com/forum/huge_text_files___40__not_binary__41___-_compress/#comment-189a7960c1ed7fb6b8bb2345e3ea8c07) +there are use-cases in which it would come in handy to have an option for a special remote like "directory" just for compression. + +For example, I use git annex for very large scientific tomographic datasets and files originating from their processing like segmentations, distance maps, skeletons. While compressing the raw data makes little sense, compression e.g. segmentations and skeletons has a huge impact on the effective files size. Since compressing files of a few GBs to TBs is time consuming, I prefer to have an uncompressed version in the working tree (so I do not use file formats that are using compression by default e.g. .nii.gz) but it would be very helpful to have the option to push precious or older versions to a remote that then uses compression. Using encryption for this is a bit of an overkill and takes considerably longer than compressing with e.g. `pbzip`. A compressed file system for this purpose is no option, because the special remote is supposed to live on a restrictive archive server. + +Though, I guess, it would be possible to write a special remote wrapper for this, I wonder if this might qualify as an officially supported option to the already existing special remotes like "directory" or "rsync". E.g. in conjunction to `encryption` something like `compression` with possible values like `pbzip`, `bzip`, `pigz` and `gzip`.