Merge branch 'master' of ssh://git-annex.branchable.com

doc/forum/enabling_git-annex-diffdriver_for_gitk.mdwn

@@ -0,0 +1,5 @@
+If `git` with `git-annex-diffdriver` is properly configured and works, is it possible to also get `gitk` to show these diffs?
+
+It seems that `gitk` in general does not apply diff-filters to symlinks and I was not able to find a way to make `gitk` show the same as `git` with `git-annex-diffdriver` does.
+
+Are there some tricks to make that possible somehow?

doc/forum/short_exlusive_refspec_notation_for_git_annex_unused_.mdwn

@@ -0,0 +1,22 @@
+Hello!
+
+
+In relation to the description of `--used-refspec` in the docs https://git-annex.branchable.com/git-annex-unused/ 
+I was wondering why it is possible to add a positive refspec without the full path of the ref e.g.
+
+`git annex unused --used-refspec  +master`
+
+while for exclusions (negative refspec) it seems to be necessary to specify the full path, e.g.
+
+`git annex unused --used-refspec  '+*:-refs/heads/master'`
+
+or globbing:
+
+`git annex unused --used-refspec  '+*:-*master'`
+
+I guess the negative form of 
+
+> Each + without a glob adds the literal value to the set.  For example, "+HEAD^" adds "HEAD^".
+
+is actually (intentionally) missing. 
+If so, this might be worth noting in the docs.

doc/git-annex.mdwn

@@ -872,7 +872,8 @@ Here are all the supported configuration settings.
   Maximum length of what is considered a filename extension when adding a
   file to a backend that preserves filename extensions. The default length
   is 4, which allows extensions like "jpeg". The dot before the extension
-  is not counted part of its length.
+  is not counted part of its length.  At most two extensions at the end of
+  a filename will be preserved, e.g. .gz or .tar.gz .
 
 * `annex.diskreserve`
 

doc/todo/add_sftp_special_remote/comment_1_28c98e879c16f56da2a89ed9d50e4398._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="Grothausmann.Roman@343b033fef650d362015d5593ec935121340e082"
+ nickname="Grothausmann.Roman"
+ avatar="http://cdn.libravatar.org/avatar/7883b2b02c31cc2b5e8672cfea4ec9da"
+ subject="Up-vote for sftp special remote"
+ date="2019-02-13T12:11:05Z"
+ content="""
+I'd really welcome an __sftp special remote__ because SFTP it is the only officially allowed way for our institution to exchange data with external collaborators and I had no luck using the [gvfs](https://github.com/grawity/code/blob/master/net/git-annex-remote-gvfs) approach. 
+
+Even without an sftp special remote, it would be great to have some short walkthrough how to use SFTP with git-annex (if SFTP is currently actually possible with e.g. gvfs).
+
+"""]]

doc/todo/add_sftp_special_remote/comment_2_1fcefc50041b363a86f596f8d4121611._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="comment 2"
+ date="2019-02-13T16:46:25Z"
+ content="""
+This can be implemented as an external special remote in Python with a combination of https://github.com/althonos/fs.sshfs and  https://github.com/Lykos153/AnnexRemote .  Implementing using https://www.pyfilesystem.org/ would make a remote that works with a range of filesystems ( https://www.pyfilesystem.org/page/index-of-filesystems/ ).   I don't have time right now but maybe you can try.
+"""]]

doc/todo/batch_operations_for_remotes.mdwn

@@ -0,0 +1 @@
+Current special remote protocol works on one file at a time.  With some remotes, a batch operation can be more efficient, e.g. querying the status of many URLs in one API call.  It would be good if special remotes could optionally implement batch versions of their operations, and these versions were used by batch-mode git-annex commands.  Or maybe, keep the current set of commands but let the remote read multiple requests and then send multiple replies?

doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__.mdwn

@@ -0,0 +1,6 @@
+Despite the possibility to [compress data on a special remote with encryption](http://git-annex.branchable.com/forum/huge_text_files___40__not_binary__41___-_compress/#comment-189a7960c1ed7fb6b8bb2345e3ea8c07)
+there are use-cases in which it would come in handy to have an option for a special remote like "directory" just for compression. 
+
+For example, I use git annex for very large scientific tomographic datasets and files originating from their processing like segmentations, distance maps, skeletons. While compressing the raw data makes little sense, compression e.g. segmentations and skeletons has a huge impact on the effective files size. Since compressing files of a few GBs to TBs is time consuming, I prefer to have an uncompressed version in the working tree (so I do not use file formats that are using compression by default e.g. .nii.gz) but it would be very helpful to have the option to push precious or older versions to a remote that then uses compression. Using encryption for this is a bit of an overkill and takes considerably longer than compressing with e.g. `pbzip`. A compressed file system for this purpose is no option, because the special remote is supposed to live on a restrictive archive server.
+
+Though, I guess, it would be possible to write a special remote wrapper for this, I wonder if this might qualify as an officially supported option to the already existing special remotes like "directory" or "rsync". E.g. in conjunction to `encryption` something like `compression` with possible values like `pbzip`, `bzip`, `pigz` and `gzip`.

doc/todo/option_for___40__fast__41___compression_on_special_remotes_like___34__directory__34__/comment_1_8620167702725aa8fb0e42dfe3820520._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="comment 1"
+ date="2019-02-13T16:48:08Z"
+ content="""
++1 for this.  Maybe, add a config option to pipe a file through a particular program before sending it to a remote?
+"""]]

doc/todo/simpler__44___trusted_export_remotes.mdwn

@@ -0,0 +1,3 @@
+Currently, some issues impede the use of export remotes: (1) they're untrusted, except for versioned ones -- and from  those keys cannot be dropped; (2) using them is different than using normal remotes: one can't just copy or move keys to them, one has to first make a tree-ish.   Maybe this could be fixed, as follows.   To copy a key to a external remote, if the key is not yet present in it, put it under .keys/aaa/bbb/keyname on the remote.  That is, take the tree-ish currently on the remote, merge .keys/aaa/bbb/keyname with it, and put that on the remote.   To drop a key from an external remote, take the tree-ish currently on the remote, drop all instances of the key from it, and push the changed tree-ish to the remote.  To git-annex-export add an option --add , which will add the tree-ish to the tree-ish currently on the remote, without losing any keys currently on the remote: take the tree-ish currently on the remote; overlay on it the treeish being exported; for any files that would be overwritten, if no copies of that key would be left, move it to .keys/aaa/bbb/keyname in the tree-ish that is then pushed to the remote.
+
+This way, can always just copy any tree to the remote, without worrying about losing data.