Merge branch 'master' into bs

doc/bugs/move_from_ssh_when_not_present_displays_misleading

@@ -0,0 +1,18 @@
+When a file is not present on a ssh remote, a move of that file fails
+like this:
+
+	move foo (from r...) 
+	  verification of content failed
+	  failed
+	move bar (from r...) 
+	  Lost connection (fd:24: hGetChar: illegal operation (handle is closed))
+	  failed
+
+Both files were not present, so two different failures, and neither message
+bears on the real reason why the move failed.
+
+(Despite the connection having closed, it then was able to move a third file
+that was still present.)
+
+IIRC there was a clear message displayed before git-annex-shell p2pstdio 
+got implemented. --[[Joey]]

doc/bugs/on_some_remotes_failing_to_detect_annex_spits_out_message_to_stderr_and_empty_lines_to_stderr__44___ignores_--json-error-messages.mdwn

@@ -0,0 +1,56 @@
+### Please describe the problem.
+
+
+### What steps will reproduce the problem?
+
+
+### What version of git-annex are you using? On what operating system?
+
+
+### Please provide any additional information below.
+
+[[!format sh """
+lena:/tmp
+$> git clone http://kumo.ovgu.de/~mih/myHP/bd2/4e4aa-7aea-11e6-9d5d-002590f97d84/
+Cloning into '4e4aa-7aea-11e6-9d5d-002590f97d84'...
+
+$> cd 4e4aa-7aea-11e6-9d5d-002590f97d84 
+
+$> git annex init                                                                
+init  (merging origin/git-annex into git-annex...)
+(recording state in git...)
+(scanning for unlocked files...)
+
+  Failed to get annex.uuid configuration of repository origin
+
+  Instead, got: "core.repositoryformatversion\n0\NULcore.filemode\ntrue\NULcore.bare\ntrue\NUL"
+
+  This is unexpected; please check the network transport!
+(Auto enabling special remote datalad-archives...)
+(Auto enabling special remote inm7-storage...)
+
+  Cannot run git-annex-remote-ria -- It is not installed in PATH (/usr/lib/git-annex.linux/bin:/usr/lib/git-core:/home/yoh/picts/mris/heudiconv-master/venvs/dev3/bin:/home/yoh/gocode/bin:/home/yoh/gocode/bin:/home/yoh/bin:/home/yoh/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/usr/local/sbin:/usr/lib/git-annex.linux/extra)
+ok
+(recording state in git...)
+
+"""]]
+
+so two issues in above:
+- not clear why it dumps git config if fetched instead of just announcing that remote has no git-annex and set to ignore
+- empty lines between all the messages go to stdout:
+
+[[!format sh """
+$> git annex init 2>/dev/null
+init  (scanning for unlocked files...)
+
+
+
+(Auto enabling special remote inm7-storage...)
+
+ok
+"""]]
+
+PS ignore "ignores --json-error-messages" part of the subject -- was detected in 7.20190819+git2-g908476a9b-1~ndall+1 during `get` but upgrade to 7.20191114+git43-ge29663773-1~ndall+1 resolved it
+
+[[!meta author=yoh]]
+[[!tag projects/datalad]]

doc/devblog/day_613__end-to-end_ByteString_milestone.mdwn

@@ -0,0 +1,17 @@
+The `bs` branch has reached a milestone: `git-annex find` and `git-annex
+get` (when all files are present) process ByteStrings end-to-end with 
+no String conversion. That sped it up by around 30% on top of the previous
+optimisations. 
+
+To get here, I spent a couple of days creating the
+[filepath-bytestring](https://joeyh.name/blog/entry/announcing_the_filepath-bytestring_haskell_library/)
+library, which git-annex will depend on. Lots more git-annex internals
+were switched to ByteString, especially everything having to do with
+statting files.
+
+Other commands, like `git-annex whereis`, still do some String
+conversions. Optimisation never ends.
+
+But the bs branch is ready to merge as-is, and the diff is 10 thousand lines,
+so not a branch I want to maintain for long. Planning to merge it after
+the next release.

doc/internals/key_format/comment_1_4ec126bffafc81fae04e183874ffce39._comment

@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="https://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ avatar="http://christian.amsuess.com/avatar/c6c0d57d63ac88f3541522c4b21198c3c7169a665a2f2d733b4f78670322ffdc"
+ subject="Key character set"
+ date="2019-12-10T10:27:58Z"
+ content="""
+Are there limitations on the character set git-annex guarantees?
+
+It appears from experiments that git-annex only uses ASCII characters in there, given both a file 'test.extü' (in UTF-8 encoding) 'test.ext\xff' produced extension-free key names in the SHA256E hash – but it'd be good to have that confirmed.
+"""]]

doc/tips/android_sync_with_adb/comment_1_305dbaac1c9c526c4b467314960e3bcd._comment

@@ -0,0 +1,27 @@
+[[!comment format=mdwn
+ username="MichaelC"
+ avatar="http://cdn.libravatar.org/avatar/248593885d551a3912e488c4bc9d311c"
+ subject="Does android adb special remote support MTP?"
+ date="2019-12-10T21:23:31Z"
+ content="""
+I'm attempting to init this special remote with my Pixel 3a android phone (which connect via MTP) but keep hitting the following error.
+
+In Debian, this will mount to /run , specifically in my case
+
+/run/user/1000/gvfs/mtp\\:host\=Google_Pixel_3a_94MBY/Internal\ shared\ storage/DCIM
+
+git annex initremote android type=adb androiddirectory=/run/user/1000/gvfs/mtp\\:host\=Google_Pixel_3a_94MBY/Internal\ shared\ storage/DCIM  encryption=none exporttree=yes importtree=yes
+initremote android 
+git-annex: adb: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)
+failed
+git-annex: initremote: 1 failed
+
+I've tried enclosing the androiddirectory in single/double quotes (based on the hope that the colon before host was the issue)
+I've also attempted to substitute a symbolic link to the directory instead - but no luck.
+
+(*) I can cd  into this directroy via xterm and browse etc.
+
+Thanks,
+M
+
+"""]]

doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_2_fa9e2c0a83bad2e0c2da3016b9bf2100._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="yarikoptic"
+ avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
+ subject="comment 2"
+ date="2019-12-10T22:01:12Z"
+ content="""
+> An external remote could also do its own checksum checking and then set `remote..annex-verify=false`
+
+that is an interesting idea, thanks!  Not sure if that makes it easy for mass consumption though since it is a feature of a external remote, not sure why it should be in the config.  Ideally it should be a property of a remote.
+
+Joey, what do you think in regard of built-in remotes?
+"""]]

doc/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/comment_3_529c38fc63540b32c51ae75529e9005e._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="Ilya_Shlyakhter"
+ avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
+ subject="annex-verify"
+ date="2019-12-11T18:13:48Z"
+ content="""
+\"it is a feature of a external remote, not sure why it should be in the config\" -- because the user might not trust an external remote's implementation of this feature.  Besides bugs, there might be [[security exploits|security/CVE-2018-10857_and_CVE-2018-10859]] if external remotes could single-handedly disable verification.
+"""]]