Lots of issues setting up assistant using SSH running on non-standard ports. Tested local pairing (which did seem to show other computer, but then wouldn't sync) and Remote Server.
### What steps will reproduce the problem?
Change Port 22 in /etc/ssh/sshd_config to Port 9999, restart ssh on both computers. With a clean local .ssh/config and directory, try to set up local pairing or remote server. It appears to work, especially if XMPP is working properly with remote server, but then some operations fail. (Iirc, metadata does sync but not data.) Ultimately, I had to go back to using Port 22 and using denyhosts/fail2ban (which occasionally erroneously ban my IP).
(Although it's probably not relevant to the bug report, it could be argued that this is security through obscurity. There is some truth in this, but scanning an entire machine is 65,535 times slower than scanning just port 22, so it introduces a real cost to bulk scanning. I almost never, ever have attacks on random ports, whereas I have dozens per day on each server on port 22, and often thousands of attacks.)
### What version of git-annex are you using? On what operating system?
When I was experiencing this issue, I was running the default on Jessie/Wheezy. Now I'm running the latest (via auto-update and distributed binary) but don't know if this is still an issue with latest versions (I switched to 22 as a workaround).
