Merge branch 'master' of ssh://git-annex.branchable.com

doc/bugs/Assistant_from_standalone_5.20131230_64bit_crashes_on_startup.mdwn

@@ -0,0 +1,29 @@
+The assistant from the standalone 64bit build crashes upon startup.
+
+### Version information
+
+     [0 zerodogg@firefly annexed]$ git annex version
+     git-annex version: 5.20131230-g4aa88d8
+     build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus XMPP Feeds Quvi TDFA
+     key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
+     remote types: git gcrypt S3 bup directory rsync web webdav glacier hook external
+     local repository version: 5
+     supported repository version: 5
+     upgrade supported from repository versions: 0 1 2 4
+     [0 zerodogg@firefly annexed]$ uname -a
+     Linux firefly 3.12.5-302.fc20.x86_64 #1 SMP Tue Dec 17 20:42:32 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
+     [0 zerodogg@firefly annexed]$ cat /etc/fedora-release 
+     Fedora release 20 (Heisenbug)
+     [0 zerodogg@firefly annexed]$ 
+
+### Error
+
+     [0 zerodogg@firefly annexed]$ git annex webapp
+     Launching web browser on file:///home/zerodogg/Documents/annexed/.git/annex/webapp.html
+     
+     (process:6045): GLib-CRITICAL **: g_slice_set_config: assertion 'sys_page_size == 0' failed
+     libgcc_s.so.1 must be installed for pthread_cancel to work
+     error: git-annex died of signal 6
+     [134 zerodogg@firefly annexed]$ ls -l /usr/lib64/libgcc_s.so.1
+     lrwxrwxrwx. 1 root root 28 2013-12-18 10:54 /usr/lib64/libgcc_s.so.1 -> libgcc_s-4.8.2-20131212.so.1
+     [0 zerodogg@firefly annexed]$ 

doc/bugs/assistant_using_the_incorrect_path_on_windows__63__.mdwn

@@ -0,0 +1,42 @@
+### Please describe the problem.
+The assistant (or rather the programs it's running) appears to be using unix-style paths instead of cygwin paths, causing several types of errors. A selection of items from the terminal:
+
+[[!format sh """
+Launching web browser on file://C:\Users\Familyroom\annex\.git\annex\webapp.html
+[2013-12-31 01:14:05 Eastern Standard Time] read: ssh-keygen ["-F","celebdil.db4
+8x.net"]
+ssh-keygen: /home/Familyroom/.ssh/known_hosts: No such file or directory
+[2013-12-31 01:14:05 Eastern Standard Time] read: ssh ["-oNumberOfPasswordPrompt
+s=0","-oStrictHostKeyChecking=no","-n","-p","22","rlbrooks@celebdil.db48x.net","
+sh -c 'echo git-annex-probe loggedin;if which git-annex-shell; then echo git-ann
+ex-probe git-annex-shell; fi;if which git; then echo git-annex-probe git; fi;if
+which rsync; then echo git-annex-probe rsync; fi;if which ~/.ssh/git-annex-shell
+; then echo git-annex-probe ~/.ssh/git-annex-shell; fi;cd '\"'\"'annex'\"'\"' &&
+ git config --list'"]
+[2013-12-31 01:14:08 Eastern Standard Time] read: ssh-keygen ["-F","celebdil.db4
+8x.net"]
+ssh-keygen: /home/Familyroom/.ssh/known_hosts: No such file or directory
+[2013-12-31 01:14:08 Eastern Standard Time] read: ssh ["-oStrictHostKeyChecking=
+no","-n","-p","22","rlbrooks@celebdil.db48x.net","sh -c 'echo git-annex-probe lo
+ggedin;if which git-annex-shell; then echo git-annex-probe git-annex-shell; fi;i
+f which git; then echo git-annex-probe git; fi;if which rsync; then echo git-ann
+ex-probe rsync; fi;if which ~/.ssh/git-annex-shell; then echo git-annex-probe ~/
+.ssh/git-annex-shell; fi;cd '\"'\"'annex'\"'\"' && git config --list'"]
+rlbrooks@celebdil.db48x.net's password:
+[2013-12-31 01:15:03 Eastern Standard Time] read: gpg ["--quiet","--trust-model"
+,"always","--with-colons","--list-secret-keys","--fixed-list-mode"]
+gpg: WARNING: using insecure memory!
+gpg: please see http://www.gnupg.org/documentation/faqs.html for more informatio
+n
+gpg: fatal: can't create directory `/home/Familyroom/.gnupg': No such file or di
+rectory
+secmem usage: 0/0 bytes in 0/0 blocks of pool 0/65536
+"""]]
+
+Naturally, I don't have a /home/Familyroom, although I do have a /c/Users/Familyroom.
+
+### What steps will reproduce the problem?
+Start creating a remote repository.
+
+### What version of git-annex are you using? On what operating system?
+Windows 7, git-annex version 5.20131230-g192d991

doc/bugs/webapp_takes_100__37___of_the_cpu.mdwn

@@ -0,0 +1,41 @@
+### Please describe the problem.
+
+I just started git-annex, and I can hear the fan of my CPU. That doesn't happen unless the CPU is maxed, and indeed:
+
+[[!format txt """
+top - 10:09:53 up 2 days, 13:13, 10 users,  load average: 2,49, 2,41, 1,59
+Tasks: 306 total,   3 running, 302 sleeping,   0 stopped,   1 zombie
+%Cpu(s): 57,2 us,  7,9 sy,  1,2 ni, 32,1 id,  1,3 wa,  0,0 hi,  0,3 si,  0,0 st
+KiB Mem:   8162960 total,  7480272 used,   682688 free,  1107616 buffers
+KiB Swap:  4022268 total,    23384 used,  3998884 free,  3469028 cached
+
+  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
+27841 anarcat   20   0  191m  45m  20m S  92,3  0,6   4:32.68 /usr/local/bin/git-annex webapp
+"""]]
+
+### What steps will reproduce the problem?
+
+I am not sure. I just started the webapp, and it seems to be scanning two huge repositories (160GB and 800GB) with lots of files. The larger one is also in the process of being sync'd to the backup drive.
+
+### What version of git-annex are you using? On what operating system?
+
+My usual bastard build from hell:
+
+[[!format txt """
+git-annex version: 5.20131109-gf2cb5b9
+build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP DNS Feeds Quvi TDFA CryptoHash
+key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
+remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
+local repository version: 3
+default repository version: 3
+supported repository versions: 3 5
+upgrade supported from repository versions: 0 1 2 4
+"""]]
+
+### Please provide any additional information below.
+
+Note that I had to close the chromium tab that was displaying the actual webapp UI, because that was taking another 50% of a CPU (I have a dual core).
+
+Note also that things calmed down since git-annex started transfering larger files - the webapp only takes 75% of the CPU now. ;) And chromium is negligible. But there clearly seems to be a degenerate case where a lot of small files get transfered that seem to freakout the web UI. -- [[anarcat]]
+
+Oh, and just copying the files using `git annex copy --to backup` doesn't use 100% of the CPU.

doc/design/external_special_remote_protocol/comment_5_3ee158e548002badae5bf44dc0442626._comment

@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8"
+ nickname="TobiasTheViking"
+ subject="Feature requests"
+ date="2013-12-31T14:05:07Z"
+ content="""
+Hook should be able to set default configuration for itself.
+
+For instance, clean flickr hook will only upload some files(notably pictures). The user shouldn't have to manage that.
+
+Other hooks have a maximum filesize(though i guess that doesn't matter once splitting works).
+
+"""]]

doc/forum/Git_Annex_Sync_Delinks_Files.mdwn

@@ -0,0 +1,11 @@
+So, I'm using git-annex to manage a group of external hard drives that I use to store a bunch of big files.
+
+When syncing tonight, though, it decided to take a whole subdirectory and replace the symlinks with the content directly when merging, or something.
+
+The remote side is not like that, and I didn't tell it to do anything like that.
+It just seems to feel like that's the thing to do, then it syncs and fails because suddenly git has many gigs of objects, which is what git-annex is supposed to solve.
+
+I've reset master a bunch of times, and it keeps choosing that path, but I don't know why.
+I've also reset master and synced/master a few times. I'm not fully sure what that one actually encodes, so I didn't want to touch it too much, but setting it to the same thing as master didn't seem to fix things either.
+
+Any idea why?

doc/forum/crashing_committers_on_files_with_special_characters_in_their_names.mdwn

@@ -0,0 +1 @@
+I encounter crashing commits on files with special characters in their names, such as äöüáí etc. Is it not possible to sync such files?

doc/forum/partial_synchronisation._android_phone/comment_2_550ae91c8a1fe060368c4682d37514b6._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkNE-H4vEcbcGndxq5daT8qUb7yIf7r1OE"
+ nickname="Łukasz"
+ subject="comment 2"
+ date="2013-12-30T22:13:15Z"
+ content="""
+thx for reply
+"""]]

doc/forum/ssh__95__exchange__95__identification:_read:_Connection_reset_by_peer.mdwn

@@ -13,4 +13,16 @@ The webapp shows the remote repo as "unfinished repository".
 
 I can access the other computer via ssh and an other git-annex repo pairing between the two computers work fine.
 
+UPDATE: In the meantime I setup another paired repo on these two computers. Everything works fine. 
+
+I added the repos from scratch again and the same errors occurs again. The repositories which do not pair are the Documents folder on my desktop and on my macbook. Is there something special with macs documents folder?
+
+    [2013-12-31 10:53:23 CET] PushRetrier: Syncing with macbook.local_Documents 
+    fatal: 'macbook.local_Documents' does not appear to be a git repository
+    fatal: Could not read from remote repository.
+
+    Please make sure you have the correct access rights
+    and the repository exists.
+
+
 juh

doc/forum/what_happens_to_deleted_files__63__/comment_1_a476174646ad3adfbbe0cafdd7d42d08._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://sameer.sbuddhe.net/blog/"
+ nickname="sameerds"
+ subject="never mind ..."
+ date="2013-12-31T04:06:43Z"
+ content="""
+... I discovered the [[`--all` option|http://git-annex.branchable.com/forum/Moving_older_version__39__s_file_content_without_doing_checkout/]] which does just what I was looking for!
+
+
+"""]]

doc/tips/finding_duplicate_files/comment_11_5efc6b6ee1dfec88512183e9679ca616._comment

@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="sameerds"
+ ip="106.51.197.116"
+ subject="a shell script that handles spaces in file names"
+ date="2013-12-31T10:24:06Z"
+ content="""
+I used the following shell pipeline to remove duplicate files in one go:
+
+    (1) git annex find --format='${key}:${file}\n' \
+    (2)    | cut -d '-' -f 4- \
+    (3)    | sort \
+    (4)    | uniq --all-repeated=separate -w 40 \
+    (5)    | awk -vRS= -vFS='\n' '{for (i = 2; i <= NF; i++) print $i}' \
+    (6)    | cut -d ':' -f 2- \
+    (7)    | xargs -d '\n' git rm
+
+1. Generate a list of keys and file names separated by a colon (':').
+2. Cut out the initial part of the key so that the hash is at the beginning of the line. The `-f 4-` ensures that dashes in the filename do not result in truncation.
+3. Sort the entire list.
+4. Uniquify and print duplicates in groups separated by blank lines. Use the first 40 characters, which matches the length of a SHA1 hash. Other hashes will require a different length.
+5. Use awk to print all but the first line in each group. The empty `-vRS` sets blank line as the record separator, and the `-vFS` sets newline as the field separator. The for-loop prints each field except the first.
+6. Cut out the key and keep only the file name by relying on the colon introduced in the first step.
+7. Use xargs to separate file names by newline, which takes care of spaces in the file names. Send this list of arguments to `git rm`.
+"""]]

doc/users/sameerds.mdwn

@@ -0,0 +1,3 @@
+Extremely grateful user of git-annex.
+
+[[http://sameer.sbuddhe.net/]]