doc/forum/Out_of_memory_error_in_fsck__44___whereis__44___find_and_status_cmds.mdwn

@@ -0,0 +1,78 @@
+Before I start on what's gone wrong, many thanks for a great program: finally a way of finding out where I've put all those files, and I enjoyed your talk in Australia. Not quite to New Zealand, but a good start :-)
+
+To play with git-annex, I decided to convert my ~/Downloads directory to a git-annex repository, as there were a wide variety of files in it, mostly easily replaceable, and also handy to have on multiple machines. In hindsight, probably wasn't a great idea, as I'd regularly forget it was a git-annex repo and move files and directories out manually, which caused all sorts of fun trying to sort out the dead symlinks. Then I after one update, the repository format changed, from WORM to xxx256 format which meant there were now both sorts in the GA object store.
+
+More recently I'd tried converting the repo to direct mode, you get the idea: lots of playing with git-annex commands, and now that I think about it, possibly some git commands too trying to repair missing files.
+
+Anyway I've ended up with a 27GB git-annex repo that now manages to kill git-annex whenever I try to check it using "git-annex fsck". 
+
+Not only does the fsck subcommand cause it to die, but also "find", "whereis" and "status". It dies on the same file (for find/whereis/fsck).
+
+e.g.
+
+    ... lots of stuff deleted ...
+    whereis 1wolf14.zip (2 copies) 
+            051f0b00-e265-11e1-894e-3b0b3f3844f2 -- Laptop
+            2c4e11e0-a1b4-11e1-9a02-73e17b04c00f -- here (myPC - Downloads)
+    ok
+    git-annex: out of memory (requested 2097152 bytes)
+
+Now "git status" for some repo data:
+
+    myPC:~/Downloads$ git annex status
+    supported backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
+    supported remote types: git S3 bup directory rsync web webdav glacier hook
+    repository mode: direct
+    trusted repositories: 0
+    semitrusted repositories: 4
+            00000000-0000-0000-0000-000000000001 -- web
+            051f0b00-e265-11e1-894e-3b0b3f3844f2 -- Laptop
+            2c4e11e0-a1b4-11e1-9a02-73e17b04c00f -- here (myPC - Downloads)
+            48fbe52a-a1b3-11e1-bb80-ebc15118871d -- netbk
+    untrusted repositories: 0
+    dead repositories: 0
+    transfers in progress: none
+    available local disk space: 100 gigabytes (+1 megabyte reserved)
+    local annex keys: 1719
+    local annex size: 27 gigabytes
+    known annex keys: git-annex: out of memory (requested 2097152 bytes)
+
+It always seems to die at about 3.5GB memory usage. This is running on Ubuntu 12.04, using the latest GA release built using cabal:
+
+    git-annex version: 4.20130227
+    local repository version: 3
+    default repository version: 3
+    supported repository versions: 3 4
+    upgrade supported from repository versions: 0 1 2
+
+There are also dead symlinks that point to directories that have meta-data but not the symlink target (manually line-wrapped):
+
+    myPC:~/Downloads$ ls -l precise*
+    lrwxrwxrwx 1 nino nino 194 Oct 21 12:48 precise-dvd-i386.iso -> 
+                 .git/annex/objects/0x/Xz/SHA256-s3590631424--
+                 b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0/
+                 SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0
+
+But looking in the symlink destination directory, there's no corresponding object, only metadata:
+
+    myPC:~/Downloads/.git/annex/objects/0x/Xz/SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0$ ls -l
+    total 8
+    -rw-rw-r-- 1 nino nino 30 Jan  8 23:15 SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0.cache
+    -rw-rw-r-- 1 nino nino 49 Jan  8 23:15 SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0.map
+
+But there is another version somewhere else.
+
+    -r--r--r-- 1 nino nino 3590631424 Mar 18  2012 ./.git/annex/objects/Kj/wM/WORM-s3590631424-m1331991509
+                        --precise-dvd-i386.iso/WORM-s3590631424-m1331991509--precise-dvd-i386.iso
+
+This actual file does exist in the "Used" directory:
+
+    -rw-r--r-- 1 nino nino 3.4G May 27  2012 precise-dvd-i386.iso
+
+I'm not so worried about the mangled repo - it's quite possibly because of clueless git/git-annex command usage - but the inability to use the fsck command is concerning
+
+I could just uninit everything, but as it dies prematurely, I'm not certain that all the contents would be restored.
+Any thoughts on how I can get git-annex (esp. fsck) to complete would be appreciated.
+
+Thanks
+Giovanni