From 49a95f151ce31f44417eee0463e9d334a581155f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Thu, 12 Apr 2012 17:51:11 -0400 Subject: [PATCH 001/220] add a todo item --- .../automatic_bookkeeping_watch_command.mdwn | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 doc/todo/automatic_bookkeeping_watch_command.mdwn diff --git a/doc/todo/automatic_bookkeeping_watch_command.mdwn b/doc/todo/automatic_bookkeeping_watch_command.mdwn new file mode 100644 index 0000000000..d7a3517a19 --- /dev/null +++ b/doc/todo/automatic_bookkeeping_watch_command.mdwn @@ -0,0 +1,42 @@ +A "git annex watch" command would help make git-annex usable by users who +don't know how to use git, or don't want to bother typing the git commands. +It would run, in the background, watching via inotify for changes, and +automatically annexing new files, etc. + +The blue sky goal would be something automated like dropbox, except fully +distributed. All files put into the repository would propigate out +to all the other clones of it, as network links allow. Note that while +dropbox allows modifying files, git-annex freezes them upon creation, +so this would not be 100% equivilant to dropbox. --[[Joey]] + +---- + +There is a `watch` branch in git that adds such a command, although currently +it only handles adding new files, and nothing else. To make this really +useful, it needs to: + +- notice deleted files and stage the deletion + (tricky; there's a race with add..) +- notice renamed files, auto-fix the symlink, and stage the new file location +- periodically auto-commit staged changes +- honor .gitignore, not adding files it excludes + +Also nice to have would be: + +- Somehow sync remotes, possibly using a push sync like dvcs-autosync + does, so they are immediately updated. +- Somehow get content that is unavilable. This is problimatic with inotify, + since we only get an event once the user has tried (and failed) to read + from the file. Perhaps instead, automatically copy content that is added + out to remotes, with the goal of all repos eventually getting a copy, + if df allows. +- Drop files that have not been used lately, or meet some other criteria + (as long as there's a copy elsewhere). +- Perhaps automatically dropunused files that have been deleted, + although I cannot see a way to do that, since by the time the inotify + deletion event arrives, the file is deleted, and we cannot see what + its symlink pointed to! Alternatievely, perhaps automatically + do an expensive unused/dropunused cleanup process. +- Support OSes other than Linux; it only uses inotify currently. + + From 8000fae9b259153c921b682afe3388791db20149 Mon Sep 17 00:00:00 2001 From: "http://mildred.pip.verisignlabs.com/" Date: Fri, 13 Apr 2012 07:28:11 +0000 Subject: [PATCH 002/220] Added a comment: Thank you a lot --- ...t_3_692f268218690437138ae0540c879425._comment | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 doc/forum/fail_to_git_annex_add_some_files:_getFileStatus:_does_not_exist__40__v_3.20111231__41__/comment_3_692f268218690437138ae0540c879425._comment diff --git a/doc/forum/fail_to_git_annex_add_some_files:_getFileStatus:_does_not_exist__40__v_3.20111231__41__/comment_3_692f268218690437138ae0540c879425._comment b/doc/forum/fail_to_git_annex_add_some_files:_getFileStatus:_does_not_exist__40__v_3.20111231__41__/comment_3_692f268218690437138ae0540c879425._comment new file mode 100644 index 0000000000..d5c84b431f --- /dev/null +++ b/doc/forum/fail_to_git_annex_add_some_files:_getFileStatus:_does_not_exist__40__v_3.20111231__41__/comment_3_692f268218690437138ae0540c879425._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="http://mildred.pip.verisignlabs.com/" + subject="Thank you a lot" + date="2012-04-13T07:28:10Z" + content=""" +Thank you, + +I imagined it was something like that. +I 'm just sorry I posted that on the forum and not on the bugs section (I hadn't discovered it at that time). but now, if people search for this error, they should find this. + +Note for Fedora users: unfortunately GHC 7.4 will not be shipped with Fedora 17 (which is still not released). The [feature page](https://fedoraproject.org/wiki/Features/GHC74) mention it for Fedora 18. I feel like I am using debian ... outdated packages the day of the release. + +And many thanks for this wonderful piece of software. + +Mildred +"""]] From ebc91c7637b8f245d467742fc33a577ca9516105 Mon Sep 17 00:00:00 2001 From: "http://mildred.pip.verisignlabs.com/" Date: Fri, 13 Apr 2012 07:57:28 +0000 Subject: [PATCH 003/220] Installation by hand using cabal / Setup.hs --- doc/install.mdwn | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/doc/install.mdwn b/doc/install.mdwn index 04d961e006..b85700e661 100644 --- a/doc/install.mdwn +++ b/doc/install.mdwn @@ -50,3 +50,13 @@ To build and use git-annex, you will need: * [ikiwiki](http://ikiwiki.info) (optional; used to build the docs) Then just [[download]] git-annex and run: `make; make install` + +## Installation by hand using cabal / Setup.hs + +You can fetch the dependencies using `cabal install`. Then, you can build git-annex by running: + + runhaskell Setup.hs configure --user + runhaskell Setup.hs build + runhaskell Setup.hs install --bindir=$HOME/bin + +The `--user` option configures the build so that it uses the packages you already have in `~/.cabal`. From 4837ad6dd3ed877513446cb948a696a90ccfdf16 Mon Sep 17 00:00:00 2001 From: "http://mildred.pip.verisignlabs.com/" Date: Fri, 13 Apr 2012 08:19:27 +0000 Subject: [PATCH 004/220] --- doc/install.mdwn | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/install.mdwn b/doc/install.mdwn index b85700e661..33426d637f 100644 --- a/doc/install.mdwn +++ b/doc/install.mdwn @@ -57,6 +57,7 @@ You can fetch the dependencies using `cabal install`. Then, you can build git-an runhaskell Setup.hs configure --user runhaskell Setup.hs build - runhaskell Setup.hs install --bindir=$HOME/bin + runhaskell Setup.hs install -The `--user` option configures the build so that it uses the packages you already have in `~/.cabal`. +The `--user` option configures the build so that it uses the packages you already have in `~/.cabal`. Binaries +will be installed in `~/.cabal/bin`, you'll need it in your PATH. From d712f5b0c4d96fc63a2a42c6e7af6139f4357733 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawm2AOTJmbCbGvmW5fxACaREraMnEVrCofo" Date: Fri, 13 Apr 2012 13:26:09 +0000 Subject: [PATCH 005/220] --- doc/bugs/sensitive.mdwn | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/bugs/sensitive.mdwn diff --git a/doc/bugs/sensitive.mdwn b/doc/bugs/sensitive.mdwn new file mode 100644 index 0000000000..21c2c4ec47 --- /dev/null +++ b/doc/bugs/sensitive.mdwn @@ -0,0 +1,17 @@ +What steps will reproduce the problem? + +> Building git-annex on the ghc7.0 branch on a Mac with the default case-insensitive file system + +What is the expected output? What do you see instead? + +> Expected: build successfully; instead: + + ld: duplicate symbol _UtilityziDiskFree_zdwa_info in dist/build/git-annex/git-annex-tmp/Utility/diskfree.o and dist/build/git-annex/git-annex-tmp/Utility/DiskFree.o for architecture x86_64 + +What version of git-annex are you using? On what operating system? + +> commit `0bd5c90ef0518f75d52f0c5889422d8233df847d` on a Mac OS 10.6 and 10.7, using the Haskell Platform 2012.04 + +Please provide any additional information below. + +> The problem is that since `DiskFree.hs` generates `DiskFree.o` and `diskfree.c` generates `diskfree.o`, a case-insensitive file system overwrites one object file with the other. Renaming `diskfree.c` to `diskfreec.c` and changing the corresponding filenames in `git-annex.cabal` fixes the problem. From 45a3cbc55bcc1462241e44ffa2b22d37895cfcd9 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawm2AOTJmbCbGvmW5fxACaREraMnEVrCofo" Date: Fri, 13 Apr 2012 13:47:14 +0000 Subject: [PATCH 006/220] rename bugs/sensitive.mdwn to bugs/case-insensitive.mdwn --- doc/bugs/{sensitive.mdwn => case-insensitive.mdwn} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename doc/bugs/{sensitive.mdwn => case-insensitive.mdwn} (100%) diff --git a/doc/bugs/sensitive.mdwn b/doc/bugs/case-insensitive.mdwn similarity index 100% rename from doc/bugs/sensitive.mdwn rename to doc/bugs/case-insensitive.mdwn From cc70792772f3254fec97bdaaeda8a11bac43907a Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 13 Apr 2012 11:13:58 -0400 Subject: [PATCH 007/220] update with manual git clone, and ghc7.0 build Seems that Fedora is going to ship with an outdated GHC. Pity. --- doc/install/Fedora.mdwn | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/doc/install/Fedora.mdwn b/doc/install/Fedora.mdwn index 7e983597b2..97ccf38ef4 100644 --- a/doc/install/Fedora.mdwn +++ b/doc/install/Fedora.mdwn @@ -1,7 +1,15 @@ -Installation recipe for Fedora 14. +Installation recipe for Fedora 14 thruough 17. 
 sudo yum install ghc cabal-install
-sudo cabal update
-cabal install git-annex --bindir=$HOME/bin
+git clone git://git.kitenet.net/git-annex
+cd git-annex
+git checkout ghc 7.0
+cabal update
+cabal configure
+cabal build
+cabal install --bindir=$HOME/bin
+ +Note: You can't just use `cabal install git-annex`, because Fedora does +not yet ship ghc 7.4. From 64c00933479a20e57f11f0cf20b88a6a68b16d2b Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 13 Apr 2012 11:15:27 -0400 Subject: [PATCH 008/220] move manual cabal install into its own page, and simplify it --- doc/install.mdwn | 14 +++----------- doc/install/cabal.mdwn | 15 +++++++++++++++ 2 files changed, 18 insertions(+), 11 deletions(-) create mode 100644 doc/install/cabal.mdwn diff --git a/doc/install.mdwn b/doc/install.mdwn index 33426d637f..a02d9d2c74 100644 --- a/doc/install.mdwn +++ b/doc/install.mdwn @@ -16,6 +16,9 @@ As a haskell package, git-annex can be installed using cabal. For example: cabal install git-annex --bindir=$HOME/bin +The above downloads the latest release. Alternatively, you can [[download]] +it yourself and [[manually_build_with_cabal|install/cabal]]. + ## Installation by hand To build and use git-annex, you will need: @@ -50,14 +53,3 @@ To build and use git-annex, you will need: * [ikiwiki](http://ikiwiki.info) (optional; used to build the docs) Then just [[download]] git-annex and run: `make; make install` - -## Installation by hand using cabal / Setup.hs - -You can fetch the dependencies using `cabal install`. Then, you can build git-annex by running: - - runhaskell Setup.hs configure --user - runhaskell Setup.hs build - runhaskell Setup.hs install - -The `--user` option configures the build so that it uses the packages you already have in `~/.cabal`. Binaries -will be installed in `~/.cabal/bin`, you'll need it in your PATH. diff --git a/doc/install/cabal.mdwn b/doc/install/cabal.mdwn new file mode 100644 index 0000000000..fe7b025e12 --- /dev/null +++ b/doc/install/cabal.mdwn @@ -0,0 +1,15 @@ +As a haskell package, git-annex can be installed using cabal. For example: + + cabal update + cabal install git-annex --bindir=$HOME/bin + +The above downloads the latest release and installs it into a ~/bin/ +directory, which you can put in your PATH. + +But maybe you want something newer (or older). Then [[download]] the version +you want, and use cabal as follows inside its source tree: + + cabal update + cabal configure + cabal build + cabal install --bindir=$HOME/bin From c56cfeba9ec134d61ce8da8fe351722ddb7ec00f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 13 Apr 2012 11:19:56 -0400 Subject: [PATCH 009/220] word wrap --- doc/install/openSUSE.mdwn | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/doc/install/openSUSE.mdwn b/doc/install/openSUSE.mdwn index 0383cbbf2a..73cbe585f6 100644 --- a/doc/install/openSUSE.mdwn +++ b/doc/install/openSUSE.mdwn @@ -1,4 +1,12 @@ -Unfortunately there is currently no git-annex rpm available for openSUSE; however it is possible to build it via cabal or from source as described on the [[install]] page. Fulfilling the dependencies listed on that page should not be a problem, except for obtaining a suitable version of the Haskell library. +Unfortunately there is currently no git-annex rpm available for openSUSE; +however it is possible to build it via cabal or from source as described on +the [[install]] page. Fulfilling the dependencies listed on that page +should not be a problem, except for obtaining a suitable version of the +Haskell library. -The last [official release of Haskell for openSUSE](https://build.opensuse.org/project/show?project=devel:languages:haskell) is quite old, and may not satisfy the dependencies needed by git-annex. Fortunately [searching the openSUSE build service](http://software.opensuse.org/search?q=cabal&baseproject=openSUSE%3A11.4&lang=en&include_home=true&exclude_debug=true) reveals that Peter Trommler has built a [newer Haskell suite](https://build.opensuse.org/project/show?project=home%3Aptrommler%3Adevel%3Alanguages%3Ahaskell) based on ghc 7.2. -To install this, simply click on the relevant "1-Click Install" link in the openSUSE build service search results. +The last [official release of Haskell for openSUSE](https://build.opensuse.org/project/show?project=devel:languages:haskell) +is quite old, and may not satisfy the dependencies needed by git-annex. +Fortunately [searching the openSUSE build service](http://software.opensuse.org/search?q=cabal&baseproject=openSUSE%3A11.4&lang=en&include_home=true&exclude_debug=true) +reveals that Peter Trommler has built a [newer Haskell suite](https://build.opensuse.org/project/show?project=home%3Aptrommler%3Adevel%3Alanguages%3Ahaskell) +based on ghc 7.2. To install this, simply click on the relevant +"1-Click Install" link in the openSUSE build service search results. From fdb246044c7b64802a9d0e90b92a978105a3704f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 13 Apr 2012 11:24:25 -0400 Subject: [PATCH 010/220] download updates --- doc/download.mdwn | 26 +++++++++---------- ..._fbd8b6d39e9d3c71791551358c863966._comment | 8 ------ ..._f85f72b33aedc3425f0c0c47867d02f3._comment | 8 ------ ..._cf6044ebe99f71158034e21197228abd._comment | 10 ------- ..._10fc013865c7542c2ed9d6c0963bb391._comment | 9 ------- ..._c6b1bc40226fc2c8ba3e558150856992._comment | 8 ------ ..._3a52993d3553deb9a413debec9a5f92d._comment | 11 -------- ..._a5eebd214b135f34b18274a682211943._comment | 8 ------ ..._59a976de6c7d333709b92f7cd5830850._comment | 8 ------ doc/install/Fedora.mdwn | 2 +- 10 files changed, 13 insertions(+), 85 deletions(-) delete mode 100644 doc/download/comment_1_fbd8b6d39e9d3c71791551358c863966._comment delete mode 100644 doc/download/comment_2_f85f72b33aedc3425f0c0c47867d02f3._comment delete mode 100644 doc/download/comment_3_cf6044ebe99f71158034e21197228abd._comment delete mode 100644 doc/download/comment_4_10fc013865c7542c2ed9d6c0963bb391._comment delete mode 100644 doc/download/comment_5_c6b1bc40226fc2c8ba3e558150856992._comment delete mode 100644 doc/download/comment_6_3a52993d3553deb9a413debec9a5f92d._comment delete mode 100644 doc/download/comment_7_a5eebd214b135f34b18274a682211943._comment delete mode 100644 doc/download/comment_8_59a976de6c7d333709b92f7cd5830850._comment diff --git a/doc/download.mdwn b/doc/download.mdwn index 30307f3088..64eee64c80 100644 --- a/doc/download.mdwn +++ b/doc/download.mdwn @@ -7,11 +7,9 @@ Other mirrors of the git repository: * `git://git.kitenet.net/git-annex` [[gitweb](http://git.kitenet.net/?p=git-annex.git;a=summary)] * [at github](https://github.com/joeyh/git-annex) -To download a tarball of a particular release, use an url like - - -From time to time, releases of git-annex are uploaded -[to hackage](http://hackage.haskell.org/package/git-annex). +Releases of git-annex are uploaded +[to hackage](http://hackage.haskell.org/package/git-annex). Get your +tarballs there, if you need them. Some operating systems include git-annex in easily prepackaged form and others need some manual work. See [[install]] for details. @@ -20,17 +18,17 @@ others need some manual work. See [[install]] for details. The git repository has some branches: -* `debian-stable` contains the latest backport of git-annex to Debian - stable. -* `no-s3` disables the S3 special remote, for systems that lack the - necessary haskell library. (merge it into master if you need it) -* `no-bloom` avoids using bloom filters. (merge it into master if you need it) -* `old-monad-control` is for systems that don't have a newer monad-control - library. -* `tweak-fetch` adds support for the git tweak-fetch hook, which has - been proposed and implemented but not yet accepted into git. * `ghc7.0` supports versions of ghc older than 7.4, which had a major change to filename encoding. +* `old-monad-control` is for systems that don't have a newer monad-control + library. +* `no-s3` disables the S3 special remote, for systems that lack the + necessary haskell library. (merge it into master if you need it) +* `no-bloom` avoids using bloom filters. (merge it into master if you need it) +* `debian-stable` contains the latest backport of git-annex to Debian + stable. +* `tweak-fetch` adds support for the git tweak-fetch hook, which has + been proposed and implemented but not yet accepted into git. * `setup` contains configuration for this website * `pristine-tar` contains [pristine-tar](http://kitenet.net/~joey/code/pristine-tar) data to create tarballs of any past git-annex release. diff --git a/doc/download/comment_1_fbd8b6d39e9d3c71791551358c863966._comment b/doc/download/comment_1_fbd8b6d39e9d3c71791551358c863966._comment deleted file mode 100644 index 488e005278..0000000000 --- a/doc/download/comment_1_fbd8b6d39e9d3c71791551358c863966._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://peter-simons.myopenid.com/" - ip="84.189.2.244" - subject="Please provide stable tarballs or zipfiles" - date="2011-03-22T13:06:58Z" - content=""" -I'm trying to package git annex for ArchLinux and NixOS. That task would be a *lot* easier, if there were proper release archives available for download. The Gitweb site offers to create snapshot tarballs on the fly, but those tarballs have a different SHA hash every time they're generated, so they cannot be used for the purposes of a distribution. A simple solution for this problem would be to enable snapshots in zip format (because zip files look the same every time they're generated). -"""]] diff --git a/doc/download/comment_2_f85f72b33aedc3425f0c0c47867d02f3._comment b/doc/download/comment_2_f85f72b33aedc3425f0c0c47867d02f3._comment deleted file mode 100644 index 5441c3e4ce..0000000000 --- a/doc/download/comment_2_f85f72b33aedc3425f0c0c47867d02f3._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus" - nickname="Jimmy" - subject="comment 2" - date="2011-03-22T14:01:37Z" - content=""" -maybe snag tarballs from ? -"""]] diff --git a/doc/download/comment_3_cf6044ebe99f71158034e21197228abd._comment b/doc/download/comment_3_cf6044ebe99f71158034e21197228abd._comment deleted file mode 100644 index b72b848f80..0000000000 --- a/doc/download/comment_3_cf6044ebe99f71158034e21197228abd._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://joey.kitenet.net/" - nickname="joey" - subject="comment 3" - date="2011-03-22T18:09:21Z" - content=""" -The tarballs produced by gitweb are actually stable. They are wrapped in a gz file with a varying timestamp however. It might be nice if gitweb passed --no-name to gzip to avoid that inconsistency. - -git-annex also has a [pristine-tar](http://kitenet.net/~joey/code/pristine-tar/) branch in git that can be used to recreate the tarballs I upload to Debian. -"""]] diff --git a/doc/download/comment_4_10fc013865c7542c2ed9d6c0963bb391._comment b/doc/download/comment_4_10fc013865c7542c2ed9d6c0963bb391._comment deleted file mode 100644 index 9bb9aa8ae3..0000000000 --- a/doc/download/comment_4_10fc013865c7542c2ed9d6c0963bb391._comment +++ /dev/null @@ -1,9 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawnOvt3TwSSDOLnoVzDNbOP1qO9OmNH5s0s" - nickname="Fraser" - subject="gitweb supplies --no-name as of 1.7.5.1" - date="2011-05-19T08:19:02Z" - content=""" -git v1.7.5.1 fixes the gitweb gzip issue. If the git instance is updated we -can have stable distributions (and I can finally write a FreeBSD port ^_^) -"""]] diff --git a/doc/download/comment_5_c6b1bc40226fc2c8ba3e558150856992._comment b/doc/download/comment_5_c6b1bc40226fc2c8ba3e558150856992._comment deleted file mode 100644 index 76ba75edc4..0000000000 --- a/doc/download/comment_5_c6b1bc40226fc2c8ba3e558150856992._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://joey.kitenet.net/" - nickname="joey" - subject="comment 5" - date="2011-05-19T16:10:35Z" - content=""" -Hmm, I've upgraded to that version, but I see nothing in its changelog, commit log, code, or runtime behavior to indicate that it's producing stable gzip output. -"""]] diff --git a/doc/download/comment_6_3a52993d3553deb9a413debec9a5f92d._comment b/doc/download/comment_6_3a52993d3553deb9a413debec9a5f92d._comment deleted file mode 100644 index 0dbd88b1e5..0000000000 --- a/doc/download/comment_6_3a52993d3553deb9a413debec9a5f92d._comment +++ /dev/null @@ -1,11 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawnOvt3TwSSDOLnoVzDNbOP1qO9OmNH5s0s" - nickname="Fraser" - subject="comment 6" - date="2011-05-22T23:02:39Z" - content=""" -Whups, the fix landed in git's `maint' branch just after 1.7.5 but 1.7.5.1 was -tagged on a different branch. - -Will look closer in future, and let you know when it's really released. -"""]] diff --git a/doc/download/comment_7_a5eebd214b135f34b18274a682211943._comment b/doc/download/comment_7_a5eebd214b135f34b18274a682211943._comment deleted file mode 100644 index 9960e0ea85..0000000000 --- a/doc/download/comment_7_a5eebd214b135f34b18274a682211943._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawnOvt3TwSSDOLnoVzDNbOP1qO9OmNH5s0s" - nickname="Fraser" - subject="comment 7" - date="2011-05-27T01:27:37Z" - content=""" -v1.7.5.3 has it. -"""]] diff --git a/doc/download/comment_8_59a976de6c7d333709b92f7cd5830850._comment b/doc/download/comment_8_59a976de6c7d333709b92f7cd5830850._comment deleted file mode 100644 index 5aa4f8c94a..0000000000 --- a/doc/download/comment_8_59a976de6c7d333709b92f7cd5830850._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="http://joey.kitenet.net/" - nickname="joey" - subject="comment 8" - date="2011-05-28T16:04:51Z" - content=""" -And that is now installed on kitenet.net and verified to work. -"""]] diff --git a/doc/install/Fedora.mdwn b/doc/install/Fedora.mdwn index 97ccf38ef4..cfdeeb8826 100644 --- a/doc/install/Fedora.mdwn +++ b/doc/install/Fedora.mdwn @@ -2,7 +2,7 @@ Installation recipe for Fedora 14 thruough 17. 
 sudo yum install ghc cabal-install
-git clone git://git.kitenet.net/git-annex
+git clone git://git-annex.branchable.com/ git-annex
 cd git-annex
 git checkout ghc 7.0
 cabal update

From 3642c723209bfb117b16c12f0ad83ef0fe025613 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 11:26:39 -0400
Subject: [PATCH 011/220] Renamed diskfree.c to avoid OSX case insensativity
 bug.

---
 Makefile                              | 2 +-
 Utility/DiskFree.hs                   | 2 +-
 Utility/{diskfree.c => libdiskfree.c} | 0
 Utility/{diskfree.h => libdiskfree.h} | 0
 debian/changelog                      | 1 +
 doc/bugs/case-insensitive.mdwn        | 3 +++
 git-annex.cabal                       | 6 +++---
 7 files changed, 9 insertions(+), 5 deletions(-)
 rename Utility/{diskfree.c => libdiskfree.c} (100%)
 rename Utility/{diskfree.h => libdiskfree.h} (100%)

diff --git a/Makefile b/Makefile
index b935140576..eb30a3833c 100644
--- a/Makefile
+++ b/Makefile
@@ -12,7 +12,7 @@ GHCMAKE=ghc $(GHCFLAGS) --make
 bins=git-annex
 mans=git-annex.1 git-annex-shell.1
 sources=Build/SysConfig.hs Utility/Touch.hs
-clibs=Utility/diskfree.o
+clibs=Utility/libdiskfree.o
 
 all=$(bins) $(mans) docs
 
diff --git a/Utility/DiskFree.hs b/Utility/DiskFree.hs
index bde6ef8caa..8d07afaf2d 100644
--- a/Utility/DiskFree.hs
+++ b/Utility/DiskFree.hs
@@ -15,7 +15,7 @@ import Foreign.C.Types
 import Foreign.C.String
 import Foreign.C.Error
 
-foreign import ccall unsafe "diskfree.h diskfree" c_diskfree
+foreign import ccall unsafe "libdiskfree.h diskfree" c_diskfree
 	:: CString -> IO CULLong
 
 getDiskFree :: String -> IO (Maybe Integer)
diff --git a/Utility/diskfree.c b/Utility/libdiskfree.c
similarity index 100%
rename from Utility/diskfree.c
rename to Utility/libdiskfree.c
diff --git a/Utility/diskfree.h b/Utility/libdiskfree.h
similarity index 100%
rename from Utility/diskfree.h
rename to Utility/libdiskfree.h
diff --git a/debian/changelog b/debian/changelog
index 0884f75744..abc3ae5120 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -7,6 +7,7 @@ git-annex (3.20120407) UNRELEASED; urgency=low
     configuration setting, doing fuzzy matching using the restricted
     Damerau-Levenshtein edit distance, just as git does. This adds a build
     dependency on the haskell edit-distance library.
+  * Renamed diskfree.c to avoid OSX case insensativity bug.
 
  -- Joey Hess   Sun, 08 Apr 2012 12:23:42 -0400
 
diff --git a/doc/bugs/case-insensitive.mdwn b/doc/bugs/case-insensitive.mdwn
index 21c2c4ec47..a917f64c28 100644
--- a/doc/bugs/case-insensitive.mdwn
+++ b/doc/bugs/case-insensitive.mdwn
@@ -15,3 +15,6 @@ What version of git-annex are you using? On what operating system?
 Please provide any additional information below.
 
 > The problem is that since `DiskFree.hs` generates `DiskFree.o` and `diskfree.c` generates `diskfree.o`, a case-insensitive file system overwrites one object file with the other.  Renaming `diskfree.c` to `diskfreec.c` and changing the corresponding filenames in `git-annex.cabal` fixes the problem.
+
+>> Man, not this again. The 80's called, they want their 
+>> unix portability wars back. [[fixed|done]]. --[[Joey]]
diff --git a/git-annex.cabal b/git-annex.cabal
index 0f28589859..df40011895 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -34,17 +34,17 @@ Executable git-annex
    base >= 4.5, base < 5, monad-control, transformers-base, lifted-base,
    IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   Other-Modules: Utility.Touch
-  C-Sources: Utility/diskfree.c
+  C-Sources: Utility/libdiskfree.c
 
 Executable git-annex-shell
   Main-Is: git-annex-shell.hs
-  C-Sources: Utility/diskfree.c
+  C-Sources: Utility/libdiskfree.c
 
 Test-Suite test
   Type: exitcode-stdio-1.0
   Main-Is: test.hs
   Build-Depends: testpack, HUnit
-  C-Sources: Utility/diskfree.c
+  C-Sources: Utility/libdiskfree.c
 
 source-repository head
   type: git

From befda6f9d26ade107f0b9cec3ea7919fda64444f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 11:37:49 -0400
Subject: [PATCH 012/220] add cabal command to install build deps

---
 doc/install/Fedora.mdwn | 1 +
 doc/install/cabal.mdwn  | 1 +
 2 files changed, 2 insertions(+)

diff --git a/doc/install/Fedora.mdwn b/doc/install/Fedora.mdwn
index cfdeeb8826..68b0056f1a 100644
--- a/doc/install/Fedora.mdwn
+++ b/doc/install/Fedora.mdwn
@@ -6,6 +6,7 @@ git clone git://git-annex.branchable.com/ git-annex
 cd git-annex
 git checkout ghc 7.0
 cabal update
+cabal install --only-dependencies
 cabal configure
 cabal build
 cabal install --bindir=$HOME/bin
diff --git a/doc/install/cabal.mdwn b/doc/install/cabal.mdwn
index fe7b025e12..180208211c 100644
--- a/doc/install/cabal.mdwn
+++ b/doc/install/cabal.mdwn
@@ -10,6 +10,7 @@ But maybe you want something newer (or older). Then [[download]] the version
 you want, and use cabal as follows inside its source tree:
 
 	cabal update
+	cabal install --only-dependencies
 	cabal configure
 	cabal build
 	cabal install --bindir=$HOME/bin

From 7a36d8ed7cdece92058a25b30e4056992d96c715 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 12:22:20 -0400
Subject: [PATCH 013/220] link to fedora ITP bug

---
 doc/install/Fedora.mdwn | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/install/Fedora.mdwn b/doc/install/Fedora.mdwn
index 68b0056f1a..a4dd4f5c87 100644
--- a/doc/install/Fedora.mdwn
+++ b/doc/install/Fedora.mdwn
@@ -14,3 +14,5 @@ cabal install --bindir=$HOME/bin
 
 Note: You can't just use `cabal install git-annex`, because Fedora does
 not yet ship ghc 7.4.
+
+[Status of getting a Fedora package](https://bugzilla.redhat.com/show_bug.cgi?id=662259)

From a4bbdffd6ae5dbb7b57dbaf6c4263ba353f9239b Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:19:04 -0400
Subject: [PATCH 014/220] wrap

---
 doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
index 14ca838cf1..17605ee0d0 100644
--- a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
+++ b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
@@ -7,6 +7,11 @@ oh my god, git-annex is amazing
 this is the revolution in fucking with gigantic piles of files that I've been waiting for
-And then my own story: I have a ton of drives. I have a lot of servers. I live in a cabin on **dialup** and often have 1 hour on broadband in a week to get everything I need. Without git-annex, managing all this would not be possible. It works perfectly for me, not a surprise since I wrote it, but still, it's a different level of "perfect" than anything I could put together before. +And then my own story: I have a ton of drives. I have a lot of servers. I +live in a cabin on **dialup** and often have 1 hour on broadband in a week +to get everything I need. Without git-annex, managing all this would not be +possible. It works perfectly for me, not a surprise since I wrote it, but +still, it's a different level of "perfect" than anything I could put +together before. [[!meta author=Joey]] From 1efc317a1da1c308c2cc8b2efeb019563308d833 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 13 Apr 2012 15:33:44 -0400 Subject: [PATCH 015/220] add some embedded tweets --- ...ll_us_how_you__39__re_using_git-annex.mdwn | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn index 17605ee0d0..ce4ccee31d 100644 --- a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn +++ b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn @@ -1,6 +1,19 @@ -Tell your git-annex stories here. Feel free to give as little or as much detail as appropriate about how you're using it. How's it working out for you? +Tell your git-annex stories here. Feel free to give as little or as much detail +as appropriate about how you're using it. How's it working out for you? -I'll start with a comment a user just posted to IRC: +I'll start with some tweets and comments from IRC: + + + + + + + + +

I want #git-annex whereis for all the stuff (not) in my room.

— topr (@derwelle) February 22, 2012 + 
 oh my god, git-annex is amazing
@@ -12,6 +25,6 @@ live in a cabin on **dialup** and often have 1 hour on broadband in a week
 to get everything I need. Without git-annex, managing all this would not be
 possible. It works perfectly for me, not a surprise since I wrote it, but
 still, it's a different level of "perfect" than anything I could put
-together before.
+together before. --[[Joey]]
 
 [[!meta author=Joey]]

From df367c92e8dc35df474057b37bba45d37e7045fc Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:35:12 -0400
Subject: [PATCH 016/220] move

---
 ...ll_us_how_you__39__re_using_git-annex.mdwn | 26 +------------------
 doc/testimonials.mdwn                         | 23 ++++++++++++++++
 2 files changed, 24 insertions(+), 25 deletions(-)
 create mode 100644 doc/testimonials.mdwn

diff --git a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
index ce4ccee31d..d289b9f50a 100644
--- a/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
+++ b/doc/forum/tell_us_how_you__39__re_using_git-annex.mdwn
@@ -1,30 +1,6 @@
 Tell your git-annex stories here. Feel free to give as little or as much detail
 as appropriate about how you're using it. How's it working out for you?
 
-I'll start with some tweets and comments from IRC:
-
-
-
-
-
-
-
-
-
I want #git-annex whereis for all the stuff (not) in my room.
— topr (@derwelle) February 22, 2012
-
-
-
-oh my god, git-annex is amazing
-this is the revolution in fucking with gigantic piles of files that I've been waiting for
-

-
-And then my own story: I have a ton of drives. I have a lot of servers. I
-live in a cabin on **dialup** and often have 1 hour on broadband in a week
-to get everything I need. Without git-annex, managing all this would not be
-possible. It works perfectly for me, not a surprise since I wrote it, but
-still, it's a different level of "perfect" than anything I could put
-together before. --[[Joey]]
+See [[testimonials]] for some other stories.
 
 [[!meta author=Joey]]
diff --git a/doc/testimonials.mdwn b/doc/testimonials.mdwn
new file mode 100644
index 0000000000..b43fc76688
--- /dev/null
+++ b/doc/testimonials.mdwn
@@ -0,0 +1,23 @@
+
+
+
+
+
+
+
+
I want #git-annex whereis for all the stuff (not) in my room.
— topr (@derwelle) February 22, 2012
+
+
+
+oh my god, git-annex is amazing
+this is the revolution in fucking with gigantic piles of files that I've been waiting for
+

+
+And then my own story: I have a ton of drives. I have a lot of servers. I
+live in a cabin on **dialup** and often have 1 hour on broadband in a week
+to get everything I need. Without git-annex, managing all this would not be
+possible. It works perfectly for me, not a surprise since I wrote it, but
+still, it's a different level of "perfect" than anything I could put
+together before. --[[Joey]]

From a077b75a80440c18b4229c2dc4028dce961df83d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:37:31 -0400
Subject: [PATCH 017/220] markup

---
 doc/testimonials.mdwn | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/doc/testimonials.mdwn b/doc/testimonials.mdwn
index b43fc76688..a340e6ebc7 100644
--- a/doc/testimonials.mdwn
+++ b/doc/testimonials.mdwn
@@ -1,14 +1,13 @@
+

 
 
-
 
 
-
 
 
I want #git-annex whereis for all the stuff (not) in my room.
— topr (@derwelle) February 22, 2012
 
+

 
 
 oh my god, git-annex is amazing

From 3d2cb43d1be45e9417707989752735a1d34b1c4e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:38:37 -0400
Subject: [PATCH 018/220] link

---
 doc/index.mdwn | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/index.mdwn b/doc/index.mdwn
index a54a1073e7..2de17df3ce 100644
--- a/doc/index.mdwn
+++ b/doc/index.mdwn
@@ -13,6 +13,7 @@ To get a feel for it, see the [[walkthrough]] or read about [[how_it_works]].
 * [[forum]]
 * [[comments]]
 * [[contact]]
+* [[testimonials]]
 * Flattr this
 
 [[News]]:

From 46a9f902fe5400e1d841b98f66d66e401868aaec Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:43:47 -0400
Subject: [PATCH 019/220] pasto

---
 doc/testimonials.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/testimonials.mdwn b/doc/testimonials.mdwn
index a340e6ebc7..60d53f0509 100644
--- a/doc/testimonials.mdwn
+++ b/doc/testimonials.mdwn
@@ -4,7 +4,7 @@
 
 
 
-
 
 
 

From becf0628b7bd86e93f8d84f2456460b4c198f17a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:44:12 -0400
Subject: [PATCH 020/220] update

---
 doc/testimonials.mdwn | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/testimonials.mdwn b/doc/testimonials.mdwn
index 60d53f0509..65bc5a5fe2 100644
--- a/doc/testimonials.mdwn
+++ b/doc/testimonials.mdwn
@@ -9,6 +9,7 @@
 
 
 
+Seen on IRC:
 
 oh my god, git-annex is amazing
 this is the revolution in fucking with gigantic piles of files that I've been waiting for

From 2cc13d1a4ad69943069e0eb91f6172eeba51733d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 13 Apr 2012 15:49:13 -0400
Subject: [PATCH 021/220] add a missing step

---
 doc/bare_repositories.mdwn | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/bare_repositories.mdwn b/doc/bare_repositories.mdwn
index bf56d81446..f40277d61d 100644
--- a/doc/bare_repositories.mdwn
+++ b/doc/bare_repositories.mdwn
@@ -26,6 +26,7 @@ Here is a quick example of how to set this up, using `origin` as the remote name
 On the server:
 
     mkdir bare-annex
+    cd bare-annex
     git init --bare
     git annex init origin
 

From 65d393a4bcff7bb5f406c18a45c3e8c9fbd28736 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Fri, 13 Apr 2012 19:49:23 +0000
Subject: [PATCH 022/220] Comment moderation

---
 ..._2_9f51947b35ee04e473655e20d56c740a._comment | 16 ++++++++++++++++
 ..._1_4884803ddee7f642a3ac995a19967a6a._comment | 17 +++++++++++++++++
 2 files changed, 33 insertions(+)
 create mode 100644 doc/forum/Preserving_file_access_rights_in_directory_tree_below_objects__47__/comment_2_9f51947b35ee04e473655e20d56c740a._comment
 create mode 100644 doc/forum/tell_us_how_you__39__re_using_git-annex/comment_1_4884803ddee7f642a3ac995a19967a6a._comment

diff --git a/doc/forum/Preserving_file_access_rights_in_directory_tree_below_objects__47__/comment_2_9f51947b35ee04e473655e20d56c740a._comment b/doc/forum/Preserving_file_access_rights_in_directory_tree_below_objects__47__/comment_2_9f51947b35ee04e473655e20d56c740a._comment
new file mode 100644
index 0000000000..5d632ab860
--- /dev/null
+++ b/doc/forum/Preserving_file_access_rights_in_directory_tree_below_objects__47__/comment_2_9f51947b35ee04e473655e20d56c740a._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlB7-aXsqwzOi2BIR_Q4sUF8sjj24H6F3c"
+ nickname="Claudius"
+ subject="comment 2"
+ date="2012-01-23T19:39:17Z"
+ content="""
+Thank you for your comment! Indeed, setting the umask to, for example, 022 has the desired effect that annex/objects etc. are executable (and in this special case also writable), my previous umask setting was 077; the \"strange\" permissions on the git directories was probably due to --shared=all, and the mode of \"440\" on the files within the git-annex tree is correct (the original file was 640 and stripped of its write permission).
+
+Using this umask setting and newgrp to switch the default group, I was successfully able to set up the repositories.
+
+However, I would like to suggest adding the execute bit to the directories below .git/annex/objects/ per default, even if the umask of the current shell differs. As the correct rights are already preserved in the actual files (minus their write permission) together with correct owner and group, the files are still protected the same way as previously, and because +x does not allow directory listings, no additional information can leak out either. Not having to set the umask to something \"sensible\" before operating git-annex would be a huge plus, too :)
+
+The reason why I am not running MPD as my user is that I am a bit wary of running an application even exposed to the local network as my main user, and I see nothing wrong with running it as its own user.
+
+Thank you again for your help and the time you put into this project!
+"""]]
diff --git a/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_1_4884803ddee7f642a3ac995a19967a6a._comment b/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_1_4884803ddee7f642a3ac995a19967a6a._comment
new file mode 100644
index 0000000000..2351378bca
--- /dev/null
+++ b/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_1_4884803ddee7f642a3ac995a19967a6a._comment
@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="http://mildred.fr/"
+ subject="just amazing"
+ date="2012-04-12T17:12:41Z"
+ content="""
+git-annex is just amazing. I just started using it and for once, I have hope to be able to organize my files a little better than now.
+
+Currently, I have a huge homedir. From time to time, I move file away in external hard drives, then forget about them. When I want to look at them back, I just can't because I have forgotten where they are. I have also a ton of files on those drives that I can't access because they are not indexed. With git-annex I have hope to put all of these files on a git repository. I will be able to see them everywhere, and find them when I need to.
+
+I might stop loosing files for once.
+
+I might avoid having multiple copies of the same things over and over again, without knowing so. and regain some more disk space.
+
+For the moment, I'm archiving my photographs. But there is one thing that might not go very well: directory hierarchies where everything is important (file owner, specific permissions, symlinks). I won't just be able to blindly annex all of these files. But for the moment I'll stick at archiving ocuments and it should be amazing.
+
+[Mildred](http://mildred.fr)
+"""]]

From dd610c913f5276d51f5f0d299757fb6f1a582d5c Mon Sep 17 00:00:00 2001
From: "http://cgray.myopenid.com/" 
Date: Sat, 14 Apr 2012 01:18:54 +0000
Subject: [PATCH 023/220] Added a comment

---
 ...comment_2_61f5054918e7b36c191454365bc7f3b7._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/tell_us_how_you__39__re_using_git-annex/comment_2_61f5054918e7b36c191454365bc7f3b7._comment

diff --git a/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_2_61f5054918e7b36c191454365bc7f3b7._comment b/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_2_61f5054918e7b36c191454365bc7f3b7._comment
new file mode 100644
index 0000000000..3bd981c5db
--- /dev/null
+++ b/doc/forum/tell_us_how_you__39__re_using_git-annex/comment_2_61f5054918e7b36c191454365bc7f3b7._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://cgray.myopenid.com/"
+ nickname="cgray"
+ subject="comment 2"
+ date="2012-04-14T01:18:53Z"
+ content="""
+Git-annex has really helped me with my media files.  I have a big NAS drive where I keep all my music, tv, and movies files, each in their own git annex.  I tend to keep the media that I want to watch or listen to on my laptop and then drop it when it is done.  This way I don't have too much on my laptop at any one time, but I have a nice selection for when I'm traveling and don't have access to my NAS.
+
+Additionally, I have a mp3 player that will format itself randomly every few months or so.  I keep my podcasts on it in a git annex and in a git annex on my laptop.  When I am done with a podcast, I can delete it from the mp3 player and then sync that information with my laptop.  With this method, I have a backup of what should be on my mp3 player, so I don't need to worry about losing it all when the mp3 player decides it's had enough.
+"""]]

From 142bde13cd33481f415a5355a9cd9e74a716e3a8 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 12:33:32 -0400
Subject: [PATCH 024/220] import juggling

---
 Utility/CopyFile.hs | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/Utility/CopyFile.hs b/Utility/CopyFile.hs
index c42506485b..01639ef2a7 100644
--- a/Utility/CopyFile.hs
+++ b/Utility/CopyFile.hs
@@ -7,10 +7,7 @@
 
 module Utility.CopyFile (copyFileExternal) where
 
-import System.Directory (doesFileExist, removeFile)
-import Control.Monad.IfElse
-
-import Utility.SafeCommand
+import Common
 import qualified Build.SysConfig as SysConfig
 
 {- The cp command is used, because I hate reinventing the wheel,

From 1ca41044e8fab2b2e859a482e2293582c04db81f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 14:01:14 -0400
Subject: [PATCH 025/220] cabal now installs git-annex-shell as a symlink to
 git-annex.

---
 Setup.hs           | 17 ++++++++++++++++-
 debian/changelog   |  1 +
 git-annex-shell.hs | 13 -------------
 git-annex.cabal    |  4 ----
 4 files changed, 17 insertions(+), 18 deletions(-)
 delete mode 100644 git-annex-shell.hs

diff --git a/Setup.hs b/Setup.hs
index 14e6a4ea71..c36d6e4fe1 100644
--- a/Setup.hs
+++ b/Setup.hs
@@ -1,12 +1,27 @@
 {- cabal setup file -}
 
 import Distribution.Simple
+import Distribution.Simple.LocalBuildInfo
+import Distribution.Simple.Setup
 import System.Cmd
+import System.FilePath
 
 import qualified Build.Configure as Configure
 
-main = defaultMainWithHooks simpleUserHooks { preConf = configure }
+main = defaultMainWithHooks simpleUserHooks
+	{ preConf = configure
+	, instHook = install
+	}
 
 configure _ _ = do
 	Configure.run Configure.tests
 	return (Nothing, [])
+
+install pkg_descr lbi userhooks flags = do
+	r <- (instHook simpleUserHooks) pkg_descr lbi userhooks flags
+	_ <- rawSystem "ln" ["-sf", "git-annex", 
+		bindir installDirs  "git-annex-shell"]
+	return r
+	where
+		installDirs = absoluteInstallDirs pkg_descr lbi $
+			fromFlag (copyDest defaultCopyFlags)
diff --git a/debian/changelog b/debian/changelog
index abc3ae5120..73b4f31b08 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -8,6 +8,7 @@ git-annex (3.20120407) UNRELEASED; urgency=low
     Damerau-Levenshtein edit distance, just as git does. This adds a build
     dependency on the haskell edit-distance library.
   * Renamed diskfree.c to avoid OSX case insensativity bug.
+  * cabal now installs git-annex-shell as a symlink to git-annex.
 
  -- Joey Hess   Sun, 08 Apr 2012 12:23:42 -0400
 
diff --git a/git-annex-shell.hs b/git-annex-shell.hs
deleted file mode 100644
index 08c1f9664d..0000000000
--- a/git-annex-shell.hs
+++ /dev/null
@@ -1,13 +0,0 @@
-{- git-annex-shell main program
- -
- - Copyright 2012 Joey Hess 
- -
- - Licensed under the GNU GPL version 3 or higher.
- -}
-
-import System.Environment
-
-import GitAnnexShell
-
-main :: IO ()
-main = run =<< getArgs
diff --git a/git-annex.cabal b/git-annex.cabal
index df40011895..6b1ebb42e6 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -36,10 +36,6 @@ Executable git-annex
   Other-Modules: Utility.Touch
   C-Sources: Utility/libdiskfree.c
 
-Executable git-annex-shell
-  Main-Is: git-annex-shell.hs
-  C-Sources: Utility/libdiskfree.c
-
 Test-Suite test
   Type: exitcode-stdio-1.0
   Main-Is: test.hs

From 626697b459669d934da6339117f6f4abfce16f38 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 14:22:33 -0400
Subject: [PATCH 026/220] cabal file now autodetects whether S3 support is
 available.

---
 Locations.hs     |  2 +-
 Makefile         |  2 +-
 Remote/List.hs   |  4 ++++
 debian/changelog |  1 +
 git-annex.cabal  | 10 +++++++++-
 5 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/Locations.hs b/Locations.hs
index d263f3d2ac..67abf2166a 100644
--- a/Locations.hs
+++ b/Locations.hs
@@ -124,7 +124,7 @@ gitAnnexBadDir r = addTrailingPathSeparator $ gitAnnexDir r  "bad"
 gitAnnexBadLocation :: Key -> Git.Repo -> FilePath
 gitAnnexBadLocation key r = gitAnnexBadDir r  keyFile key
 
-{- .git/annex/*unused is used to number possibly unused keys -}
+{- .git/annex/foounused is used to number possibly unused keys -}
 gitAnnexUnusedLog :: FilePath -> Git.Repo -> FilePath
 gitAnnexUnusedLog prefix r = gitAnnexDir r  (prefix ++ "unused")
 
diff --git a/Makefile b/Makefile
index eb30a3833c..87aa8c0767 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
 PREFIX=/usr
 IGNORE=-ignore-package monads-fd
-BASEFLAGS=-Wall $(IGNORE) -outputdir tmp -IUtility
+BASEFLAGS=-Wall $(IGNORE) -outputdir tmp -IUtility -cpp -DWITH_S3
 GHCFLAGS=-O2 $(BASEFLAGS)
 
 ifdef PROFILE
diff --git a/Remote/List.hs b/Remote/List.hs
index 57dfa43ebf..c09341fb53 100644
--- a/Remote/List.hs
+++ b/Remote/List.hs
@@ -18,7 +18,9 @@ import Config
 import Remote.Helper.Hooks
 
 import qualified Remote.Git
+#ifdef WITH_S3
 import qualified Remote.S3
+#endif
 import qualified Remote.Bup
 import qualified Remote.Directory
 import qualified Remote.Rsync
@@ -28,7 +30,9 @@ import qualified Remote.Hook
 remoteTypes :: [RemoteType]
 remoteTypes =
 	[ Remote.Git.remote
+#ifdef WITH_S3
 	, Remote.S3.remote
+#endif
 	, Remote.Bup.remote
 	, Remote.Directory.remote
 	, Remote.Rsync.remote
diff --git a/debian/changelog b/debian/changelog
index 73b4f31b08..7b9fcde3fc 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -9,6 +9,7 @@ git-annex (3.20120407) UNRELEASED; urgency=low
     dependency on the haskell edit-distance library.
   * Renamed diskfree.c to avoid OSX case insensativity bug.
   * cabal now installs git-annex-shell as a symlink to git-annex.
+  * cabal file now autodetects whether S3 support is available.
 
  -- Joey Hess   Sun, 08 Apr 2012 12:23:42 -0400
 
diff --git a/git-annex.cabal b/git-annex.cabal
index 6b1ebb42e6..58370daf4b 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -26,15 +26,23 @@ Description:
  etc that are associated with annexed files but that benefit from full
  revision control.
 
+Flag S3
+  Description: Enable S3 support
+
 Executable git-annex
   Main-Is: git-annex.hs
   Build-Depends: MissingH, hslogger, directory, filepath,
    unix, containers, utf8-string, network, mtl, bytestring, old-locale, time,
-   pcre-light, extensible-exceptions, dataenc, SHA, process, hS3, json, HTTP,
+   pcre-light, extensible-exceptions, dataenc, SHA, process, json, HTTP,
    base >= 4.5, base < 5, monad-control, transformers-base, lifted-base,
    IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   Other-Modules: Utility.Touch
   C-Sources: Utility/libdiskfree.c
+  Extensions: CPP
+
+  if flag(S3)
+    Build-Depends: hS3
+    CPP-Options: -DWITH_S3
 
 Test-Suite test
   Type: exitcode-stdio-1.0

From 61e5663097a2cf779d136b5a28500460293e9e3c Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 15:23:13 -0400
Subject: [PATCH 027/220] make cabal test work again

---
 git-annex.cabal | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/git-annex.cabal b/git-annex.cabal
index 58370daf4b..395042c9a6 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -47,8 +47,13 @@ Executable git-annex
 Test-Suite test
   Type: exitcode-stdio-1.0
   Main-Is: test.hs
-  Build-Depends: testpack, HUnit
+  Build-Depends: testpack, HUnit, MissingH, hslogger, directory, filepath,
+   unix, containers, utf8-string, network, mtl, bytestring, old-locale, time,
+   pcre-light, extensible-exceptions, dataenc, SHA, process, json, HTTP,
+   base >= 4.5, base < 5, monad-control, transformers-base, lifted-base,
+   IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   C-Sources: Utility/libdiskfree.c
+  Extensions: CPP
 
 source-repository head
   type: git

From 70538dac844f00ccbdd0f2d0283fbf4e707cb84a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 16:01:08 -0400
Subject: [PATCH 028/220] compute distance in correct direction

---
 Git/AutoCorrect.hs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Git/AutoCorrect.hs b/Git/AutoCorrect.hs
index a18bf56197..a1ef14779b 100644
--- a/Git/AutoCorrect.hs
+++ b/Git/AutoCorrect.hs
@@ -34,7 +34,7 @@ fuzzymatches :: String -> (c -> String) -> [c] -> [c]
 fuzzymatches input showchoice choices = fst $ unzip $
 	sortBy comparecost $ filter similarEnough $ zip choices costs
         where
-                distance v = restrictedDamerauLevenshteinDistance gitEditCosts v input
+                distance = restrictedDamerauLevenshteinDistance gitEditCosts input
                 costs = map (distance . showchoice) choices
                 comparecost a b = compare (snd a) (snd b)
                 similarEnough (_, cst) = cst < similarityFloor

From 42cbb41ada884b21d7bbd369003b71752260c44e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 14 Apr 2012 16:01:22 -0400
Subject: [PATCH 029/220] always run autocorrect code on fuzzy matches, even if
 there is just 1

---
 CmdLine.hs | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/CmdLine.hs b/CmdLine.hs
index 5330f40fc9..ebaef5369d 100644
--- a/CmdLine.hs
+++ b/CmdLine.hs
@@ -46,19 +46,19 @@ dispatch fuzzyok allargs allcmds commonoptions header getgitrepo = do
 	where
 		err msg = msg ++ "\n\n" ++ usage header allcmds commonoptions
 		cmd = Prelude.head cmds
-		(cmds, name, args) = findCmd fuzzyok allargs allcmds err
+		(fuzzy, cmds, name, args) = findCmd fuzzyok allargs allcmds err
 		(flags, params) = getOptCmd args cmd commonoptions err
-		checkfuzzy = when (length cmds > 1) $
+		checkfuzzy = when fuzzy $
 			inRepo $ Git.AutoCorrect.prepare name cmdname cmds
 
 {- Parses command line params far enough to find the Command to run, and
  - returns the remaining params.
  - Does fuzzy matching if necessary, which may result in multiple Commands. -}
-findCmd :: Bool -> Params -> [Command] -> (String -> String) -> ([Command], String, Params)
+findCmd :: Bool -> Params -> [Command] -> (String -> String) -> (Bool, [Command], String, Params)
 findCmd fuzzyok argv cmds err
 	| isNothing name = error $ err "missing command"
-	| not (null exactcmds) = (exactcmds, fromJust name, args)
-	| fuzzyok && not (null inexactcmds) = (inexactcmds, fromJust name, args)
+	| not (null exactcmds) = (False, exactcmds, fromJust name, args)
+	| fuzzyok && not (null inexactcmds) = (True, inexactcmds, fromJust name, args)
 	| otherwise = error $ err $ "unknown command " ++ fromJust name
 	where
 		(name, args) = findname argv []

From 4d221a7a80d9b02b523536b146fe94ce6448cfa5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 16 Apr 2012 16:35:47 -0400
Subject: [PATCH 030/220] the no-s3 branch is not needed, and there is a new
 no-ifelse branch

---
 doc/download.mdwn | 4 ++--
 doc/install.mdwn  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/doc/download.mdwn b/doc/download.mdwn
index 64eee64c80..f0f17e141d 100644
--- a/doc/download.mdwn
+++ b/doc/download.mdwn
@@ -22,8 +22,8 @@ The git repository has some branches:
   had a major change to filename encoding.
 * `old-monad-control` is for systems that don't have a newer monad-control
   library.
-* `no-s3` disables the S3 special remote, for systems that lack the
-  necessary haskell library. (merge it into master if you need it)
+* `no-ifelse` avoids using the IFelse library
+  (merge it into master if you need it)
 * `no-bloom` avoids using bloom filters. (merge it into master if you need it)
 * `debian-stable` contains the latest backport of git-annex to Debian
   stable.
diff --git a/doc/install.mdwn b/doc/install.mdwn
index a02d9d2c74..2590060e69 100644
--- a/doc/install.mdwn
+++ b/doc/install.mdwn
@@ -35,7 +35,7 @@ To build and use git-annex, you will need:
   * [TestPack](http://hackage.haskell.org/cgi-bin/hackage-scripts/package/testpack)
   * [QuickCheck 2](http://hackage.haskell.org/package/QuickCheck)
   * [HTTP](http://hackage.haskell.org/package/HTTP)
-  * [hS3](http://hackage.haskell.org/package/hS3)
+  * [hS3](http://hackage.haskell.org/package/hS3) (optional)
   * [json](http://hackage.haskell.org/package/json)
   * [IfElse](http://hackage.haskell.org/package/IfElse)
   * [bloomfilter](http://hackage.haskell.org/package/bloomfilter)

From aa353d1400512174ff3c5ccce89d1ebdd1a5be12 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 17 Apr 2012 18:37:40 -0400
Subject: [PATCH 031/220] use LANGUAGE CPP pragma, avoids running cpp on all
 the other sources

---
 Makefile       | 2 +-
 Remote/List.hs | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 87aa8c0767..94dc05a819 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
 PREFIX=/usr
 IGNORE=-ignore-package monads-fd
-BASEFLAGS=-Wall $(IGNORE) -outputdir tmp -IUtility -cpp -DWITH_S3
+BASEFLAGS=-Wall $(IGNORE) -outputdir tmp -IUtility -DWITH_S3
 GHCFLAGS=-O2 $(BASEFLAGS)
 
 ifdef PROFILE
diff --git a/Remote/List.hs b/Remote/List.hs
index c09341fb53..14a1771b48 100644
--- a/Remote/List.hs
+++ b/Remote/List.hs
@@ -1,3 +1,5 @@
+{-# LANGUAGE CPP #-}
+
 {- git-annex remote list
  -
  - Copyright 2011 Joey Hess 

From 840315c3507d6a3df0d747f423244c132e41e3fa Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 18 Apr 2012 12:22:22 -0400
Subject: [PATCH 032/220] releasing version 3.20120418

---
 debian/changelog | 4 ++--
 git-annex.cabal  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 7b9fcde3fc..e63e82f0e5 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,4 @@
-git-annex (3.20120407) UNRELEASED; urgency=low
+git-annex (3.20120418) unstable; urgency=low
 
   * bugfix: Adding a dotfile also caused all non-dotfiles to be added.
   * bup: Properly handle key names with spaces or other things that are
@@ -11,7 +11,7 @@ git-annex (3.20120407) UNRELEASED; urgency=low
   * cabal now installs git-annex-shell as a symlink to git-annex.
   * cabal file now autodetects whether S3 support is available.
 
- -- Joey Hess   Sun, 08 Apr 2012 12:23:42 -0400
+ -- Joey Hess   Wed, 18 Apr 2012 12:11:32 -0400
 
 git-annex (3.20120406) unstable; urgency=low
 
diff --git a/git-annex.cabal b/git-annex.cabal
index 395042c9a6..041b077dc5 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -1,5 +1,5 @@
 Name: git-annex
-Version: 3.20120407
+Version: 3.20120418
 Cabal-Version: >= 1.8
 License: GPL
 Maintainer: Joey Hess 

From f08943016708f8d4f25c557f255ef81e387bc522 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 18 Apr 2012 12:23:22 -0400
Subject: [PATCH 033/220] add news item for git-annex 3.20120418

---
 doc/news/version_3.20120230.mdwn | 13 -------------
 doc/news/version_3.20120418.mdwn | 12 ++++++++++++
 2 files changed, 12 insertions(+), 13 deletions(-)
 delete mode 100644 doc/news/version_3.20120230.mdwn
 create mode 100644 doc/news/version_3.20120418.mdwn

diff --git a/doc/news/version_3.20120230.mdwn b/doc/news/version_3.20120230.mdwn
deleted file mode 100644
index 52ac369e01..0000000000
--- a/doc/news/version_3.20120230.mdwn
+++ /dev/null
@@ -1,13 +0,0 @@
-git-annex 3.20120230 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * "here" can be used to refer to the current repository,
-     which can read better than the old "." (which still works too).
-   * Directory special remotes now support chunking files written to them,
-     avoiding writing files larger than a specified size.
-   * Add progress bar display to the directory special remote.
-   * Add configurable hooks that are run when git-annex starts and stops
-     using a remote: remote.name.annex-start-command and
-     remote.name.annex-stop-command
-   * Fix a bug in symlink calculation code, that triggered in rare
-     cases where an annexed file is in a subdirectory that nearly
-     matched to the .git/annex/object/xx/yy subdirectories."""]]
\ No newline at end of file
diff --git a/doc/news/version_3.20120418.mdwn b/doc/news/version_3.20120418.mdwn
new file mode 100644
index 0000000000..93968a83e6
--- /dev/null
+++ b/doc/news/version_3.20120418.mdwn
@@ -0,0 +1,12 @@
+git-annex 3.20120418 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * bugfix: Adding a dotfile also caused all non-dotfiles to be added.
+   * bup: Properly handle key names with spaces or other things that are
+     not legal git refs.
+   * git-annex (but not git-annex-shell) supports the git help.autocorrect
+     configuration setting, doing fuzzy matching using the restricted
+     Damerau-Levenshtein edit distance, just as git does. This adds a build
+     dependency on the haskell edit-distance library.
+   * Renamed diskfree.c to avoid OSX case insensativity bug.
+   * cabal now installs git-annex-shell as a symlink to git-annex.
+   * cabal file now autodetects whether S3 support is available."""]]
\ No newline at end of file

From d75771b0ab88ebec9b573fb1b3ef257e3b19c3ef Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 18 Apr 2012 13:17:30 -0400
Subject: [PATCH 034/220] clear errno after successful call

When preparing the debian stable backport, I am seeing a call to statvfs()
succeed, but also set errno to 2 (ENOENT). Not sure why this happens;
I am in a schroot when it does happen, or perhaps stable's libc is a little
broken and sets errno incorrectly. Anyway, it should be perfectly fine to
clear errno after the successful call, rather than before it.
---
 Utility/libdiskfree.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Utility/libdiskfree.c b/Utility/libdiskfree.c
index b68abd0c47..54e8c08942 100644
--- a/Utility/libdiskfree.c
+++ b/Utility/libdiskfree.c
@@ -58,9 +58,10 @@ unsigned long long int diskfree(const char *path) {
 	unsigned long long int available, blocksize;
 	struct STATSTRUCT buf;
 
-	errno = 0;
-	if (STATCALL(path, &buf) != 0)
+	if (STATCALL(path, &buf) != 0) {
 		return 0; /* errno is set */
+	}
+	errno = 0;
 
 	available = buf.f_bavail;
 	blocksize = buf.f_bsize;

From 37061c019d816ff9ddd30cf23833539102741844 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 18 Apr 2012 13:23:33 -0400
Subject: [PATCH 035/220] tweak

---
 Utility/DiskFree.hs   | 2 +-
 Utility/libdiskfree.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Utility/DiskFree.hs b/Utility/DiskFree.hs
index 8d07afaf2d..ff70705621 100644
--- a/Utility/DiskFree.hs
+++ b/Utility/DiskFree.hs
@@ -18,7 +18,7 @@ import Foreign.C.Error
 foreign import ccall unsafe "libdiskfree.h diskfree" c_diskfree
 	:: CString -> IO CULLong
 
-getDiskFree :: String -> IO (Maybe Integer)
+getDiskFree :: FilePath -> IO (Maybe Integer)
 getDiskFree path = withFilePath path $ \c_path -> do
 	free <- c_diskfree c_path
 	ifM (safeErrno <$> getErrno)
diff --git a/Utility/libdiskfree.c b/Utility/libdiskfree.c
index 54e8c08942..a37cb75713 100644
--- a/Utility/libdiskfree.c
+++ b/Utility/libdiskfree.c
@@ -58,10 +58,10 @@ unsigned long long int diskfree(const char *path) {
 	unsigned long long int available, blocksize;
 	struct STATSTRUCT buf;
 
-	if (STATCALL(path, &buf) != 0) {
+	if (STATCALL(path, &buf) != 0)
 		return 0; /* errno is set */
-	}
-	errno = 0;
+	else
+		errno = 0;
 
 	available = buf.f_bavail;
 	blocksize = buf.f_bsize;

From c908bd3b97eb84bbd8399951a1cf0ece4d824ea2 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 20 Apr 2012 11:31:30 -0400
Subject: [PATCH 036/220] some updates

---
 doc/internals.mdwn | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/doc/internals.mdwn b/doc/internals.mdwn
index b2fd1e5545..a69a747e5d 100644
--- a/doc/internals.mdwn
+++ b/doc/internals.mdwn
@@ -39,6 +39,9 @@ space and then the description, followed by a timestamp. Example:
 	e605dca6-446a-11e0-8b2a-002170d25c55 laptop timestamp=1317929189.157237s
 	26339d22-446b-11e0-9101-002170d25c55 usb disk timestamp=1317929330.769997s
 
+If there are multiple lines for the same uuid, the one with the most recent
+timestamp wins. git-annex union merges this and other files.
+
 ## `remotes.log`
 
 Holds persistent configuration settings for [[special_remotes]] such as
@@ -80,7 +83,7 @@ Example:
 These files are designed to be auto-merged using git's [[union merge driver|git-union-merge]].
 The timestamps allow the most recent information to be identified.
 
-## `remote/web/aaa/bbb/*.log`
+## `aaa/bbb/*.log.web`
 
 These log files record urls used by the
 [[web_special_remote|special_remotes/web]]. Their format is similar

From 262017e17d466599f5b17313d69c995e844d59c6 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 20 Apr 2012 14:57:57 -0400
Subject: [PATCH 037/220] export a more generalized checkDiskSpace

---
 Annex/Content.hs  | 45 +++++++++++++++++++++++----------------------
 Command/Unlock.hs |  3 +--
 2 files changed, 24 insertions(+), 24 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 8542d8775d..494c9c10c1 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -127,15 +127,15 @@ getViaTmp key action = do
 	-- When the temp file already exists, count the space
 	-- it is using as free.
 	e <- liftIO $ doesFileExist tmp
-	if e
-		then do
-			stat <- liftIO $ getFileStatus tmp
-			checkDiskSpace' (fromIntegral $ fileSize stat) key
-		else checkDiskSpace key
-
-	when e $ liftIO $ allowWrite tmp
-
-	getViaTmpUnchecked key action
+	alreadythere <- if e
+		then fromIntegral . fileSize <$> liftIO (getFileStatus tmp)
+		else return 0
+	ifM (checkDiskSpace Nothing key alreadythere)
+		( do
+			when e $ liftIO $ allowWrite tmp
+			getViaTmpUnchecked key action
+		, return False
+		)
 
 prepTmp :: Key -> Annex FilePath
 prepTmp key = do
@@ -169,22 +169,23 @@ withTmp key action = do
 	return res
 
 {- Checks that there is disk space available to store a given key,
- - throwing an error if not. -}
-checkDiskSpace :: Key -> Annex ()
-checkDiskSpace = checkDiskSpace' 0
-
-checkDiskSpace' :: Integer -> Key -> Annex ()
-checkDiskSpace' adjustment key = do
+ - in a destination (or the annex) printing a warning if not. -}
+checkDiskSpace :: Maybe FilePath -> Key -> Integer -> Annex Bool
+checkDiskSpace destination key alreadythere = do
 	reserve <- getDiskReserve
-	free <- inRepo $ getDiskFree . gitAnnexDir
+	free <- liftIO . getDiskFree =<< dir
+	force <- Annex.getState Annex.force
 	case (free, keySize key) of
-		(Just have, Just need) ->
-			when (need + reserve > have + adjustment) $
-				needmorespace (need + reserve - have - adjustment)
-		_ -> return ()
+		(Just have, Just need) -> do
+			let ok = need + reserve > have + alreadythere || force
+			unless ok $
+				needmorespace (need + reserve - have - alreadythere)
+			return ok
+		_ -> return True
 	where
-		needmorespace n = unlessM (Annex.getState Annex.force) $
-			error $ "not enough free space, need " ++ 
+		dir = maybe (fromRepo gitAnnexDir) return destination
+		needmorespace n =
+			warning $ "not enough free space, need " ++ 
 				roughSize storageUnits True n ++
 				" more" ++ forcemsg
 		forcemsg = " (use --force to override this check or adjust annex.diskreserve)"
diff --git a/Command/Unlock.hs b/Command/Unlock.hs
index afee101459..aeb2705160 100644
--- a/Command/Unlock.hs
+++ b/Command/Unlock.hs
@@ -34,8 +34,7 @@ start file (key, _) = do
 perform :: FilePath -> Key -> CommandPerform
 perform dest key = do
 	unlessM (inAnnex key) $ error "content not present"
-	
-	checkDiskSpace key
+	unlessM (checkDiskSpace Nothing key 0) $ error "cannot unlock"
 
 	src <- inRepo $ gitAnnexLocation key
 	tmpdest <- fromRepo $ gitAnnexTmpLocation key

From e807502666b2fd649f133766be7a605a6338d9b2 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 20 Apr 2012 16:14:29 -0400
Subject: [PATCH 038/220] had the wrong name for this

---
 Config.hs        | 2 +-
 debian/changelog | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/Config.hs b/Config.hs
index 10a66e47b1..087cb4043b 100644
--- a/Config.hs
+++ b/Config.hs
@@ -93,6 +93,6 @@ getTrustLevel r = fromRepo $ Git.Config.getMaybe $ remoteConfig r "trustlevel"
 {- Gets annex.diskreserve setting. -}
 getDiskReserve :: Annex Integer
 getDiskReserve = fromMaybe megabyte . readSize dataUnits
-	<$> getConfig "diskreserve" ""
+	<$> getConfig "annex.diskreserve" ""
 	where
 		megabyte = 1000000
diff --git a/debian/changelog b/debian/changelog
index e63e82f0e5..4520a05469 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+git-annex (3.20120419) UNRELEASED; urgency=low
+
+  * Fix use of annex.diskreserve config setting.
+
+ -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
+
 git-annex (3.20120418) unstable; urgency=low
 
   * bugfix: Adding a dotfile also caused all non-dotfiles to be added.

From b65e257b13773bd44ccf9cc734c42927e94ed929 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 20 Apr 2012 16:16:13 -0400
Subject: [PATCH 039/220] inverted logic

---
 Annex/Content.hs | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 494c9c10c1..932f1b7553 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -177,8 +177,9 @@ checkDiskSpace destination key alreadythere = do
 	force <- Annex.getState Annex.force
 	case (free, keySize key) of
 		(Just have, Just need) -> do
-			let ok = need + reserve > have + alreadythere || force
-			unless ok $
+			let ok = (need + reserve <= have + alreadythere) || force
+			unless ok $ do
+				liftIO $ print (need, reserve, have, alreadythere)
 				needmorespace (need + reserve - have - alreadythere)
 			return ok
 		_ -> return True

From 5cc76098ca7b702772ccf37a47f03da088148003 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 20 Apr 2012 16:24:44 -0400
Subject: [PATCH 040/220] Directory special remotes now check
 annex.diskreserve.

---
 Remote/Directory.hs | 52 ++++++++++++++++++++++++++-------------------
 debian/changelog    |  1 +
 2 files changed, 31 insertions(+), 22 deletions(-)

diff --git a/Remote/Directory.hs b/Remote/Directory.hs
index 3627d9a9ad..fd5a6f0b19 100644
--- a/Remote/Directory.hs
+++ b/Remote/Directory.hs
@@ -10,7 +10,7 @@ module Remote.Directory (remote) where
 import qualified Data.ByteString.Lazy.Char8 as L
 import qualified Data.ByteString.Char8 as S
 import qualified Data.Map as M
-import Control.Exception (bracket)
+import qualified Control.Exception as E
 
 import Common.Annex
 import Types.Remote
@@ -22,6 +22,7 @@ import Remote.Helper.Encryptable
 import Crypto
 import Utility.DataUnits
 import Data.Int
+import Annex.Content
 
 remote :: RemoteType
 remote = RemoteType {
@@ -125,7 +126,7 @@ store :: FilePath -> ChunkSize -> Key -> Annex Bool
 store d chunksize k = do
 	src <- inRepo $ gitAnnexLocation k
 	metered k $ \meterupdate -> 
-		liftIO $ catchBoolIO $ storeHelper d chunksize k $ \dests ->
+		storeHelper d chunksize k $ \dests ->
 			case chunksize of
 				Nothing -> do
 					let dest = Prelude.head dests
@@ -140,7 +141,7 @@ storeEncrypted :: FilePath -> ChunkSize -> (Cipher, Key) -> Key -> Annex Bool
 storeEncrypted d chunksize (cipher, enck) k = do
 	src <- inRepo $ gitAnnexLocation k
 	metered k $ \meterupdate ->
-		liftIO $ catchBoolIO $ storeHelper d chunksize enck $ \dests ->
+		storeHelper d chunksize enck $ \dests ->
 			withEncryptedContent cipher (L.readFile src) $ \s ->
 				case chunksize of
 					Nothing -> do
@@ -165,7 +166,7 @@ storeSplit' :: MeterUpdate -> Int64 -> [FilePath] -> [S.ByteString] -> [FilePath
 storeSplit' _ _ [] _ _ = error "ran out of dests"
 storeSplit' _ _  _ [] c = return $ reverse c
 storeSplit' meterupdate chunksize (d:dests) bs c = do
-	bs' <- bracket (openFile d WriteMode) hClose (feed chunksize bs)
+	bs' <- E.bracket (openFile d WriteMode) hClose (feed chunksize bs)
 	storeSplit' meterupdate chunksize dests bs' (d:c)
 	where
 		feed _ [] _ = return []
@@ -190,7 +191,7 @@ meteredWriteFile meterupdate dest b =
  - meter after each chunk. The feeder is called to get more chunks. -}
 meteredWriteFile' :: MeterUpdate -> FilePath -> s -> (s -> IO (s, [S.ByteString])) -> IO ()
 meteredWriteFile' meterupdate dest startstate feeder =
-	bracket (openFile dest WriteMode) hClose (feed startstate [])
+	E.bracket (openFile dest WriteMode) hClose (feed startstate [])
 	where
 		feed state [] h = do
 			(state', cs) <- feeder state
@@ -207,31 +208,38 @@ meteredWriteFile' meterupdate dest startstate feeder =
  - The stored files are only put into their final place once storage is
  - complete.
  -}
-storeHelper :: FilePath -> ChunkSize -> Key -> ([FilePath] -> IO [FilePath]) -> IO Bool
-storeHelper d chunksize key a = do
-	let dir = parentDir desttemplate
-	createDirectoryIfMissing True dir
-	allowWrite dir
-	stored <- a tmpdests
-	forM_ stored $ \f -> do
-		let dest = detmpprefix f
-		renameFile f dest
-		preventWrite dest
-	when (chunksize /= Nothing) $ do
-		let chunkcount = chunkCount desttemplate
-		_ <- tryIO $ allowWrite chunkcount
-		writeFile chunkcount (show $ length stored)
-		preventWrite chunkcount
-	preventWrite dir
-	return (not $ null stored)
+storeHelper :: FilePath -> ChunkSize -> Key -> ([FilePath] -> IO [FilePath]) -> Annex Bool
+storeHelper d chunksize key a = prep <&&> check <&&> go
 	where
 		desttemplate = Prelude.head $ locations d key
+		dir = parentDir desttemplate
 		tmpdests = case chunksize of
 			Nothing -> [desttemplate ++ tmpprefix]
 			Just _ -> map (++ tmpprefix) (chunkStream desttemplate)
 		tmpprefix = ".tmp"
 		detmpprefix f = take (length f - tmpprefixlen) f
 		tmpprefixlen = length tmpprefix
+		prep = liftIO $ catchBoolIO $ do
+			createDirectoryIfMissing True dir
+			allowWrite dir
+			return True
+		{- The size is not exactly known when encrypting the key;
+		 - this assumes that at least the size of the key is
+		 - needed as free space. -}
+		check = checkDiskSpace (Just dir) key 0
+		go = liftIO $ catchBoolIO $ do
+			stored <- a tmpdests
+			forM_ stored $ \f -> do
+				let dest = detmpprefix f
+				renameFile f dest
+				preventWrite dest
+			when (chunksize /= Nothing) $ do
+				let chunkcount = chunkCount desttemplate
+				_ <- tryIO $ allowWrite chunkcount
+				writeFile chunkcount (show $ length stored)
+				preventWrite chunkcount
+			preventWrite dir
+			return (not $ null stored)
 
 retrieve :: FilePath -> ChunkSize -> Key -> FilePath -> Annex Bool
 retrieve d chunksize k f = metered k $ \meterupdate ->
diff --git a/debian/changelog b/debian/changelog
index 4520a05469..eeb4bdfe47 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,6 +1,7 @@
 git-annex (3.20120419) UNRELEASED; urgency=low
 
   * Fix use of annex.diskreserve config setting.
+  * Directory special remotes now check annex.diskreserve.
 
  -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
 

From c8940c02e3d5577bbe3acdb79d0641226fa6d529 Mon Sep 17 00:00:00 2001
From: "http://christian.amsuess.com/chrysn" 
Date: Sat, 21 Apr 2012 15:07:47 +0000
Subject: [PATCH 041/220] getting "permission denied" in fsck, looking for
 cause

---
 ...ion_denied__34___in_fsck_on_shared_repo.mdwn | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo.mdwn

diff --git a/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo.mdwn b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo.mdwn
new file mode 100644
index 0000000000..f13aed2c2f
--- /dev/null
+++ b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo.mdwn
@@ -0,0 +1,17 @@
+i'm getting errors in ``git annex fsck`` on a shared bare git repo with git-annex 3.20120418 local repo version 3:
+
+``git-annex: ${PATH}/${MYREPO}.git/annex/objects/${HA}/${SH}/SHA1-${HASH}/SHA1-${HASH}: setFileMode: permission denied (Operation not permitted)``
+
+the repository is shared among several users in a common group, and the repo is set up with sticky group, and with appropriate umasks, everything should work.
+
+however, even with the file having permissions -rw-rw-r-- in the directory with permissions drwxrwsr-x, owned by someone else but by a group i'm currently in (as verified by issuing `groups`), i get said error message.
+
+a strace reveals that the failing syscall is:
+
+``[pid 17626] chmod("${FILENAME}", 0100555) = -1 EPERM (Operation not permitted)``
+
+(maybe related: git annex looks for the file in another ${HA}/${SH} combination (of three digits instead of two digits each) before, i take it this is just a new feature not used by the data in my repo? also, i should add that the repository dates back to git-annex 0.13.)
+
+as a workaround, i'm currently ``sudo chown``ing all files to me before the check.
+
+why does fsck try to set permissions even if they are ok? is this a bug in my setup, and if yes, how is a shared repository set up correctly?

From cbf9a9420e5d0fd9af2ad7d2864618386a75cfc3 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 11:57:17 -0400
Subject: [PATCH 042/220] avoid chown call if the current file mode is same as
 new

Not only an optimisation. fsck always tried to preventWrite to make sure
file modes are good, and in a shared repo, that will fail on directories
not owned by the current user. Although there may be other problems with
such a setup.
---
 Utility/FileMode.hs | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/Utility/FileMode.hs b/Utility/FileMode.hs
index 571b035039..4992690c6a 100644
--- a/Utility/FileMode.hs
+++ b/Utility/FileMode.hs
@@ -7,16 +7,24 @@
 
 module Utility.FileMode where
 
-import System.Posix.Files
+import Common
+
 import System.Posix.Types
 import Foreign (complement)
 
+modifyFileMode :: FilePath -> (FileMode -> FileMode) -> IO ()
+modifyFileMode f convert = do
+	s <- getFileStatus f
+	let cur = fileMode s
+	let new = convert cur
+	when (new /= cur) $
+		setFileMode f new
+
 {- Removes a FileMode from a file.
  - For example, call with otherWriteMode to chmod o-w -}
 unsetFileMode :: FilePath -> FileMode -> IO ()
-unsetFileMode f m = do
-	s <- getFileStatus f
-	setFileMode f $ fileMode s `intersectFileModes` complement m
+unsetFileMode f m = modifyFileMode f $
+	\cur -> cur `intersectFileModes` complement m
 
 {- Removes the write bits from a file. -}
 preventWrite :: FilePath -> IO ()
@@ -27,9 +35,8 @@ preventWrite f = unsetFileMode f writebits
 
 {- Turns a file's write bit back on. -}
 allowWrite :: FilePath -> IO ()
-allowWrite f = do
-	s <- getFileStatus f
-	setFileMode f $ fileMode s `unionFileModes` ownerWriteMode
+allowWrite f = modifyFileMode f $
+	\cur -> cur `unionFileModes` ownerWriteMode
 
 {- Checks if a file mode indicates it's a symlink. -}
 isSymLink :: FileMode -> Bool

From 9c87e3014050742faf0b32ad9a4190df062ce94c Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sat, 21 Apr 2012 16:09:19 +0000
Subject: [PATCH 043/220] Added a comment

---
 ...comment_1_3a5202ef2116ebb5559b6f4d920755fc._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_1_3a5202ef2116ebb5559b6f4d920755fc._comment

diff --git a/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_1_3a5202ef2116ebb5559b6f4d920755fc._comment b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_1_3a5202ef2116ebb5559b6f4d920755fc._comment
new file mode 100644
index 0000000000..5a5cafa721
--- /dev/null
+++ b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_1_3a5202ef2116ebb5559b6f4d920755fc._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 1"
+ date="2012-04-21T16:09:19Z"
+ content="""
+Well, the modes you show are wrong. Nothing in the annex should be writable. fsck needs to fix those. (It's true that it also always chmods even correct mode files/directories.. I've made a change avoiding that.)
+
+I have not thought or tried shared git annex repos with multiple unix users writing to them. ([[tips/Using_gitolite_with_git-annex]] would be an alternative.) Seems to me that removing content from the annex would also be a problem, since the directory will need to be chmodded to allow deleting the content from it, and that will fail if it's owned by someone else.  Perhaps git-annex needs to honor core.sharedRepository and avoid these nice safeguards on file modes then.
+"""]]

From b4a5e39ee62020380fc0dcf7aecaaf593d44dba5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 14:06:36 -0400
Subject: [PATCH 044/220] Support git's core.sharedRepository configuration

This is incomplete, it does not honor it yet for hash directories
and other annex bookkeeping files. Some of that is not needed for a bare
repo; some of it may be.
---
 Annex/Content.hs        | 104 +++++++++++++++++++++++++++-------------
 Command/Fsck.hs         |   7 ++-
 Command/Unannex.hs      |   6 +--
 Command/Unlock.hs       |   2 +-
 Git/SharedRepository.hs |  27 +++++++++++
 Utility/FileMode.hs     |  60 +++++++++++++++++++----
 debian/changelog        |   1 +
 7 files changed, 155 insertions(+), 52 deletions(-)
 create mode 100644 Git/SharedRepository.hs

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 932f1b7553..616e4128af 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -23,10 +23,11 @@ module Annex.Content (
 	saveState,
 	downloadUrl,
 	preseedTmp,
+	freezeContent,
+	thawContent,
+	freezeContentDir,
 ) where
 
-import Control.Exception (bracket_)
-import System.Posix.Types
 import System.IO.Unsafe (unsafeInterleaveIO)
 
 import Common.Annex
@@ -44,6 +45,7 @@ import Utility.DataUnits
 import Utility.CopyFile
 import Config
 import Annex.Exception
+import Git.SharedRepository
 
 {- Checks if a given key's content is currently present. -}
 inAnnex :: Key -> Annex Bool
@@ -57,8 +59,10 @@ inAnnex' a key = do
 {- A safer check; the key's content must not only be present, but
  - is not in the process of being removed. -}
 inAnnexSafe :: Key -> Annex (Maybe Bool)
-inAnnexSafe = inAnnex' $ \f -> openForLock f False >>= check
+inAnnexSafe = inAnnex' $ \f -> openforlock f >>= check
 	where
+		openforlock f = catchMaybeIO $
+			openFd f ReadOnly Nothing defaultFileFlags
 		check Nothing = return is_missing
 		check (Just h) = do
 			v <- getLock h (ReadLock, AbsoluteSeek, 0, 0)
@@ -75,30 +79,27 @@ inAnnexSafe = inAnnex' $ \f -> openForLock f False >>= check
 lockContent :: Key -> Annex a -> Annex a
 lockContent key a = do
 	file <- inRepo $ gitAnnexLocation key
-	bracketIO (openForLock file True >>= lock) unlock a
+	bracketIO (openforlock file >>= lock) unlock a
 	where
+		{- Since files are stored with the write bit disabled, have
+		 - to fiddle with permissions to open for an exclusive lock. -}
+		openforlock f = catchMaybeIO $ ifM (doesFileExist f)
+			( withModifiedFileMode f
+				(\cur -> cur `unionFileModes` ownerWriteMode)
+				open
+			, open
+			)
+			where
+				open = openFd f ReadWrite Nothing defaultFileFlags
 		lock Nothing = return Nothing
-		lock (Just l) = do
-			v <- tryIO $ setLock l (WriteLock, AbsoluteSeek, 0, 0)
+		lock (Just fd) = do
+			v <- tryIO $ setLock fd (WriteLock, AbsoluteSeek, 0, 0)
 			case v of
 				Left _ -> error "content is locked"
-				Right _ -> return $ Just l
+				Right _ -> return $ Just fd
 		unlock Nothing = return ()
 		unlock (Just l) = closeFd l
 
-openForLock :: FilePath -> Bool -> IO (Maybe Fd)
-openForLock file writelock = bracket_ prep cleanup go
-	where
-		go = catchMaybeIO $ openFd file mode Nothing defaultFileFlags
-		mode = if writelock then ReadWrite else ReadOnly
-		{- Since files are stored with the write bit disabled,
-		 - have to fiddle with permissions to open for an
-		 - exclusive lock. -}
-		forwritelock a = 
-			when writelock $ whenM (doesFileExist file) a
-		prep = forwritelock $ allowWrite file
-		cleanup = forwritelock $ preventWrite file
-
 {- Calculates the relative path to use to link a file to a key. -}
 calcGitLink :: FilePath -> Key -> Annex FilePath
 calcGitLink file key = do
@@ -132,7 +133,7 @@ getViaTmp key action = do
 		else return 0
 	ifM (checkDiskSpace Nothing key alreadythere)
 		( do
-			when e $ liftIO $ allowWrite tmp
+			when e $ thawContent tmp
 			getViaTmpUnchecked key action
 		, return False
 		)
@@ -216,14 +217,15 @@ moveAnnex :: Key -> FilePath -> Annex ()
 moveAnnex key src = do
 	dest <- inRepo $ gitAnnexLocation key
 	let dir = parentDir dest
-	liftIO $ ifM (doesFileExist dest)
-		( removeFile src
+	ifM (liftIO $ doesFileExist dest)
+		( liftIO $ removeFile src
 		, do
-			createDirectoryIfMissing True dir
-			allowWrite dir -- in case the directory already exists
-			moveFile src dest
-			preventWrite dest
-			preventWrite dir
+			liftIO $ do
+				createDirectoryIfMissing True dir
+				allowWrite dir -- in case the directory already exists
+				moveFile src dest
+			freezeContent dest
+			freezeContentDir dest
 		)
 
 withObjectLoc :: Key -> ((FilePath, FilePath) -> Annex a) -> Annex a
@@ -254,10 +256,9 @@ removeAnnex key = withObjectLoc key $ \(dir, file) -> do
 {- Moves a key's file out of .git/annex/objects/ -}
 fromAnnex :: Key -> FilePath -> Annex ()
 fromAnnex key dest = withObjectLoc key $ \(dir, file) -> do
-	liftIO $ do
-		allowWrite dir
-		allowWrite file
-		moveFile file dest
+	liftIO $ allowWrite dir
+	thawContent file
+	liftIO $ moveFile file dest
 	cleanObjectLoc key
 
 {- Moves a key out of .git/annex/objects/ into .git/annex/bad, and
@@ -321,7 +322,7 @@ preseedTmp key file = go =<< inAnnex key
 		go False = return False
 		go True = do
 			ok <- copy
-			when ok $ liftIO $ allowWrite file
+			when ok $ thawContent file
 			return ok
 		copy = ifM (liftIO $ doesFileExist file)
 				( return True
@@ -329,3 +330,40 @@ preseedTmp key file = go =<< inAnnex key
 					s <- inRepo $ gitAnnexLocation key
 					liftIO $ copyFileExternal s file
 				)
+
+{- Blocks writing to an annexed file. The file is made unwritable
+ - to avoid accidental edits. core.sharedRepository may change
+ - who can read it. -}
+freezeContent :: FilePath -> Annex ()
+freezeContent file = liftIO . go =<< fromRepo getSharedRepository
+	where
+		go GroupShared = do
+			preventWrite file
+			groupRead file
+		go AllShared = do
+			preventWrite file
+			allRead file
+		go _ = preventWrite file
+
+{- Allows writing to an annexed file that freezeContent was called on
+ - before. -}
+thawContent :: FilePath -> Annex ()
+thawContent file = liftIO . go =<< fromRepo getSharedRepository
+	where
+		go GroupShared = groupWriteRead file
+		go AllShared = groupWriteRead file
+		go _ = allowWrite file
+
+{- Blocks writing to the directory an annexed file is in, to prevent the
+ - file accidentially being deleted. However, if core.sharedRepository
+ - is set, this is not done, since the group must be allowed to delete the
+ - file.
+ -}
+freezeContentDir :: FilePath -> Annex ()
+freezeContentDir file = liftIO . go =<< fromRepo getSharedRepository
+	where
+		dir = parentDir file
+		go GroupShared = groupWriteRead dir
+		go AllShared = groupWriteRead dir
+		go _ = preventWrite dir
+
diff --git a/Command/Fsck.hs b/Command/Fsck.hs
index dac3bfac96..c60101fc79 100644
--- a/Command/Fsck.hs
+++ b/Command/Fsck.hs
@@ -166,10 +166,9 @@ verifyLocationLog key desc = do
 	-- Since we're checking that a key's file is present, throw
 	-- in a permission fixup here too.
 	when present $ do
-		f <- inRepo $ gitAnnexLocation key
-		liftIO $ do
-			preventWrite f
-			preventWrite (parentDir f)
+		file <- inRepo $ gitAnnexLocation key
+		freezeContent file
+		freezeContentDir file
 
 	u <- getUUID
 	verifyLocationLog' key desc present u (logChange key u)
diff --git a/Command/Unannex.hs b/Command/Unannex.hs
index 1e7313711c..bf931adfd4 100644
--- a/Command/Unannex.hs
+++ b/Command/Unannex.hs
@@ -10,7 +10,6 @@ module Command.Unannex where
 import Common.Annex
 import Command
 import qualified Annex
-import Utility.FileMode
 import Logs.Location
 import Annex.Content
 import qualified Git.Command
@@ -51,9 +50,8 @@ cleanup file key = do
 		( do
 			-- fast mode: hard link to content in annex
 			src <- inRepo $ gitAnnexLocation key
-			liftIO $ do
-				createLink src file
-				allowWrite file
+			liftIO $ createLink src file
+			thawContent file
 		, do
 			fromAnnex key file
 			logStatus key InfoMissing
diff --git a/Command/Unlock.hs b/Command/Unlock.hs
index aeb2705160..3ac50a0ebb 100644
--- a/Command/Unlock.hs
+++ b/Command/Unlock.hs
@@ -46,6 +46,6 @@ perform dest key = do
 			liftIO $ do
 				removeFile dest
 				moveFile tmpdest dest
-				allowWrite dest
+			thawContent dest
 			next $ return True
                 else error "copy failed!"
diff --git a/Git/SharedRepository.hs b/Git/SharedRepository.hs
new file mode 100644
index 0000000000..f3efa8fde9
--- /dev/null
+++ b/Git/SharedRepository.hs
@@ -0,0 +1,27 @@
+{- git core.sharedRepository handling
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Git.SharedRepository where
+
+import Data.Char
+
+import Common
+import Git
+import qualified Git.Config
+
+data SharedRepository = UnShared | GroupShared | AllShared | UmaskShared Int
+
+getSharedRepository :: Repo -> SharedRepository
+getSharedRepository r =
+	case map toLower $ Git.Config.get "core.sharedrepository" "" r of
+		"1" -> GroupShared
+		"group" -> GroupShared
+		"true" -> GroupShared
+		"all" -> AllShared
+		"world" -> AllShared
+		"everybody" -> AllShared
+		v -> maybe UnShared UmaskShared (readish v)
diff --git a/Utility/FileMode.hs b/Utility/FileMode.hs
index 4992690c6a..98c7124c2a 100644
--- a/Utility/FileMode.hs
+++ b/Utility/FileMode.hs
@@ -1,6 +1,6 @@
 {- File mode utilities.
  -
- - Copyright 2010 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -9,16 +9,36 @@ module Utility.FileMode where
 
 import Common
 
+import Control.Exception (bracket)
 import System.Posix.Types
 import Foreign (complement)
 
+combineModes :: [FileMode] -> FileMode
+combineModes [] = undefined
+combineModes [m] = m
+combineModes (m:ms) = foldl unionFileModes m ms
+
+{- Applies a conversion function to a file's mode. -}
 modifyFileMode :: FilePath -> (FileMode -> FileMode) -> IO ()
 modifyFileMode f convert = do
+	_ <- modifyFileMode' f convert
+	return ()
+modifyFileMode' :: FilePath -> (FileMode -> FileMode) -> IO FileMode
+modifyFileMode' f convert = do
 	s <- getFileStatus f
-	let cur = fileMode s
-	let new = convert cur
-	when (new /= cur) $
+	let old = fileMode s
+	let new = convert old
+	when (new /= old) $
 		setFileMode f new
+	return old
+
+{- Runs an action after changing a file's mode, then restores the old mode. -}
+withModifiedFileMode :: FilePath -> (FileMode -> FileMode) -> IO a -> IO a
+withModifiedFileMode file convert a = bracket setup cleanup go
+	where
+		setup = modifyFileMode' file convert
+		cleanup oldmode = modifyFileMode file (const oldmode)
+		go _ = a
 
 {- Removes a FileMode from a file.
  - For example, call with otherWriteMode to chmod o-w -}
@@ -28,23 +48,43 @@ unsetFileMode f m = modifyFileMode f $
 
 {- Removes the write bits from a file. -}
 preventWrite :: FilePath -> IO ()
-preventWrite f = unsetFileMode f writebits
+preventWrite f = unsetFileMode f $ combineModes writebits
 	where
-		writebits = foldl unionFileModes ownerWriteMode
-					[groupWriteMode, otherWriteMode]
+		writebits = [ownerWriteMode, groupWriteMode, otherWriteMode]
 
 {- Turns a file's write bit back on. -}
 allowWrite :: FilePath -> IO ()
 allowWrite f = modifyFileMode f $
 	\cur -> cur `unionFileModes` ownerWriteMode
 
+{- Allows owner and group to read and write to a file. -}
+groupWriteRead :: FilePath -> IO ()
+groupWriteRead f = modifyFileMode f $ \cur -> combineModes
+	[ cur
+	, ownerWriteMode, groupWriteMode
+	, ownerReadMode, groupReadMode
+	]
+
+{- Allows group to read a file. -}
+groupRead :: FilePath -> IO ()
+groupRead f = modifyFileMode f $ \cur -> combineModes
+	[ cur
+	, ownerReadMode, groupReadMode
+	]
+
+{- Allows all to read a file. -}
+allRead :: FilePath -> IO ()
+allRead f = modifyFileMode f $ \cur -> combineModes
+	[ cur
+	, ownerReadMode, groupReadMode, otherReadMode
+	]
+
 {- Checks if a file mode indicates it's a symlink. -}
 isSymLink :: FileMode -> Bool
 isSymLink mode = symbolicLinkMode `intersectFileModes` mode == symbolicLinkMode
 
 {- Checks if a file has any executable bits set. -}
 isExecutable :: FileMode -> Bool
-isExecutable mode = ebits `intersectFileModes` mode /= 0
+isExecutable mode = combineModes ebits `intersectFileModes` mode /= 0
 	where
-		ebits = ownerExecuteMode `unionFileModes`
-			groupExecuteMode `unionFileModes` otherExecuteMode
+		ebits = [ownerExecuteMode, groupExecuteMode, otherExecuteMode]
diff --git a/debian/changelog b/debian/changelog
index eeb4bdfe47..3c56fda51a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -2,6 +2,7 @@ git-annex (3.20120419) UNRELEASED; urgency=low
 
   * Fix use of annex.diskreserve config setting.
   * Directory special remotes now check annex.diskreserve.
+  * Support git's core.sharedRepository configuration.
 
  -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
 

From 7e45712d194aa2b231083c3ccee3668f053e5717 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 16:01:56 -0400
Subject: [PATCH 045/220] better file mode setting code

---
 Annex/Content.hs    | 12 +++++-----
 Command/Unlock.hs   |  1 -
 Utility/FileMode.hs | 57 +++++++++++++++++++--------------------------
 3 files changed, 30 insertions(+), 40 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 616e4128af..b216b861d9 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -337,12 +337,12 @@ preseedTmp key file = go =<< inAnnex key
 freezeContent :: FilePath -> Annex ()
 freezeContent file = liftIO . go =<< fromRepo getSharedRepository
 	where
-		go GroupShared = do
-			preventWrite file
-			groupRead file
-		go AllShared = do
-			preventWrite file
-			allRead file
+		go GroupShared = modifyFileMode file $
+			removeModes writeModes .
+			addModes [ownerReadMode, groupReadMode]
+		go AllShared = modifyFileMode file $
+			removeModes writeModes .
+			addModes readModes
 		go _ = preventWrite file
 
 {- Allows writing to an annexed file that freezeContent was called on
diff --git a/Command/Unlock.hs b/Command/Unlock.hs
index 3ac50a0ebb..f3ffd31ba6 100644
--- a/Command/Unlock.hs
+++ b/Command/Unlock.hs
@@ -11,7 +11,6 @@ import Common.Annex
 import Command
 import Annex.Content
 import Utility.CopyFile
-import Utility.FileMode
 
 def :: [Command]
 def =
diff --git a/Utility/FileMode.hs b/Utility/FileMode.hs
index 98c7124c2a..f3db709231 100644
--- a/Utility/FileMode.hs
+++ b/Utility/FileMode.hs
@@ -13,11 +13,6 @@ import Control.Exception (bracket)
 import System.Posix.Types
 import Foreign (complement)
 
-combineModes :: [FileMode] -> FileMode
-combineModes [] = undefined
-combineModes [m] = m
-combineModes (m:ms) = foldl unionFileModes m ms
-
 {- Applies a conversion function to a file's mode. -}
 modifyFileMode :: FilePath -> (FileMode -> FileMode) -> IO ()
 modifyFileMode f convert = do
@@ -32,6 +27,15 @@ modifyFileMode' f convert = do
 		setFileMode f new
 	return old
 
+{- Adds the specified FileModes to the input mode, leaving the rest
+ - unchanged. -}
+addModes :: [FileMode] -> FileMode -> FileMode
+addModes ms m = combineModes (m:ms)
+
+{- Removes the specified FileModes from the input mode. -}
+removeModes :: [FileMode] -> FileMode -> FileMode
+removeModes ms m = m `intersectFileModes` complement (combineModes ms)
+
 {- Runs an action after changing a file's mode, then restores the old mode. -}
 withModifiedFileMode :: FilePath -> (FileMode -> FileMode) -> IO a -> IO a
 withModifiedFileMode file convert a = bracket setup cleanup go
@@ -40,45 +44,27 @@ withModifiedFileMode file convert a = bracket setup cleanup go
 		cleanup oldmode = modifyFileMode file (const oldmode)
 		go _ = a
 
-{- Removes a FileMode from a file.
- - For example, call with otherWriteMode to chmod o-w -}
-unsetFileMode :: FilePath -> FileMode -> IO ()
-unsetFileMode f m = modifyFileMode f $
-	\cur -> cur `intersectFileModes` complement m
+writeModes :: [FileMode]
+writeModes = [ownerWriteMode, groupWriteMode, otherWriteMode]
+
+readModes :: [FileMode]
+readModes = [ownerReadMode, groupReadMode, otherReadMode]
 
 {- Removes the write bits from a file. -}
 preventWrite :: FilePath -> IO ()
-preventWrite f = unsetFileMode f $ combineModes writebits
-	where
-		writebits = [ownerWriteMode, groupWriteMode, otherWriteMode]
+preventWrite f = modifyFileMode f $ removeModes writeModes
 
-{- Turns a file's write bit back on. -}
+{- Turns a file's owner write bit back on. -}
 allowWrite :: FilePath -> IO ()
-allowWrite f = modifyFileMode f $
-	\cur -> cur `unionFileModes` ownerWriteMode
+allowWrite f = modifyFileMode f $ addModes [ownerWriteMode]
 
 {- Allows owner and group to read and write to a file. -}
 groupWriteRead :: FilePath -> IO ()
-groupWriteRead f = modifyFileMode f $ \cur -> combineModes
-	[ cur
-	, ownerWriteMode, groupWriteMode
+groupWriteRead f = modifyFileMode f $ addModes
+	[ ownerWriteMode, groupWriteMode
 	, ownerReadMode, groupReadMode
 	]
 
-{- Allows group to read a file. -}
-groupRead :: FilePath -> IO ()
-groupRead f = modifyFileMode f $ \cur -> combineModes
-	[ cur
-	, ownerReadMode, groupReadMode
-	]
-
-{- Allows all to read a file. -}
-allRead :: FilePath -> IO ()
-allRead f = modifyFileMode f $ \cur -> combineModes
-	[ cur
-	, ownerReadMode, groupReadMode, otherReadMode
-	]
-
 {- Checks if a file mode indicates it's a symlink. -}
 isSymLink :: FileMode -> Bool
 isSymLink mode = symbolicLinkMode `intersectFileModes` mode == symbolicLinkMode
@@ -88,3 +74,8 @@ isExecutable :: FileMode -> Bool
 isExecutable mode = combineModes ebits `intersectFileModes` mode /= 0
 	where
 		ebits = [ownerExecuteMode, groupExecuteMode, otherExecuteMode]
+
+combineModes :: [FileMode] -> FileMode
+combineModes [] = undefined
+combineModes [m] = m
+combineModes (m:ms) = foldl unionFileModes m ms

From b98b69e8c6d9b873a864b79cff857882f67ee576 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 16:59:49 -0400
Subject: [PATCH 046/220] honor core.sharedRepository when making all the other
 files in the annex

Lock files, directories, etc.
---
 Annex/Branch.hs        |  2 ++
 Annex/Content.hs       | 22 ++++++++++-----
 Annex/Journal.hs       | 41 +++++++++++++---------------
 Annex/LockPool.hs      |  5 +++-
 Annex/Perms.hs         | 61 ++++++++++++++++++++++++++++++++++++++++++
 Annex/Ssh.hs           |  5 +++-
 Remote/Helper/Hooks.hs |  5 +++-
 Utility/FileMode.hs    | 11 ++++++++
 8 files changed, 119 insertions(+), 33 deletions(-)
 create mode 100644 Annex/Perms.hs

diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index 52089ac97d..e5976c2c01 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -36,6 +36,7 @@ import qualified Git.UnionMerge
 import Git.HashObject
 import qualified Git.Index
 import Annex.CatFile
+import Annex.Perms
 
 {- Name of the branch that is used to store git-annex's information. -}
 name :: Git.Ref
@@ -308,6 +309,7 @@ setIndexSha :: Git.Ref -> Annex ()
 setIndexSha ref = do
         lock <- fromRepo gitAnnexIndexLock
 	liftIO $ writeFile lock $ show ref ++ "\n"
+	setAnnexPerm lock
 
 {- Checks if there are uncommitted changes in the branch's index or journal. -}
 unCommitted :: Annex Bool
diff --git a/Annex/Content.hs b/Annex/Content.hs
index b216b861d9..7022364d09 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -46,6 +46,7 @@ import Utility.CopyFile
 import Config
 import Annex.Exception
 import Git.SharedRepository
+import Annex.Perms
 
 {- Checks if a given key's content is currently present. -}
 inAnnex :: Key -> Annex Bool
@@ -141,7 +142,7 @@ getViaTmp key action = do
 prepTmp :: Key -> Annex FilePath
 prepTmp key = do
 	tmp <- fromRepo $ gitAnnexTmpLocation key
-	liftIO $ createDirectoryIfMissing True (parentDir tmp)
+	createAnnexDirectory (parentDir tmp)
 	return tmp
 
 {- Like getViaTmp, but does not check that there is enough disk space
@@ -216,14 +217,11 @@ checkDiskSpace destination key alreadythere = do
 moveAnnex :: Key -> FilePath -> Annex ()
 moveAnnex key src = do
 	dest <- inRepo $ gitAnnexLocation key
-	let dir = parentDir dest
 	ifM (liftIO $ doesFileExist dest)
 		( liftIO $ removeFile src
 		, do
-			liftIO $ do
-				createDirectoryIfMissing True dir
-				allowWrite dir -- in case the directory already exists
-				moveFile src dest
+			createContentDir dest
+			liftIO $ moveFile src dest
 			freezeContent dest
 			freezeContentDir dest
 		)
@@ -268,8 +266,8 @@ moveBad key = do
 	src <- inRepo $ gitAnnexLocation key
 	bad <- fromRepo gitAnnexBadDir
 	let dest = bad  takeFileName src
+	createAnnexDirectory (parentDir dest)
 	liftIO $ do
-		createDirectoryIfMissing True (parentDir dest)
 		allowWrite (parentDir src)
 		moveFile src dest
 	cleanObjectLoc key
@@ -367,3 +365,13 @@ freezeContentDir file = liftIO . go =<< fromRepo getSharedRepository
 		go AllShared = groupWriteRead dir
 		go _ = preventWrite dir
 
+{- Makes the directory tree to store an annexed file's content,
+ - with appropriate permissions on each level. -}
+createContentDir :: FilePath -> Annex ()
+createContentDir dest = do
+	unlessM (liftIO $ doesDirectoryExist dir) $
+		createAnnexDirectory dir 
+	-- might have already existed with restricted perms
+	liftIO $ allowWrite dir
+	where
+		dir = parentDir dest
diff --git a/Annex/Journal.hs b/Annex/Journal.hs
index 34c4d98c88..ff103180ee 100644
--- a/Annex/Journal.hs
+++ b/Annex/Journal.hs
@@ -16,6 +16,7 @@ import System.IO.Binary
 import Common.Annex
 import Annex.Exception
 import qualified Git
+import Annex.Perms
 
 {- Records content for a file in the branch to the journal.
  -
@@ -23,22 +24,20 @@ import qualified Git
  - avoids git needing to rewrite the index after every change. -}
 setJournalFile :: FilePath -> String -> Annex ()
 setJournalFile file content = do
-	g <- gitRepo
-	liftIO $ doRedo (write g) $ do
-		createDirectoryIfMissing True $ gitAnnexJournalDir g
-		createDirectoryIfMissing True $ gitAnnexTmpDir g
-	where
-		-- journal file is written atomically
-		write g = do
-			let jfile = journalFile g file
-			let tmpfile = gitAnnexTmpDir g  takeFileName jfile
-			writeBinaryFile tmpfile content
-			moveFile tmpfile jfile
+	createAnnexDirectory =<< fromRepo gitAnnexJournalDir
+	createAnnexDirectory =<< fromRepo gitAnnexTmpDir
+	-- journal file is written atomically
+	jfile <- fromRepo $ journalFile file
+	tmp <- fromRepo gitAnnexTmpDir
+	let tmpfile = tmp  takeFileName jfile
+	liftIO $ do
+		writeBinaryFile tmpfile content
+		moveFile tmpfile jfile
 
 {- Gets any journalled content for a file in the branch. -}
 getJournalFile :: FilePath -> Annex (Maybe String)
 getJournalFile file = inRepo $ \g -> catchMaybeIO $
-	readFileStrict $ journalFile g file
+	readFileStrict $ journalFile file g
 
 {- List of files that have updated content in the journal. -}
 getJournalledFiles :: Annex [FilePath]
@@ -62,8 +61,8 @@ journalDirty = not . null <$> getJournalFiles
  - used in the branch is not necessary, and all the files are put directly
  - in the journal directory.
  -}
-journalFile :: Git.Repo -> FilePath -> FilePath
-journalFile repo file = gitAnnexJournalDir repo  concatMap mangle file
+journalFile :: FilePath -> Git.Repo -> FilePath
+journalFile file repo = gitAnnexJournalDir repo  concatMap mangle file
 	where
 		mangle '/' = "_"
 		mangle '_' = "__"
@@ -79,16 +78,12 @@ fileJournal = replace "//" "_" . replace "_" "/"
 lockJournal :: Annex a -> Annex a
 lockJournal a = do
 	file <- fromRepo gitAnnexJournalLock
-	bracketIO (lock file) unlock a
+	createAnnexDirectory $ takeDirectory file
+	mode <- annexFileMode
+	bracketIO (lock file mode) unlock a
 	where
-		lock file = do
-			l <- doRedo (createFile file stdFileMode) $
-				createDirectoryIfMissing True $ takeDirectory file
+		lock file mode = do
+			l <- noUmask mode $ createFile file mode
 			waitToSetLock l (WriteLock, AbsoluteSeek, 0, 0)
 			return l
 		unlock = closeFd
-
-{- Runs an action, catching failure and running something to fix it up, and
- - retrying if necessary. -}
-doRedo :: IO a -> IO b -> IO a
-doRedo a b = catchIO a $ const $ b >> a
diff --git a/Annex/LockPool.hs b/Annex/LockPool.hs
index 3fede5739b..3eb1363eed 100644
--- a/Annex/LockPool.hs
+++ b/Annex/LockPool.hs
@@ -12,6 +12,7 @@ import System.Posix.Types (Fd)
 
 import Common.Annex
 import Annex
+import Annex.Perms
 
 {- Create a specified lock file, and takes a shared lock. -}
 lockFile :: FilePath -> Annex ()
@@ -19,7 +20,9 @@ lockFile file = go =<< fromPool file
 	where
 		go (Just _) = return () -- already locked
 		go Nothing = do
-			fd <- liftIO $ openFd file ReadOnly (Just stdFileMode) defaultFileFlags
+			mode <- annexFileMode
+			fd <- liftIO $ noUmask mode $
+				openFd file ReadOnly (Just mode) defaultFileFlags
 			liftIO $ waitToSetLock fd (ReadLock, AbsoluteSeek, 0, 0)
 			changePool $ M.insert file fd
 
diff --git a/Annex/Perms.hs b/Annex/Perms.hs
new file mode 100644
index 0000000000..2b54077ca6
--- /dev/null
+++ b/Annex/Perms.hs
@@ -0,0 +1,61 @@
+{- git-annex file permissions
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Annex.Perms (
+	setAnnexPerm,
+	annexFileMode,
+	createAnnexDirectory,
+	noUmask,
+) where
+
+import Common.Annex
+import Utility.FileMode
+import Git.SharedRepository
+
+import System.Posix.Types
+
+{- Sets appropriate file mode for a file or directory in the annex,
+ - other than the content files and content directory. Normally,
+ - use the default mode, but with core.sharedRepository set,
+ - allow the group to write, etc. -}
+setAnnexPerm :: FilePath -> Annex ()
+setAnnexPerm file = liftIO . go =<< fromRepo getSharedRepository
+	where
+		go GroupShared = groupWriteRead file
+		go AllShared = modifyFileMode file $ addModes $
+			[ ownerWriteMode, groupWriteMode ] ++ readModes
+		go _ = return ()
+
+{- Gets the appropriate mode to use for creating a file in the annex
+ - (other than content files, which are locked down more). -}
+annexFileMode :: Annex FileMode
+annexFileMode = go <$> fromRepo getSharedRepository
+	where
+		go GroupShared = sharedmode
+		go AllShared = combineModes (sharedmode:readModes)
+		go _ = stdFileMode
+		sharedmode = combineModes
+			[ ownerWriteMode, groupWriteMode
+			, ownerReadMode, groupReadMode
+			]
+
+{- Creates a directory inside the gitAnnexDir, including any parent
+ - directories. Makes directories with appropriate permissions. -}
+createAnnexDirectory :: FilePath -> Annex ()
+createAnnexDirectory dir = traverse dir [] =<< top
+	where
+		top = parentDir <$> fromRepo gitAnnexDir
+		traverse d below stop
+			| d `equalFilePath` stop = done
+			| otherwise = ifM (liftIO $ doesDirectoryExist d)
+				( done
+				, traverse (parentDir d) (d:below) stop
+				)
+			where
+				done = forM_ below $ \p -> do
+					liftIO $ createDirectory p
+					setAnnexPerm p
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index e6cd6a9263..c9e6e29515 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -17,6 +17,7 @@ import Annex.LockPool
 import qualified Git
 import Config
 import qualified Build.SysConfig as SysConfig
+import Annex.Perms
 
 {- Generates parameters to ssh to a given host (or user@host) on a given
  - port, with connection caching. -}
@@ -74,7 +75,9 @@ sshCleanup = do
 			-- be stopped.
 			let lockfile = socket2lock socketfile
 			unlockFile lockfile
-			fd <- liftIO $ openFd lockfile ReadWrite (Just stdFileMode) defaultFileFlags
+			mode <- annexFileMode
+			fd <- liftIO $ noUmask mode $
+				openFd lockfile ReadWrite (Just mode) defaultFileFlags
 			v <- liftIO $ tryIO $
 				setLock fd (WriteLock, AbsoluteSeek, 0, 0)
 			case v of
diff --git a/Remote/Helper/Hooks.hs b/Remote/Helper/Hooks.hs
index 2864a8ed58..de731bd6e6 100644
--- a/Remote/Helper/Hooks.hs
+++ b/Remote/Helper/Hooks.hs
@@ -14,6 +14,7 @@ import Types.Remote
 import qualified Annex
 import Annex.LockPool
 import Config
+import Annex.Perms
 
 {- Modifies a remote's access functions to first run the
  - annex-start-command hook, and trigger annex-stop-command on shutdown.
@@ -75,7 +76,9 @@ runHooks r starthook stophook a = do
 			-- succeeds, we're the only process using this remote,
 			-- so can stop it.
 			unlockFile lck
-			fd <- liftIO $ openFd lck ReadWrite (Just stdFileMode) defaultFileFlags
+			mode <- annexFileMode
+			fd <- liftIO $ noUmask mode $
+				openFd lck ReadWrite (Just mode) defaultFileFlags
 			v <- liftIO $ tryIO $
 				setLock fd (WriteLock, AbsoluteSeek, 0, 0)
 			case v of
diff --git a/Utility/FileMode.hs b/Utility/FileMode.hs
index f3db709231..c0f2ad589c 100644
--- a/Utility/FileMode.hs
+++ b/Utility/FileMode.hs
@@ -75,6 +75,17 @@ isExecutable mode = combineModes ebits `intersectFileModes` mode /= 0
 	where
 		ebits = [ownerExecuteMode, groupExecuteMode, otherExecuteMode]
 
+{- Runs an action without that pesky umask influencing it, unless the
+ - passed FileMode is the standard one. -}
+noUmask :: FileMode -> IO a -> IO a
+noUmask mode a
+	| mode == stdFileMode = a
+	| otherwise = bracket setup cleanup go
+	where
+		setup = setFileCreationMask nullFileMode
+		cleanup = setFileCreationMask
+		go _ = a
+
 combineModes :: [FileMode] -> FileMode
 combineModes [] = undefined
 combineModes [m] = m

From cab63b89f2470d0874e72a4a9e088206fb554c94 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 19:42:49 -0400
Subject: [PATCH 047/220] cache parsed core.sharedrepository

---
 Annex.hs       |  3 +++
 Annex/Perms.hs | 13 +++++++++++--
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/Annex.hs b/Annex.hs
index ef95ff174c..2b58c2bd49 100644
--- a/Annex.hs
+++ b/Annex.hs
@@ -37,6 +37,7 @@ import qualified Git
 import qualified Git.Config
 import Git.CatFile
 import Git.CheckAttr
+import Git.SharedRepository
 import qualified Git.Queue
 import Types.Backend
 import qualified Types.Remote
@@ -88,6 +89,7 @@ data AnnexState = AnnexState
 	, forcebackend :: Maybe String
 	, forcenumcopies :: Maybe Int
 	, limit :: Matcher (FilePath -> Annex Bool)
+	, shared :: Maybe SharedRepository
 	, forcetrust :: TrustMap
 	, trustmap :: Maybe TrustMap
 	, ciphers :: M.Map EncryptedCipher Cipher
@@ -113,6 +115,7 @@ newState gitrepo = AnnexState
 	, forcebackend = Nothing
 	, forcenumcopies = Nothing
 	, limit = Left []
+	, shared = Nothing
 	, forcetrust = M.empty
 	, trustmap = Nothing
 	, ciphers = M.empty
diff --git a/Annex/Perms.hs b/Annex/Perms.hs
index 2b54077ca6..12dfdd6679 100644
--- a/Annex/Perms.hs
+++ b/Annex/Perms.hs
@@ -15,15 +15,24 @@ module Annex.Perms (
 import Common.Annex
 import Utility.FileMode
 import Git.SharedRepository
+import qualified Annex
 
 import System.Posix.Types
 
+withShared :: (SharedRepository -> Annex a) -> Annex a
+withShared a = maybe startup a =<< Annex.getState Annex.shared
+	where
+		startup = do
+			shared <- fromRepo getSharedRepository
+			Annex.changeState $ \s -> s { Annex.shared = Just shared }
+			a shared
+
 {- Sets appropriate file mode for a file or directory in the annex,
  - other than the content files and content directory. Normally,
  - use the default mode, but with core.sharedRepository set,
  - allow the group to write, etc. -}
 setAnnexPerm :: FilePath -> Annex ()
-setAnnexPerm file = liftIO . go =<< fromRepo getSharedRepository
+setAnnexPerm file = withShared $ liftIO . go
 	where
 		go GroupShared = groupWriteRead file
 		go AllShared = modifyFileMode file $ addModes $
@@ -33,7 +42,7 @@ setAnnexPerm file = liftIO . go =<< fromRepo getSharedRepository
 {- Gets the appropriate mode to use for creating a file in the annex
  - (other than content files, which are locked down more). -}
 annexFileMode :: Annex FileMode
-annexFileMode = go <$> fromRepo getSharedRepository
+annexFileMode = withShared $ return . go
 	where
 		go GroupShared = sharedmode
 		go AllShared = combineModes (sharedmode:readModes)

From be36aaca5bccecdff7e48b020ee62b0db3bc38f1 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sat, 21 Apr 2012 23:46:42 +0000
Subject: [PATCH 048/220] Added a comment

---
 .../comment_2_86663eeb75b0477f53c45f26c8e4b051._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_2_86663eeb75b0477f53c45f26c8e4b051._comment

diff --git a/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_2_86663eeb75b0477f53c45f26c8e4b051._comment b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_2_86663eeb75b0477f53c45f26c8e4b051._comment
new file mode 100644
index 0000000000..1c9bfbfe40
--- /dev/null
+++ b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_2_86663eeb75b0477f53c45f26c8e4b051._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 2"
+ date="2012-04-21T23:46:42Z"
+ content="""
+All right, I've made all the changes so it supports `core.sharedRepository`.
+"""]]

From bee420bd2d0cbe16489b061b208083e2b8ba9d0e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 23:04:59 -0400
Subject: [PATCH 049/220] in which I discover void

void :: Functor f => f a -> f () -- ah, of course that's useful :)
---
 Annex/Branch.hs        |  9 +++------
 Annex/Ssh.hs           | 11 ++++-------
 Remote/Git.hs          |  6 ++----
 Remote/Helper/Hooks.hs |  5 ++---
 Utility/FileMode.hs    |  4 +---
 Utility/Inotify.hs     | 10 ++++------
 6 files changed, 16 insertions(+), 29 deletions(-)

diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index e5976c2c01..ce1dd58cec 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -65,9 +65,7 @@ siblingBranches = inRepo $ Git.Ref.matchingUniq name
 
 {- Creates the branch, if it does not already exist. -}
 create :: Annex ()
-create = do
-	_ <- getBranch
-	return ()
+create = void $ getBranch
 
 {- Returns the ref of the branch, creating it first if necessary. -}
 getBranch :: Annex Git.Ref
@@ -325,10 +323,9 @@ setUnCommitted = do
 	liftIO $ writeFile file "1"
 
 setCommitted :: Annex ()
-setCommitted = do
+setCommitted = void $ do
 	file <- fromRepo gitAnnexIndexDirty
-	_ <- liftIO $ tryIO $ removeFile file
-	return ()
+	liftIO $ tryIO $ removeFile file
 
 {- Stages the journal into the index. -}
 stageJournal :: Annex ()
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index c9e6e29515..02a1ee705a 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -87,20 +87,17 @@ sshCleanup = do
 		stopssh socketfile = do
 			let (host, port) = socket2hostport socketfile
 			(_, params) <- sshInfo (host, port)
-			_ <- liftIO $ do
+			void $ liftIO $ do
 				-- "ssh -O stop" is noisy on stderr even with -q
 				let cmd = unwords $ toCommand $
 					[ Params "-O stop"
 					] ++ params ++ [Param host]
-				_ <- boolSystem "sh"
+				boolSystem "sh"
 					[ Param "-c"
 					, Param $ "ssh " ++ cmd ++ " >/dev/null 2>/dev/null"
 					]
-				--try $ removeFile socketfile
-				return ()
-			-- Cannot remove the lock file; other processes may
-			-- be waiting on our exclusive lock to use it.
-			return ()
+				-- Cannot remove the lock file; other processes may
+				-- be waiting on our exclusive lock to use it.
 
 hostport2socket :: String -> Maybe Integer -> FilePath
 hostport2socket host Nothing = host
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 541b050994..d71872b277 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -313,7 +313,7 @@ commitOnCleanup r a = go `after` a
 		cleanup
 			| not $ Git.repoIsUrl r = liftIO $ onLocal r $
 				Annex.Branch.commit "update"
-			| otherwise = do
+			| otherwise = void $ do
 				Just (shellcmd, shellparams) <-
 					git_annex_shell r "commit" []
 				-- Throw away stderr, since the remote may not
@@ -322,6 +322,4 @@ commitOnCleanup r a = go `after` a
 				let cmd = shellcmd ++ " "
 					++ unwords (map shellEscape $ toCommand shellparams)
 					++ ">/dev/null 2>/dev/null"
-				_ <- liftIO $
-					boolSystem "sh" [Param "-c", Param cmd]
-				return ()
+				liftIO $ boolSystem "sh" [Param "-c", Param cmd]
diff --git a/Remote/Helper/Hooks.hs b/Remote/Helper/Hooks.hs
index de731bd6e6..40484b2a7c 100644
--- a/Remote/Helper/Hooks.hs
+++ b/Remote/Helper/Hooks.hs
@@ -47,9 +47,8 @@ runHooks r starthook stophook a = do
 	where
 		remoteid = show (uuid r)
 		run Nothing = return ()
-		run (Just command) = liftIO $ do
-			_ <- boolSystem "sh" [Param "-c", Param command]
-			return ()
+		run (Just command) = void $ liftIO $
+			boolSystem "sh" [Param "-c", Param command]
 		firstrun lck = do
 			-- Take a shared lock; This indicates that git-annex
 			-- is using the remote, and prevents other instances
diff --git a/Utility/FileMode.hs b/Utility/FileMode.hs
index c0f2ad589c..353de7b92a 100644
--- a/Utility/FileMode.hs
+++ b/Utility/FileMode.hs
@@ -15,9 +15,7 @@ import Foreign (complement)
 
 {- Applies a conversion function to a file's mode. -}
 modifyFileMode :: FilePath -> (FileMode -> FileMode) -> IO ()
-modifyFileMode f convert = do
-	_ <- modifyFileMode' f convert
-	return ()
+modifyFileMode f convert = void $ modifyFileMode' f convert
 modifyFileMode' :: FilePath -> (FileMode -> FileMode) -> IO FileMode
 modifyFileMode' f convert = do
 	s <- getFileStatus f
diff --git a/Utility/Inotify.hs b/Utility/Inotify.hs
index 049737c08a..0a261ecfe0 100644
--- a/Utility/Inotify.hs
+++ b/Utility/Inotify.hs
@@ -53,10 +53,9 @@ watchDir i test add del dir = watchDir' False i test add del dir
 watchDir' :: Bool -> INotify -> (FilePath -> Bool) -> Maybe (FilePath -> IO ()) -> Maybe (FilePath -> IO ()) -> FilePath -> IO ()
 watchDir' scan i test add del dir = do
 	if test dir
-		then do
+		then void $ do
 			_ <- addWatch i watchevents dir go
-			_ <- mapM walk =<< dirContents dir
-			return ()
+			mapM walk =<< dirContents dir
 		else return ()
 	where
 		watchevents
@@ -92,6 +91,5 @@ waitForTermination = do
 		check keyboardSignal mv
 	takeMVar mv
 	where
-		check sig mv = do
-			_ <- installHandler sig (CatchOnce $ putMVar mv ()) Nothing
-			return ()
+		check sig mv = void $
+			installHandler sig (CatchOnce $ putMVar mv ()) Nothing

From ed79596b758935a3f22bf6803bc082a6bbe10f58 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 21 Apr 2012 23:32:33 -0400
Subject: [PATCH 050/220] noop

---
 Annex/Content.hs       |  6 +++---
 Annex/LockPool.hs      |  7 +++----
 Annex/Perms.hs         |  2 +-
 Annex/Ssh.hs           |  2 +-
 Annex/Version.hs       |  2 +-
 CmdLine.hs             |  2 +-
 Command/Fsck.hs        |  2 +-
 Command/Status.hs      |  5 ++---
 Command/Unused.hs      |  2 +-
 Command/Whereis.hs     | 12 ++++++------
 Git/Command.hs         |  4 ++--
 Git/Construct.hs       |  2 +-
 Git/UnionMerge.hs      |  2 +-
 GitAnnexShell.hs       |  4 ++--
 Logs/Location.hs       |  2 +-
 Logs/UUID.hs           |  2 +-
 Messages.hs            |  6 +++---
 Remote.hs              |  2 +-
 Remote/Directory.hs    |  3 ++-
 Remote/Helper/Hooks.hs |  4 ++--
 Remote/S3.hs           |  6 +++---
 Upgrade/V1.hs          |  2 +-
 Utility/Directory.hs   |  3 ++-
 Utility/Inotify.hs     | 10 +++++-----
 Utility/Monad.hs       |  4 ++++
 Utility/Touch.hsc      |  7 +++----
 Utility/Url.hs         |  3 ++-
 27 files changed, 56 insertions(+), 52 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 7022364d09..c5771af28e 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -98,7 +98,7 @@ lockContent key a = do
 			case v of
 				Left _ -> error "content is locked"
 				Right _ -> return $ Just fd
-		unlock Nothing = return ()
+		unlock Nothing = noop
 		unlock (Just l) = closeFd l
 
 {- Calculates the relative path to use to link a file to a key. -}
@@ -237,10 +237,10 @@ cleanObjectLoc key = do
 	file <- inRepo $ gitAnnexLocation key
 	liftIO $ removeparents file (3 :: Int)
 	where
-		removeparents _ 0 = return ()
+		removeparents _ 0 = noop
 		removeparents file n = do
 			let dir = parentDir file
-			maybe (return ()) (const $ removeparents dir (n-1))
+			maybe noop (const $ removeparents dir (n-1))
 				=<< catchMaybeIO (removeDirectory dir)
 
 {- Removes a key's file from .git/annex/objects/ -}
diff --git a/Annex/LockPool.hs b/Annex/LockPool.hs
index 3eb1363eed..b99a8ec4df 100644
--- a/Annex/LockPool.hs
+++ b/Annex/LockPool.hs
@@ -18,7 +18,7 @@ import Annex.Perms
 lockFile :: FilePath -> Annex ()
 lockFile file = go =<< fromPool file
 	where
-		go (Just _) = return () -- already locked
+		go (Just _) = noop -- already locked
 		go Nothing = do
 			mode <- annexFileMode
 			fd <- liftIO $ noUmask mode $
@@ -27,10 +27,9 @@ lockFile file = go =<< fromPool file
 			changePool $ M.insert file fd
 
 unlockFile :: FilePath -> Annex ()
-unlockFile file = go =<< fromPool file
+unlockFile file = maybe noop go =<< fromPool file
 	where
-		go Nothing = return ()
-		go (Just fd) = do
+		go fd = do
 			liftIO $ closeFd fd
 			changePool $ M.delete file
 
diff --git a/Annex/Perms.hs b/Annex/Perms.hs
index 12dfdd6679..c54908b439 100644
--- a/Annex/Perms.hs
+++ b/Annex/Perms.hs
@@ -37,7 +37,7 @@ setAnnexPerm file = withShared $ liftIO . go
 		go GroupShared = groupWriteRead file
 		go AllShared = modifyFileMode file $ addModes $
 			[ ownerWriteMode, groupWriteMode ] ++ readModes
-		go _ = return ()
+		go _ = noop
 
 {- Gets the appropriate mode to use for creating a file in the annex
  - (other than content files, which are locked down more). -}
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index 02a1ee705a..6a230312ab 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -81,7 +81,7 @@ sshCleanup = do
 			v <- liftIO $ tryIO $
 				setLock fd (WriteLock, AbsoluteSeek, 0, 0)
 			case v of
-				Left _ -> return ()
+				Left _ -> noop
 				Right _ -> stopssh socketfile
 			liftIO $ closeFd fd
 		stopssh socketfile = do
diff --git a/Annex/Version.hs b/Annex/Version.hs
index cf5d224842..a1d0402445 100644
--- a/Annex/Version.hs
+++ b/Annex/Version.hs
@@ -35,7 +35,7 @@ setVersion = setConfig versionField defaultVersion
 
 checkVersion :: Version -> Annex ()
 checkVersion v
-	| v `elem` supportedVersions = return ()
+	| v `elem` supportedVersions = noop
 	| v `elem` upgradableVersions = err "Upgrade this repository: git-annex upgrade"
 	| otherwise = err "Upgrade git-annex."
 	where
diff --git a/CmdLine.hs b/CmdLine.hs
index ebaef5369d..910f228b60 100644
--- a/CmdLine.hs
+++ b/CmdLine.hs
@@ -88,7 +88,7 @@ tryRun = tryRun' 0
 tryRun' :: Integer -> Annex.AnnexState -> Command -> [CommandCleanup] -> IO ()
 tryRun' errnum _ cmd []
 	| errnum > 0 = error $ cmdname cmd ++ ": " ++ show errnum ++ " failed"
-	| otherwise = return ()
+	| otherwise = noop
 tryRun' errnum state cmd (a:as) = do
 	r <- run
 	handle $! r
diff --git a/Command/Fsck.hs b/Command/Fsck.hs
index c60101fc79..38b1bbbacd 100644
--- a/Command/Fsck.hs
+++ b/Command/Fsck.hs
@@ -85,7 +85,7 @@ performRemote key file backend numcopies remote =
 			t <- fromRepo gitAnnexTmpDir
 			let tmp = t  "fsck" ++ show pid ++ "." ++ keyFile key
 			liftIO $ createDirectoryIfMissing True t
-			let cleanup = liftIO $ catchIO (removeFile tmp) (const $ return ())
+			let cleanup = liftIO $ catchIO (removeFile tmp) (const noop)
 			cleanup
 			cleanup `after` a tmp
 		getfile tmp =
diff --git a/Command/Status.hs b/Command/Status.hs
index 1ee36d8b47..0c6eda0b2a 100644
--- a/Command/Status.hs
+++ b/Command/Status.hs
@@ -108,12 +108,11 @@ nojson :: StatState String -> String -> StatState String
 nojson a _ = a
 
 showStat :: Stat -> StatState ()
-showStat s = calc =<< s
+showStat s = maybe noop calc =<< s
 	where
-		calc (Just (desc, a)) = do
+		calc (desc, a) = do
 			(lift . showHeader) desc
 			lift . showRaw =<< a
-		calc Nothing = return ()
 
 supported_backends :: Stat
 supported_backends = stat "supported backends" $ json unwords $
diff --git a/Command/Unused.hs b/Command/Unused.hs
index bc721635b7..5bdadcf44a 100644
--- a/Command/Unused.hs
+++ b/Command/Unused.hs
@@ -268,7 +268,7 @@ withKeysReferencedInGitRef a ref = do
 	showAction $ "checking " ++ Git.Ref.describe ref
 	go =<< inRepo (LsTree.lsTree ref)
 	where
-		go [] = return ()
+		go [] = noop
 		go (l:ls)
 			| isSymLink (LsTree.mode l) = do
 				content <- L.decodeUtf8 <$> catFile ref (LsTree.file l)
diff --git a/Command/Whereis.hs b/Command/Whereis.hs
index d4d268d937..eb6ea7c56d 100644
--- a/Command/Whereis.hs
+++ b/Command/Whereis.hs
@@ -46,9 +46,9 @@ perform remotemap key = do
 		untrustedheader = "The following untrusted locations may also have copies:\n"
 
 performRemote :: Key -> Remote -> Annex () 
-performRemote key remote = case whereisKey remote of
-	Nothing -> return ()
-	Just a -> do
-		ls <- a key
-		unless (null ls) $ showLongNote $
-			unlines $ map (\l -> name remote ++ ": " ++ l) ls
+performRemote key remote = maybe noop go $ whereisKey remote
+	where
+		go a = do
+			ls <- a key
+			unless (null ls) $ showLongNote $ unlines $
+				map (\l -> name remote ++ ": " ++ l) ls
diff --git a/Git/Command.hs b/Git/Command.hs
index 50d4455fe7..bb82d13395 100644
--- a/Git/Command.hs
+++ b/Git/Command.hs
@@ -79,5 +79,5 @@ pipeNullSplit params repo =
 reap :: IO ()
 reap = do
 	-- throws an exception when there are no child processes
-	r <- catchDefaultIO (getAnyProcessStatus False True) Nothing
-	maybe (return ()) (const reap) r
+	catchDefaultIO (getAnyProcessStatus False True) Nothing
+		>>= maybe noop (const reap)
diff --git a/Git/Construct.hs b/Git/Construct.hs
index 49905f818b..3f3ea97476 100644
--- a/Git/Construct.hs
+++ b/Git/Construct.hs
@@ -48,7 +48,7 @@ import qualified Git.Url as Url
 fromCurrent :: IO Repo
 fromCurrent = do
 	r <- maybe fromCwd fromPath =<< getEnv "GIT_DIR"
-	maybe (return ()) changeWorkingDirectory =<< getEnv "GIT_WORK_TREE"
+	maybe noop changeWorkingDirectory =<< getEnv "GIT_WORK_TREE"
 	unsetEnv "GIT_DIR"
 	unsetEnv "GIT_WORK_TREE"
 	return r
diff --git a/Git/UnionMerge.hs b/Git/UnionMerge.hs
index 90bbf5c4cc..d68bb61ab1 100644
--- a/Git/UnionMerge.hs
+++ b/Git/UnionMerge.hs
@@ -97,7 +97,7 @@ calc_merge :: CatFileHandle -> [String] -> Repo -> Streamer
 calc_merge ch differ repo streamer = gendiff >>= go
 	where
 		gendiff = pipeNullSplit (map Param differ) repo
-		go [] = return ()
+		go [] = noop
 		go (info:file:rest) = mergeFile info file ch repo >>=
 			maybe (go rest) (\l -> streamer l >> go rest)
 		go (_:[]) = error "calc_merge parse error"
diff --git a/GitAnnexShell.hs b/GitAnnexShell.hs
index 0cf81f0e21..6633037138 100644
--- a/GitAnnexShell.hs
+++ b/GitAnnexShell.hs
@@ -52,7 +52,7 @@ options = Option.common ++
 	where
 		checkuuid expected = getUUID >>= check
 			where
-				check u | u == toUUID expected = return ()
+				check u | u == toUUID expected = noop
 				check NoUUID = unexpected "uninitialized repository"
 				check u = unexpected $ "UUID " ++ fromUUID u
 				unexpected s = error $
@@ -107,7 +107,7 @@ checkNotLimited = checkEnv "GIT_ANNEX_SHELL_LIMITED"
 
 checkNotReadOnly :: String -> IO ()
 checkNotReadOnly cmd
-	| cmd `elem` map cmdname cmds_readonly = return ()
+	| cmd `elem` map cmdname cmds_readonly = noop
 	| otherwise = checkEnv "GIT_ANNEX_SHELL_READONLY"
 
 checkEnv :: String -> IO ()
diff --git a/Logs/Location.hs b/Logs/Location.hs
index b6d59b928c..e27ece5d46 100644
--- a/Logs/Location.hs
+++ b/Logs/Location.hs
@@ -30,7 +30,7 @@ import Logs.Presence
 {- Log a change in the presence of a key's value in a repository. -}
 logChange :: Key -> UUID -> LogStatus -> Annex ()
 logChange key (UUID u) s = addLog (logFile key) =<< logNow s u
-logChange _ NoUUID _ = return ()
+logChange _ NoUUID _ = noop
 
 {- Returns a list of repository UUIDs that, according to the log, have
  - the value of a key.
diff --git a/Logs/UUID.hs b/Logs/UUID.hs
index 18cbee61e4..d825e11273 100644
--- a/Logs/UUID.hs
+++ b/Logs/UUID.hs
@@ -73,7 +73,7 @@ recordUUID u = go . M.lookup u =<< uuidMap
 	where
 		go (Just "") = set
 		go Nothing = set
-		go _ = return ()
+		go _ = noop
 		set = describeUUID u ""
 
 {- Read the uuidLog into a simple Map.
diff --git a/Messages.hs b/Messages.hs
index 73a7d976fd..af7eb88b43 100644
--- a/Messages.hs
+++ b/Messages.hs
@@ -72,8 +72,8 @@ metered key a = Annex.getState Annex.output >>= go (keySize key)
 				incrP progress n
 				displayMeter stdout meter
 			liftIO $ clearMeter stdout meter
-			return r	
-                go _ _ = a (const $ return ())
+			return r
+                go _ _ = a (const noop)
 
 showSideAction :: String -> Annex ()
 showSideAction s = handle q $
@@ -160,7 +160,7 @@ handle json normal = Annex.getState Annex.output >>= go
 		go Annex.JSONOutput = liftIO $ flushed json
 
 q :: Monad m => m ()
-q = return ()
+q = noop
 
 flushed :: IO () -> IO ()
 flushed a = a >> hFlush stdout
diff --git a/Remote.hs b/Remote.hs
index aac45fae9d..e9e66990c5 100644
--- a/Remote.hs
+++ b/Remote.hs
@@ -194,7 +194,7 @@ showLocations key exclude = do
 		message rs us = message rs [] ++ message [] us
 
 showTriedRemotes :: [Remote] -> Annex ()
-showTriedRemotes [] = return ()	
+showTriedRemotes [] = noop
 showTriedRemotes remotes =
 	showLongNote $ "Unable to access these remotes: " ++
 		join ", " (map name remotes)
diff --git a/Remote/Directory.hs b/Remote/Directory.hs
index fd5a6f0b19..7521e70135 100644
--- a/Remote/Directory.hs
+++ b/Remote/Directory.hs
@@ -195,7 +195,8 @@ meteredWriteFile' meterupdate dest startstate feeder =
 	where
 		feed state [] h = do
 			(state', cs) <- feeder state
-			if null cs then return () else feed state' cs h
+			unless (null cs) $
+				feed state' cs h
 		feed state (c:cs) h = do
 			S.hPut h c
 			meterupdate $ toInteger $ S.length c
diff --git a/Remote/Helper/Hooks.hs b/Remote/Helper/Hooks.hs
index 40484b2a7c..d85959062e 100644
--- a/Remote/Helper/Hooks.hs
+++ b/Remote/Helper/Hooks.hs
@@ -46,7 +46,7 @@ runHooks r starthook stophook a = do
 	a
 	where
 		remoteid = show (uuid r)
-		run Nothing = return ()
+		run Nothing = noop
 		run (Just command) = void $ liftIO $
 			boolSystem "sh" [Param "-c", Param command]
 		firstrun lck = do
@@ -81,7 +81,7 @@ runHooks r starthook stophook a = do
 			v <- liftIO $ tryIO $
 				setLock fd (WriteLock, AbsoluteSeek, 0, 0)
 			case v of
-				Left _ -> return ()
+				Left _ -> noop
 				Right _ -> run stophook
 			liftIO $ closeFd fd
 
diff --git a/Remote/S3.hs b/Remote/S3.hs
index a688ffcf34..18d4915dcb 100644
--- a/Remote/S3.hs
+++ b/Remote/S3.hs
@@ -93,7 +93,7 @@ s3Setup u c = handlehost $ M.lookup "host" c
 
 		archiveorg = do
 			showNote "Internet Archive mode"
-			maybe (error "specify bucket=") (const $ return ()) $
+			maybe (error "specify bucket=") (const noop) $
 				M.lookup "bucket" archiveconfig
 			use archiveconfig
 			where
@@ -237,13 +237,13 @@ genBucket c = do
 	showAction "checking bucket"
 	loc <- liftIO $ getBucketLocation conn bucket 
 	case loc of
-		Right _ -> return ()
+		Right _ -> noop
 		Left err@(NetworkError _) -> s3Error err
 		Left (AWSError _ _) -> do
 			showAction $ "creating bucket in " ++ datacenter
 			res <- liftIO $ createBucketIn conn bucket datacenter
 			case res of
-				Right _ -> return ()
+				Right _ -> noop
 				Left err -> s3Error err
 	where
 		bucket = fromJust $ M.lookup "bucket" c
diff --git a/Upgrade/V1.hs b/Upgrade/V1.hs
index 62e3b3b313..a8005b2644 100644
--- a/Upgrade/V1.hs
+++ b/Upgrade/V1.hs
@@ -89,7 +89,7 @@ updateSymlinks = do
 		fixlink f = do
 			r <- lookupFile1 f
 			case r of
-				Nothing -> return ()
+				Nothing -> noop
 				Just (k, _) -> do
 					link <- calcGitLink f k
 					liftIO $ removeFile f
diff --git a/Utility/Directory.hs b/Utility/Directory.hs
index 40e65d6349..e6622d31ee 100644
--- a/Utility/Directory.hs
+++ b/Utility/Directory.hs
@@ -19,6 +19,7 @@ import Control.Applicative
 import Utility.SafeCommand
 import Utility.TempFile
 import Utility.Exception
+import Utility.Monad
 
 {- Lists the contents of a directory.
  - Unlike getDirectoryContents, paths are not relative to the directory. -}
@@ -34,7 +35,7 @@ dirContents d = map (d ) . filter notcruft <$> getDirectoryContents d
 moveFile :: FilePath -> FilePath -> IO ()
 moveFile src dest = tryIO (rename src dest) >>= onrename
 	where
-		onrename (Right _) = return ()
+		onrename (Right _) = noop
 		onrename (Left e)
 			| isPermissionError e = rethrow
 			| isDoesNotExistError e = rethrow
diff --git a/Utility/Inotify.hs b/Utility/Inotify.hs
index 0a261ecfe0..d41e997d61 100644
--- a/Utility/Inotify.hs
+++ b/Utility/Inotify.hs
@@ -56,7 +56,7 @@ watchDir' scan i test add del dir = do
 		then void $ do
 			_ <- addWatch i watchevents dir go
 			mapM walk =<< dirContents dir
-		else return ()
+		else noop
 	where
 		watchevents
 			| isJust add && isJust del =
@@ -68,19 +68,19 @@ watchDir' scan i test add del dir = do
 		recurse = watchDir' scan i test add del
 		walk f = ifM (catchBoolIO $ Files.isDirectory <$> getFileStatus f)
 			( recurse f
-			, if scan && isJust add then fromJust add f else return ()
+			, when (scan && isJust add) $ fromJust add f
 			)
 
-		go (Created { isDirectory = False }) = return ()
+		go (Created { isDirectory = False }) = noop
 		go (Created { filePath = subdir }) = Just recurse <@> subdir
 		go (Closed { maybeFilePath = Just f }) = add <@> f
 		go (MovedIn { isDirectory = False, filePath = f }) = add <@> f
 		go (MovedOut { isDirectory = False, filePath = f }) = del <@> f
 		go (Deleted { isDirectory = False, filePath = f }) = del <@> f
-		go _ = return ()
+		go _ = noop
 		
 		Just a <@> f = a $ dir  f
-		Nothing <@> _ = return ()
+		Nothing <@> _ = noop
 
 {- Pauses the main thread, letting children run until program termination. -}
 waitForTermination :: IO ()
diff --git a/Utility/Monad.hs b/Utility/Monad.hs
index 9c85d31ca8..2c9b9e9e07 100644
--- a/Utility/Monad.hs
+++ b/Utility/Monad.hs
@@ -49,3 +49,7 @@ observe observer a = do
 {- b `after` a runs first a, then b, and returns the value of a -}
 after :: Monad m => m b -> m a -> m a
 after = observe . const
+
+{- do nothing -}
+noop :: Monad m => m ()
+noop = return ()
diff --git a/Utility/Touch.hsc b/Utility/Touch.hsc
index b53eab634e..e2dba79ab2 100644
--- a/Utility/Touch.hsc
+++ b/Utility/Touch.hsc
@@ -106,9 +106,8 @@ touchBoth file atime mtime follow =
 	withFilePath file $ \f -> do
 		pokeArray ptr [atime, mtime]
 		r <- syscall f ptr
-		if (r /= 0)
-			then throwErrno "touchBoth"
-			else return ()
+		when (r /= 0) $
+			throwErrno "touchBoth"
 	where
 		syscall = if follow
 			then c_lutimes
@@ -116,6 +115,6 @@ touchBoth file atime mtime follow =
 
 #else
 #warning "utimensat and lutimes not available; building without symlink timestamp preservation support"
-touchBoth _ _ _ _ = return ()
+touchBoth _ _ _ _ = noop
 #endif
 #endif
diff --git a/Utility/Url.hs b/Utility/Url.hs
index 86d66d83b5..20c5db574d 100644
--- a/Utility/Url.hs
+++ b/Utility/Url.hs
@@ -17,6 +17,7 @@ import Common
 import qualified Network.Browser as Browser
 import Network.HTTP
 import Network.URI
+import Utility.Monad
 
 type URLString = String
 
@@ -95,7 +96,7 @@ request url requesttype = go 5 url
 			case rspCode rsp of
 				(3,0,x) | x /= 5 -> redir (n - 1) u rsp
 				_ -> return rsp
-		ignore = const $ return ()
+		ignore = const noop
 		redir n u rsp = case retrieveHeaders HdrLocation rsp of
 			[] -> return rsp
 			(Header _ newu:_) ->

From 5fbe83f595bf5957376544ee83b3cc46cc2323ed Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 00:07:21 -0400
Subject: [PATCH 051/220] add

---
 doc/todo/http_headers.mdwn | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 doc/todo/http_headers.mdwn

diff --git a/doc/todo/http_headers.mdwn b/doc/todo/http_headers.mdwn
new file mode 100644
index 0000000000..99f083b7ae
--- /dev/null
+++ b/doc/todo/http_headers.mdwn
@@ -0,0 +1,6 @@
+The IA would find it useful to be able to control the http headers
+git-annex get, addurl, etc uses. This will allow setting cookies, for
+example.
+
+* annex-web-headers=blah 
+* Perhaps also annex-web-headers-command=blah

From 84ac8c58db30637db4fb88566530b6293f12dab0 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 01:13:09 -0400
Subject: [PATCH 052/220] Add annex.httpheaders and annex.httpheader-command
 config settings

Allow custom headers to be sent with all HTTP requests.

(Requested by the Internet Archive)
---
 Annex/Content.hs   |  3 ++-
 Command/AddUrl.hs  |  9 ++++++---
 Config.hs          |  9 +++++++++
 Git/Config.hs      |  4 ++++
 Remote/Git.hs      | 14 ++++++++------
 Remote/Web.hs      |  3 ++-
 Utility/Url.hs     | 38 ++++++++++++++++++++++----------------
 debian/changelog   |  3 +++
 doc/git-annex.mdwn | 10 ++++++++++
 9 files changed, 66 insertions(+), 27 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index c5771af28e..01ee7d83d9 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -310,7 +310,8 @@ saveState oneshot = do
 downloadUrl :: [Url.URLString] -> FilePath -> Annex Bool
 downloadUrl urls file = do
 	o <- map Param . words <$> getConfig "annex.web-options" ""
-	liftIO $ anyM (\u -> Url.download u o file) urls
+	headers <- getHttpHeaders
+	liftIO $ anyM (\u -> Url.download u headers o file) urls
 
 {- Copies a key's content, when present, to a temp file.
  - This is used to speed up some rsyncs. -}
diff --git a/Command/AddUrl.hs b/Command/AddUrl.hs
index c87399f5dc..089606e85d 100644
--- a/Command/AddUrl.hs
+++ b/Command/AddUrl.hs
@@ -20,6 +20,7 @@ import Annex.Content
 import Logs.Web
 import qualified Option
 import Types.Key
+import Config
 
 def :: [Command]
 def = [withOptions [fileOption, pathdepthOption] $
@@ -53,8 +54,9 @@ perform url file = ifAnnexed file addurl geturl
 			liftIO $ createDirectoryIfMissing True (parentDir file)
 			ifM (Annex.getState Annex.fast)
 				( nodownload url file , download url file )
-		addurl (key, _backend) =
-			ifM (liftIO $ Url.check url $ keySize key)
+		addurl (key, _backend) = do
+			headers <- getHttpHeaders
+			ifM (liftIO $ Url.check url headers $ keySize key)
 				( do
 					setUrlPresent key url
 					next $ return True
@@ -81,7 +83,8 @@ download url file = do
 
 nodownload :: String -> FilePath -> CommandPerform
 nodownload url file = do
-	(exists, size) <- liftIO $ Url.exists url
+	headers <- getHttpHeaders
+	(exists, size) <- liftIO $ Url.exists url headers
 	if exists
 		then do
 			let key = Backend.URL.fromUrl url size
diff --git a/Config.hs b/Config.hs
index 087cb4043b..065ee48f39 100644
--- a/Config.hs
+++ b/Config.hs
@@ -96,3 +96,12 @@ getDiskReserve = fromMaybe megabyte . readSize dataUnits
 	<$> getConfig "annex.diskreserve" ""
 	where
 		megabyte = 1000000
+
+{- Gets annex.httpheaders or annex.httpheaders-command setting,
+ - splitting it into lines. -}
+getHttpHeaders :: Annex [String]
+getHttpHeaders = do
+	cmd <- getConfig "annex.httpheaders-command" ""
+	if (null cmd)
+		then fromRepo $ Git.Config.getList "annex.httpheaders"
+		else lines . snd <$> liftIO (pipeFrom "sh" ["-c", cmd])
diff --git a/Git/Config.hs b/Git/Config.hs
index 8190a62ad3..38b9ade455 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -20,6 +20,10 @@ import qualified Git.Construct
 get :: String -> String -> Repo -> String
 get key defaultValue repo = M.findWithDefault defaultValue key (config repo)
 
+{- Returns a list with each line of a multiline config setting. -}
+getList :: String -> Repo -> [String]
+getList key repo = M.findWithDefault [] key (fullconfig repo)
+
 {- Returns a single git config setting, if set. -}
 getMaybe :: String -> Repo -> Maybe String
 getMaybe key repo = M.lookup key (config repo)
diff --git a/Remote/Git.hs b/Remote/Git.hs
index d71872b277..35928b96cb 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -94,7 +94,9 @@ tryGitConfigRead :: Git.Repo -> Annex Git.Repo
 tryGitConfigRead r 
 	| not $ M.null $ Git.config r = return r -- already read
 	| Git.repoIsSsh r = store $ onRemote r (pipedconfig, r) "configlist" []
-	| Git.repoIsHttp r = store $ safely geturlconfig
+	| Git.repoIsHttp r = do
+		headers <- getHttpHeaders
+		store $ safely $ geturlconfig headers
 	| Git.repoIsUrl r = return r
 	| otherwise = store $ safely $ onLocal r $ do 
 		ensureInitialized
@@ -109,8 +111,8 @@ tryGitConfigRead r
 			pOpen ReadFromPipe cmd (toCommand params) $
 				Git.Config.hRead r
 
-		geturlconfig = do
-			s <- Url.get (Git.repoLocation r ++ "/config")
+		geturlconfig headers = do
+			s <- Url.get (Git.repoLocation r ++ "/config") headers
 			withTempFile "git-annex.tmp" $ \tmpfile h -> do
 				hPutStr h s
 				hClose h
@@ -136,16 +138,16 @@ tryGitConfigRead r
  -}
 inAnnex :: Git.Repo -> Key -> Annex (Either String Bool)
 inAnnex r key
-	| Git.repoIsHttp r = checkhttp
+	| Git.repoIsHttp r = checkhttp =<< getHttpHeaders
 	| Git.repoIsUrl r = checkremote
 	| otherwise = checklocal
 	where
-		checkhttp = liftIO $ go undefined $ keyUrls r key
+		checkhttp headers = liftIO $ go undefined $ keyUrls r key
 			where
 				go e [] = return $ Left e
 				go _ (u:us) = do
 					res <- catchMsgIO $
-						Url.check u (keySize key)
+						Url.check u headers (keySize key)
 					case res of
 						Left e -> go e us
 						v -> return v
diff --git a/Remote/Web.hs b/Remote/Web.hs
index 81e6ca321c..5fc592326c 100644
--- a/Remote/Web.hs
+++ b/Remote/Web.hs
@@ -83,4 +83,5 @@ checkKey key = do
 checkKey' :: Key -> [URLString] -> Annex Bool
 checkKey' key us = untilTrue us $ \u -> do
 	showAction $ "checking " ++ u
-	liftIO $ Url.check u (keySize key)
+	headers <- getHttpHeaders
+	liftIO $ Url.check u headers (keySize key)
diff --git a/Utility/Url.hs b/Utility/Url.hs
index 20c5db574d..465ef855c8 100644
--- a/Utility/Url.hs
+++ b/Utility/Url.hs
@@ -17,14 +17,16 @@ import Common
 import qualified Network.Browser as Browser
 import Network.HTTP
 import Network.URI
-import Utility.Monad
+import Data.Either
 
 type URLString = String
 
+type Headers = [String]
+
 {- Checks that an url exists and could be successfully downloaded,
  - also checking that its size, if available, matches a specified size. -}
-check :: URLString -> Maybe Integer -> IO Bool
-check url expected_size = handle <$> exists url
+check :: URLString -> Headers -> Maybe Integer -> IO Bool
+check url headers expected_size = handle <$> exists url headers
 	where
 		handle (False, _) = False
 		handle (True, Nothing) = True
@@ -32,12 +34,12 @@ check url expected_size = handle <$> exists url
 
 {- Checks that an url exists and could be successfully downloaded,
  - also returning its size if available. -}
-exists :: URLString -> IO (Bool, Maybe Integer)
-exists url =
+exists :: URLString -> Headers -> IO (Bool, Maybe Integer)
+exists url headers =
 	case parseURI url of
 		Nothing -> return (False, Nothing)
 		Just u -> do
-			r <- request u HEAD
+			r <- request u headers HEAD
 			case rspCode r of
 				(2,_,_) -> return (True, size r)
 				_ -> return (False, Nothing)
@@ -51,26 +53,27 @@ exists url =
  - would not be appropriate to test at configure time and build support
  - for only one in.
  -}
-download :: URLString -> [CommandParam] -> FilePath -> IO Bool
-download url options file = ifM (inPath "wget") (wget , curl)
+download :: URLString -> Headers -> [CommandParam] -> FilePath -> IO Bool
+download url headers options file = ifM (inPath "wget") (wget , curl)
 	where
-		wget = go "wget" [Params "-c -O"]
+		headerparams = map (\h -> Param $ "--header=" ++ h) headers
+		wget = go "wget" $ Params "-c -O" : headerparams
 		{- Uses the -# progress display, because the normal
 		 - one is very confusing when resuming, showing
 		 - the remainder to download as the whole file,
 		 - and not indicating how much percent was
 		 - downloaded before the resume. -}
-		curl = go "curl" [Params "-L -C - -# -o"]
+		curl = go "curl" $ Params "-L -C - -# -o" : headerparams
 		go cmd opts = boolSystem cmd $
 			options++opts++[File file, File url]
 
 {- Downloads a small file. -}
-get :: URLString -> IO String
-get url =
+get :: URLString -> Headers -> IO String
+get url headers =
 	case parseURI url of
 		Nothing -> error "url parse error"
 		Just u -> do
-			r <- request u GET
+			r <- request u headers GET
 			case rspCode r of
 				(2,_,_) -> return $ rspBody r
 				_ -> error $ rspReason r
@@ -82,8 +85,8 @@ get url =
  - This does its own redirect following because Browser's is buggy for HEAD
  - requests.
  -}
-request :: URI -> RequestMethod -> IO (Response String)
-request url requesttype = go 5 url
+request :: URI -> Headers -> RequestMethod -> IO (Response String)
+request url headers requesttype = go 5 url
 	where
 		go :: Int -> URI -> IO (Response String)
 		go 0 _ = error "Too many redirects "
@@ -92,7 +95,8 @@ request url requesttype = go 5 url
 				Browser.setErrHandler ignore
 				Browser.setOutHandler ignore
 				Browser.setAllowRedirects False
-				snd <$> Browser.request (mkRequest requesttype u :: Request_String)
+				let req = mkRequest requesttype u :: Request_String
+				snd <$> Browser.request (addheaders req)
 			case rspCode rsp of
 				(3,0,x) | x /= 5 -> redir (n - 1) u rsp
 				_ -> return rsp
@@ -105,3 +109,5 @@ request url requesttype = go 5 url
 					Just newURI -> go n newURI_abs
 						where
 							newURI_abs = fromMaybe newURI (newURI `relativeTo` u)
+		addheaders req = setHeaders req (rqHeaders req ++ userheaders)
+		userheaders = rights $ map parseHeader headers
diff --git a/debian/changelog b/debian/changelog
index 3c56fda51a..c55932e3e0 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -3,6 +3,9 @@ git-annex (3.20120419) UNRELEASED; urgency=low
   * Fix use of annex.diskreserve config setting.
   * Directory special remotes now check annex.diskreserve.
   * Support git's core.sharedRepository configuration.
+  * Add annex.httpheaders and annex.httpheader-command config
+    settings, to allow custom headers to be sent with all HTTP requests.
+    (Requested by the Internet Archive)
 
  -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 72301c0719..098d520010 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -713,6 +713,16 @@ Here are all the supported configuration settings.
   (wget is always used in preference to curl if available).
   For example, to force ipv4 only, set it to "-4"
 
+* `annex.http-headers`
+
+  HTTP headers to send when downloading from the web. Multiple lines of
+  this option can be set, one per header.
+
+* `annex.http-headers-command`
+
+  If set, the command is run and each line of its output is used as a HTTP
+  header. This overrides annex.http-headers.
+
 * `remote..rsyncurl`
 
   Used by rsync special remotes, this configures

From f8fc79cf58fa8c3acf9948a162a81850ffbbccb5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 01:16:08 -0400
Subject: [PATCH 053/220] fixed

---
 doc/todo/http_headers.mdwn | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/todo/http_headers.mdwn b/doc/todo/http_headers.mdwn
index 99f083b7ae..9f61bdc931 100644
--- a/doc/todo/http_headers.mdwn
+++ b/doc/todo/http_headers.mdwn
@@ -4,3 +4,5 @@ example.
 
 * annex-web-headers=blah 
 * Perhaps also annex-web-headers-command=blah
+
+[[done]]

From 42e4145a170a12f557fbe58344bb115061452d2a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 01:20:17 -0400
Subject: [PATCH 054/220] bugfixes

---
 Config.hs      | 4 ++--
 Utility/Url.hs | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/Config.hs b/Config.hs
index 065ee48f39..3feb246e53 100644
--- a/Config.hs
+++ b/Config.hs
@@ -101,7 +101,7 @@ getDiskReserve = fromMaybe megabyte . readSize dataUnits
  - splitting it into lines. -}
 getHttpHeaders :: Annex [String]
 getHttpHeaders = do
-	cmd <- getConfig "annex.httpheaders-command" ""
+	cmd <- getConfig "annex.http-headers-command" ""
 	if (null cmd)
-		then fromRepo $ Git.Config.getList "annex.httpheaders"
+		then fromRepo $ Git.Config.getList "annex.http-headers"
 		else lines . snd <$> liftIO (pipeFrom "sh" ["-c", cmd])
diff --git a/Utility/Url.hs b/Utility/Url.hs
index 465ef855c8..b75229e1bc 100644
--- a/Utility/Url.hs
+++ b/Utility/Url.hs
@@ -57,13 +57,13 @@ download :: URLString -> Headers -> [CommandParam] -> FilePath -> IO Bool
 download url headers options file = ifM (inPath "wget") (wget , curl)
 	where
 		headerparams = map (\h -> Param $ "--header=" ++ h) headers
-		wget = go "wget" $ Params "-c -O" : headerparams
+		wget = go "wget" $ headerparams ++ [Params "-c -O"]
 		{- Uses the -# progress display, because the normal
 		 - one is very confusing when resuming, showing
 		 - the remainder to download as the whole file,
 		 - and not indicating how much percent was
 		 - downloaded before the resume. -}
-		curl = go "curl" $ Params "-L -C - -# -o" : headerparams
+		curl = go "curl" $ headerparams ++ [Params "-L -C - -# -o"]
 		go cmd opts = boolSystem cmd $
 			options++opts++[File file, File url]
 

From 2d0290bda0b62b03a65659d5c238a5d173cdf7f5 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 22 Apr 2012 05:28:21 +0000
Subject: [PATCH 055/220]

---
 .../error_in_installation_of_base-4.5.0.0.mdwn     | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
 create mode 100644 doc/forum/error_in_installation_of_base-4.5.0.0.mdwn

diff --git a/doc/forum/error_in_installation_of_base-4.5.0.0.mdwn b/doc/forum/error_in_installation_of_base-4.5.0.0.mdwn
new file mode 100644
index 0000000000..673222ed6d
--- /dev/null
+++ b/doc/forum/error_in_installation_of_base-4.5.0.0.mdwn
@@ -0,0 +1,14 @@
+Hi,
+
+I was trying to install git-annex, but then, I got a warning saying that I need to install base-4.5.0.0 first.
+
+So, I did "sudo cabal install base-4.5.0.0". Everything was going well, until I got this error:
+
+config.status: error: cannot find input file: `base.buildinfo.in'
+cabal: Error: some packages failed to install:
+base-4.5.0.0 failed during the configure step. The exception was:
+ExitFailure 1
+
+I tried to look for information on the internet, but I did not find anything useful.
+
+I know that this is not totally related to git-annex, but anyone has any thoughts on this?

From e36b2c7b3d72161019ac2295a4f658b4dc2ab837 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 01:31:19 -0400
Subject: [PATCH 056/220] typo

---
 doc/install/Fedora.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/install/Fedora.mdwn b/doc/install/Fedora.mdwn
index a4dd4f5c87..50f1d78180 100644
--- a/doc/install/Fedora.mdwn
+++ b/doc/install/Fedora.mdwn
@@ -4,7 +4,7 @@ Installation recipe for Fedora 14 thruough 17.
 sudo yum install ghc cabal-install
 git clone git://git-annex.branchable.com/ git-annex
 cd git-annex
-git checkout ghc 7.0
+git checkout ghc7.0
 cabal update
 cabal install --only-dependencies
 cabal configure

From 744b33c240ec476de4de616738d192fa919ae003 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sun, 22 Apr 2012 05:39:28 +0000
Subject: [PATCH 057/220] Added a comment

---
 ..._0b2f79c014e0dd9badd52b8b6aa47e0c._comment | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 doc/forum/error_in_installation_of_base-4.5.0.0/comment_1_0b2f79c014e0dd9badd52b8b6aa47e0c._comment

diff --git a/doc/forum/error_in_installation_of_base-4.5.0.0/comment_1_0b2f79c014e0dd9badd52b8b6aa47e0c._comment b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_1_0b2f79c014e0dd9badd52b8b6aa47e0c._comment
new file mode 100644
index 0000000000..5a52a28ab1
--- /dev/null
+++ b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_1_0b2f79c014e0dd9badd52b8b6aa47e0c._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 1"
+ date="2012-04-22T05:39:28Z"
+ content="""
+git-annex needs ghc 7.4, that's why it depends on that base version that comes with it. So you either need to upgrade your ghc, or you can build from the `ghc7.0` branch in [[git|download]], like this:
+
+
+git clone git://git-annex.branchable.com/ git-annex
+cd git-annex
+git checkout ghc7.0
+cabal update
+cabal install --only-dependencies
+cabal configure
+cabal build
+cabal install --bindir=$HOME/bin
+

+"""]]

From 54c2cb8558e8e9747c8f721604cd47b060a69e90 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 22 Apr 2012 14:09:33 +0000
Subject: [PATCH 058/220] Added a comment

---
 ..._3badd64e48fbb174cd7de1ac9589bedf._comment | 31 +++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 doc/forum/error_in_installation_of_base-4.5.0.0/comment_2_3badd64e48fbb174cd7de1ac9589bedf._comment

diff --git a/doc/forum/error_in_installation_of_base-4.5.0.0/comment_2_3badd64e48fbb174cd7de1ac9589bedf._comment b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_2_3badd64e48fbb174cd7de1ac9589bedf._comment
new file mode 100644
index 0000000000..3ad1a11388
--- /dev/null
+++ b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_2_3badd64e48fbb174cd7de1ac9589bedf._comment
@@ -0,0 +1,31 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 2"
+ date="2012-04-22T14:09:33Z"
+ content="""
+Thanks for the fast response!
+
+Unfortunately, I had another problem:
+
+==================================
+Building git-annex-3.20120419...
+Utility/libdiskfree.c: In function ‘diskfree’:
+
+Utility/libdiskfree.c:61:0:
+     warning: ‘statfs64’ is deprecated (declared at /usr/include/sys/mount.h:379)
+[  6 of 157] Compiling Build.SysConfig  ( Build/SysConfig.hs, dist/build/git-annex/git-annex-tmp/Build/SysConfig.o )
+[ 15 of 157] Compiling Utility.Touch    ( dist/build/git-annex/git-annex-tmp/Utility/Touch.hs, dist/build/git-annex/git-annex-tmp/Utility/Touch.o )
+
+Utility/Touch.hsc:118:21: Not in scope: `noop'
+cabal: Error: some packages failed to install:
+git-annex-3.20120419 failed during the building phase. The exception was:
+ExitFailure 1
+==================================
+
+I also tried to look for information on the internet, and I did not find anything useful.
+Any idea of what happened?
+
+Thanks again!
+
+"""]]

From 94e479dd1b71c0495baae690cf0eabb282919417 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 01:49:15 -0400
Subject: [PATCH 059/220] simplfy versioned base dependency

---
 git-annex.cabal | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/git-annex.cabal b/git-annex.cabal
index 041b077dc5..0635de6b17 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -34,7 +34,7 @@ Executable git-annex
   Build-Depends: MissingH, hslogger, directory, filepath,
    unix, containers, utf8-string, network, mtl, bytestring, old-locale, time,
    pcre-light, extensible-exceptions, dataenc, SHA, process, json, HTTP,
-   base >= 4.5, base < 5, monad-control, transformers-base, lifted-base,
+   base == 4.5.*, base < 5, monad-control, transformers-base, lifted-base,
    IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   Other-Modules: Utility.Touch
   C-Sources: Utility/libdiskfree.c
@@ -50,7 +50,7 @@ Test-Suite test
   Build-Depends: testpack, HUnit, MissingH, hslogger, directory, filepath,
    unix, containers, utf8-string, network, mtl, bytestring, old-locale, time,
    pcre-light, extensible-exceptions, dataenc, SHA, process, json, HTTP,
-   base >= 4.5, base < 5, monad-control, transformers-base, lifted-base,
+   base == 4.5.*, monad-control, transformers-base, lifted-base,
    IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   C-Sources: Utility/libdiskfree.c
   Extensions: CPP

From f7667742098e8147f54e15e57b130dafda156017 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 11:21:23 -0400
Subject: [PATCH 060/220] unbreak code inside ifdef

---
 Utility/Touch.hsc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Utility/Touch.hsc b/Utility/Touch.hsc
index e2dba79ab2..0b1ca3d9bd 100644
--- a/Utility/Touch.hsc
+++ b/Utility/Touch.hsc
@@ -115,6 +115,6 @@ touchBoth file atime mtime follow =
 
 #else
 #warning "utimensat and lutimes not available; building without symlink timestamp preservation support"
-touchBoth _ _ _ _ = noop
+touchBoth _ _ _ _ = return ()
 #endif
 #endif

From b5fc3746a8f3d390aad9d7966815e9b72e8f2eee Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sun, 22 Apr 2012 15:23:26 +0000
Subject: [PATCH 061/220] Added a comment

---
 .../comment_3_d8190061ac1c683a7b699cf42e9db694._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/error_in_installation_of_base-4.5.0.0/comment_3_d8190061ac1c683a7b699cf42e9db694._comment

diff --git a/doc/forum/error_in_installation_of_base-4.5.0.0/comment_3_d8190061ac1c683a7b699cf42e9db694._comment b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_3_d8190061ac1c683a7b699cf42e9db694._comment
new file mode 100644
index 0000000000..d9de5089e9
--- /dev/null
+++ b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_3_d8190061ac1c683a7b699cf42e9db694._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 3"
+ date="2012-04-22T15:23:26Z"
+ content="""
+That's my fault, I made a change last night that caused the noop problem. Fixed now.
+"""]]

From 1b876daad10b05dde49d8fef4befa70a11e8d9ed Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 11:24:39 -0400
Subject: [PATCH 062/220] remove unneeded dependency

---
 git-annex.cabal | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/git-annex.cabal b/git-annex.cabal
index 0635de6b17..da0e9f4de7 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -34,7 +34,7 @@ Executable git-annex
   Build-Depends: MissingH, hslogger, directory, filepath,
    unix, containers, utf8-string, network, mtl, bytestring, old-locale, time,
    pcre-light, extensible-exceptions, dataenc, SHA, process, json, HTTP,
-   base == 4.5.*, base < 5, monad-control, transformers-base, lifted-base,
+   base == 4.5.*, monad-control, transformers-base, lifted-base,
    IfElse, text, QuickCheck >= 2.1, bloomfilter, edit-distance
   Other-Modules: Utility.Touch
   C-Sources: Utility/libdiskfree.c

From 1db09af14cf41dff96be23935e4a690b95c0f6b8 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 22 Apr 2012 11:42:38 -0400
Subject: [PATCH 063/220] fix names

---
 debian/changelog | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/changelog b/debian/changelog
index c55932e3e0..1962a6b669 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -3,7 +3,7 @@ git-annex (3.20120419) UNRELEASED; urgency=low
   * Fix use of annex.diskreserve config setting.
   * Directory special remotes now check annex.diskreserve.
   * Support git's core.sharedRepository configuration.
-  * Add annex.httpheaders and annex.httpheader-command config
+  * Add annex.http-headers and annex.http-headers-command config
     settings, to allow custom headers to be sent with all HTTP requests.
     (Requested by the Internet Archive)
 

From cd8934e4a2268431d4cc5dc8e665cf5303dfdb46 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 22 Apr 2012 16:08:56 +0000
Subject: [PATCH 064/220] Added a comment

---
 .../comment_4_49a4fcd2dc4f97d4055b5051feea5e3b._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/error_in_installation_of_base-4.5.0.0/comment_4_49a4fcd2dc4f97d4055b5051feea5e3b._comment

diff --git a/doc/forum/error_in_installation_of_base-4.5.0.0/comment_4_49a4fcd2dc4f97d4055b5051feea5e3b._comment b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_4_49a4fcd2dc4f97d4055b5051feea5e3b._comment
new file mode 100644
index 0000000000..ba99334511
--- /dev/null
+++ b/doc/forum/error_in_installation_of_base-4.5.0.0/comment_4_49a4fcd2dc4f97d4055b5051feea5e3b._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 4"
+ date="2012-04-22T16:08:55Z"
+ content="""
+Thanks, it worked now!
+"""]]

From 450201dcee2bd0900cf29da4ddf04a41457e3662 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Mon, 23 Apr 2012 04:03:17 +0000
Subject: [PATCH 065/220]

---
 doc/forum/What_can_be_done_in_case_of_conflict.mdwn | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 doc/forum/What_can_be_done_in_case_of_conflict.mdwn

diff --git a/doc/forum/What_can_be_done_in_case_of_conflict.mdwn b/doc/forum/What_can_be_done_in_case_of_conflict.mdwn
new file mode 100644
index 0000000000..42167a6091
--- /dev/null
+++ b/doc/forum/What_can_be_done_in_case_of_conflict.mdwn
@@ -0,0 +1,7 @@
+Hi,
+
+How can I resolve the conflict when it occurs?
+
+Suppose I have 2 branches (master, current), When I merge these branches or while doing cherry-pick, if I get conflict how can I resolve it?
+
+Thank You

From c3e6fb165ee21aa944f034288b23ca6e0c0b9a1f Mon Sep 17 00:00:00 2001
From: "http://christian.amsuess.com/chrysn" 
Date: Mon, 23 Apr 2012 14:14:29 +0000
Subject: [PATCH 066/220] Added a comment

---
 ...comment_3_c336b2b07cd006d378e5be9639ff17ec._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_3_c336b2b07cd006d378e5be9639ff17ec._comment

diff --git a/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_3_c336b2b07cd006d378e5be9639ff17ec._comment b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_3_c336b2b07cd006d378e5be9639ff17ec._comment
new file mode 100644
index 0000000000..fd75f2f856
--- /dev/null
+++ b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_3_c336b2b07cd006d378e5be9639ff17ec._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://christian.amsuess.com/chrysn"
+ nickname="chrysn"
+ subject="comment 3"
+ date="2012-04-23T14:14:28Z"
+ content="""
+thanks, that's great. will there be a way to have sharedRepository work for shared remotes (rsync, directory) too, or is that better taken care of by acls?
+
+@not thought of shared repos: we're having our family photo archive spread over our laptops, and backed up on our home storage server and on an rsync+encryption off-site server, with everyone naturally having their own accounts on all systems -- just if you need a use case.
+"""]]

From 7de646596faf4533aca65787539b5e1e244f9e68 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Mon, 23 Apr 2012 14:29:03 +0000
Subject: [PATCH 067/220] Added a comment

---
 ...mment_1_5ca86b099dfa08a50f656ea03bf1dcd9._comment | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 doc/forum/What_can_be_done_in_case_of_conflict/comment_1_5ca86b099dfa08a50f656ea03bf1dcd9._comment

diff --git a/doc/forum/What_can_be_done_in_case_of_conflict/comment_1_5ca86b099dfa08a50f656ea03bf1dcd9._comment b/doc/forum/What_can_be_done_in_case_of_conflict/comment_1_5ca86b099dfa08a50f656ea03bf1dcd9._comment
new file mode 100644
index 0000000000..6f1d241300
--- /dev/null
+++ b/doc/forum/What_can_be_done_in_case_of_conflict/comment_1_5ca86b099dfa08a50f656ea03bf1dcd9._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 1"
+ date="2012-04-23T14:29:03Z"
+ content="""
+You handle conflicts in annexed files the same as you would handle them in other binary files checked into git.
+
+For example, you might choose to `git rm` or `git add` the file to resolve the conflict.
+
+[[Previous_discussion|forum/A_really_stupid_question]]
+"""]]

From 5cabab81540268aaf4da56c4fd897951483756b1 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Mon, 23 Apr 2012 14:35:40 +0000
Subject: [PATCH 068/220] Added a comment

---
 ...comment_4_1339cd27ca2955f30b01ecf4da7d6fe8._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_4_1339cd27ca2955f30b01ecf4da7d6fe8._comment

diff --git a/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_4_1339cd27ca2955f30b01ecf4da7d6fe8._comment b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_4_1339cd27ca2955f30b01ecf4da7d6fe8._comment
new file mode 100644
index 0000000000..568f11330c
--- /dev/null
+++ b/doc/forum/__34__permission_denied__34___in_fsck_on_shared_repo/comment_4_1339cd27ca2955f30b01ecf4da7d6fe8._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 4"
+ date="2012-04-23T14:35:39Z"
+ content="""
+I'm not currently planning to support sharedRepository perms on special remotes. I suppose I could be convinced otherwise, it's perhaps doable for the ones you mention (rsync might be tricky). (bup special remote already supports it of course.)
+
+thanks for the use case!
+"""]]

From eedde3454901887f1a29766b15f60e26be64583a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 23 Apr 2012 10:37:05 -0400
Subject: [PATCH 069/220] show amount of reserved space

---
 Command/Status.hs | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/Command/Status.hs b/Command/Status.hs
index 0c6eda0b2a..16bcec7cce 100644
--- a/Command/Status.hs
+++ b/Command/Status.hs
@@ -69,6 +69,7 @@ fast_stats =
 	, remote_list SemiTrusted "semitrusted"
 	, remote_list UnTrusted "untrusted"
 	, remote_list DeadTrusted "dead"
+	, disk_size
 	]
 slow_stats :: [Stat]
 slow_stats = 
@@ -78,7 +79,6 @@ slow_stats =
 	, local_annex_size
 	, known_annex_keys
 	, known_annex_size
-	, disk_size
 	, bloom_info
 	, backend_usage
 	]
@@ -175,8 +175,12 @@ disk_size = stat "available local disk space" $ json id $ lift $
 		<$> getDiskReserve
 		<*> inRepo (getDiskFree . gitAnnexDir)
 	where
-		calcfree reserve (Just have) =
-			roughSize storageUnits False $ nonneg $ have - reserve
+		calcfree reserve (Just have) = unwords
+			[ roughSize storageUnits False $ nonneg $ have - reserve
+			, "(+" ++ roughSize storageUnits False reserve
+			, "reserved)"
+			]
+			
 		calcfree _ _ = "unknown"
 		nonneg x
 			| x >= 0 = x

From 6050c7cca789311eaa12e561c23e2589df1fa6b5 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Tue, 24 Apr 2012 21:10:28 +0000
Subject: [PATCH 070/220]

---
 doc/forum/retrieving_previous_versions.mdwn | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 doc/forum/retrieving_previous_versions.mdwn

diff --git a/doc/forum/retrieving_previous_versions.mdwn b/doc/forum/retrieving_previous_versions.mdwn
new file mode 100644
index 0000000000..7626b7935b
--- /dev/null
+++ b/doc/forum/retrieving_previous_versions.mdwn
@@ -0,0 +1,7 @@
+Hi,
+
+This might be a stupid question, but I did not find any information about it.
+Can I retrieve previous versions of a file?
+Let's say, I wanna do a "git annex get file", but considering a specific commit id. Is it possible? Are all the versions of the files kept inside .git/annex/objects?
+
+Thanks!

From 64e0e9d2cd68c453a02d71f21043312312aa099d Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Tue, 24 Apr 2012 21:14:15 +0000
Subject: [PATCH 071/220] Added a comment

---
 ...omment_1_a4e83f688d4ec9177e7bf520f12ed26d._comment | 11 +++++++++++
 1 file changed, 11 insertions(+)
 create mode 100644 doc/forum/retrieving_previous_versions/comment_1_a4e83f688d4ec9177e7bf520f12ed26d._comment

diff --git a/doc/forum/retrieving_previous_versions/comment_1_a4e83f688d4ec9177e7bf520f12ed26d._comment b/doc/forum/retrieving_previous_versions/comment_1_a4e83f688d4ec9177e7bf520f12ed26d._comment
new file mode 100644
index 0000000000..ab2ecee317
--- /dev/null
+++ b/doc/forum/retrieving_previous_versions/comment_1_a4e83f688d4ec9177e7bf520f12ed26d._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 1"
+ date="2012-04-24T21:14:15Z"
+ content="""
+To get to a specific version of a file, you need to have a tag or a branch that includes that version of the file. Check out the branch and `git annex get $file`. 
+
+(Of course, even without a tag or branch, old file versions are retained, unless dropped with `unused`/`dropunused`.
+So you could even `git checkout $COMMITID`.)
+"""]]

From a2ee4386bb8f64046a5838038c89492dddcce94f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkO9tsPZkAxEulq2pGCdwz4md-LqB0RcMw"
 
Date: Wed, 25 Apr 2012 22:56:20 +0000
Subject: [PATCH 072/220] Added a comment: Problems with Base & Crypto

---
 ..._e6109a964064a2a799768a370e57801d._comment | 30 +++++++++++++++++++
 1 file changed, 30 insertions(+)
 create mode 100644 doc/install/OSX/comment_4_e6109a964064a2a799768a370e57801d._comment

diff --git a/doc/install/OSX/comment_4_e6109a964064a2a799768a370e57801d._comment b/doc/install/OSX/comment_4_e6109a964064a2a799768a370e57801d._comment
new file mode 100644
index 0000000000..be3ba2be44
--- /dev/null
+++ b/doc/install/OSX/comment_4_e6109a964064a2a799768a370e57801d._comment
@@ -0,0 +1,30 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkO9tsPZkAxEulq2pGCdwz4md-LqB0RcMw"
+ nickname="Reimund"
+ subject="Problems with Base & Crypto"
+ date="2012-04-25T22:56:18Z"
+ content="""
+I got the following error message trying to install git-annex:
+
+    cabal: cannot configure git-annex-3.20120418. It requires base >=4.5 && <5
+    For the dependency on base >=4.5 && <5 there are these packages: base-4.5.0.0.
+    However none of them are available.
+    base-4.5.0.0 was excluded because of the top level dependency base -any
+
+These are the steps I performed to make it work
+
+1. Download [Ghc 7.4](http://www.haskell.org/ghc/download).
+2. Run `sudo cabal install git-annex --bindir=$HOME/bin`.
+3. Compilation of the Crypto-4.2.4 dependency failed since it's not updated to work with Ghc 7.4. You need to patch SHA2.hs (steps below).
+4. Run `sudo cabal install git-annex --bindir=$HOME/bin` a second time.
+
+The steps I did to patch the SHA2.hs file in Crypto-4.2.4:
+
+1. `cabal unpack crypto-4.2.4`
+2. `cd Crypto-4.2.4`
+3. `patch -p1 < crypto-4.2.4-ghc-7.4.patch`
+4. `sudo cabal install`.
+
+PS: I used [this patchfile](http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-haskell/crypto/files/crypto-4.2.4-ghc-7.4.patch?revision=1.1).
+Then I did the last step a third time.
+"""]]

From e0b7012ccc405dedb556b8c940eb66e42304bc73 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 27 Apr 2012 12:21:38 -0400
Subject: [PATCH 073/220] uninit: Clear annex.uuid from .git/config. Closes:
 #670639

---
 Annex/UUID.hs    | 8 +++++++-
 Init.hs          | 4 +++-
 debian/changelog | 1 +
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/Annex/UUID.hs b/Annex/UUID.hs
index e8306de903..5459cc7fee 100644
--- a/Annex/UUID.hs
+++ b/Annex/UUID.hs
@@ -16,12 +16,14 @@ module Annex.UUID (
 	getRepoUUID,
 	getUncachedUUID,
 	prepUUID,
-	genUUID
+	genUUID,
+	removeRepoUUID,
 ) where
 
 import Common.Annex
 import qualified Git
 import qualified Git.Config
+import qualified Git.Command
 import qualified Build.SysConfig as SysConfig
 import Config
 
@@ -61,6 +63,10 @@ getRepoUUID r = do
 			when (g /= r) $ storeUUID cachekey u
 		cachekey = remoteConfig r "uuid"
 
+removeRepoUUID :: Annex ()
+removeRepoUUID = inRepo $ Git.Command.run "config"
+	[Param "--unset", Param configkey]
+
 getUncachedUUID :: Git.Repo -> UUID
 getUncachedUUID = toUUID . Git.Config.get configkey ""
 
diff --git a/Init.hs b/Init.hs
index 9f1988a394..a0e16e8815 100644
--- a/Init.hs
+++ b/Init.hs
@@ -29,7 +29,9 @@ initialize mdescription = do
 	maybe (recordUUID u) (describeUUID u) mdescription
 
 uninitialize :: Annex ()
-uninitialize = gitPreCommitHookUnWrite
+uninitialize = do
+	gitPreCommitHookUnWrite
+	removeRepoUUID
 
 {- Will automatically initialize if there is already a git-annex
    branch from somewhere. Otherwise, require a manual init
diff --git a/debian/changelog b/debian/changelog
index 1962a6b669..06990a0f8a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -6,6 +6,7 @@ git-annex (3.20120419) UNRELEASED; urgency=low
   * Add annex.http-headers and annex.http-headers-command config
     settings, to allow custom headers to be sent with all HTTP requests.
     (Requested by the Internet Archive)
+  * uninit: Clear annex.uuid from .git/config. Closes: #670639
 
  -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
 

From 76102c1c7541e7b10c3a3fbe242e9856fef955b3 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 27 Apr 2012 13:23:52 -0400
Subject: [PATCH 074/220] display "Recording state in git..." when staging the
 journal

A bit tricky to avoid printing it twice in a row when there are queued git
commands to run and journal to stage.

Added a generic way to run an action that may output multiple side
messages, with only the first displayed.
---
 Annex.hs          |  8 +++-----
 Annex/Branch.hs   |  1 +
 Annex/Content.hs  |  4 ++--
 Annex/Queue.hs    |  9 ++++-----
 Messages.hs       | 45 +++++++++++++++++++++++++++++++++++----------
 Option.hs         | 10 ++++++----
 Types/Messages.hs | 20 ++++++++++++++++++++
 Upgrade/V1.hs     |  2 +-
 8 files changed, 72 insertions(+), 27 deletions(-)
 create mode 100644 Types/Messages.hs

diff --git a/Annex.hs b/Annex.hs
index 2b58c2bd49..441e460beb 100644
--- a/Annex.hs
+++ b/Annex.hs
@@ -10,7 +10,6 @@
 module Annex (
 	Annex,
 	AnnexState(..),
-	OutputType(..),
 	new,
 	newState,
 	run,
@@ -44,6 +43,7 @@ import qualified Types.Remote
 import Types.Crypto
 import Types.BranchState
 import Types.TrustLevel
+import Types.Messages
 import Utility.State
 import qualified Utility.Matcher
 import qualified Data.Map as M
@@ -69,8 +69,6 @@ instance MonadBaseControl IO Annex where
 		where
 			unStAnnex (StAnnex st) = st
 
-data OutputType = NormalOutput | QuietOutput | JSONOutput
-
 type Matcher a = Either [Utility.Matcher.Token a] (Utility.Matcher.Matcher a)
 
 -- internal state storage
@@ -78,7 +76,7 @@ data AnnexState = AnnexState
 	{ repo :: Git.Repo
 	, backends :: [BackendA Annex]
 	, remotes :: [Types.Remote.RemoteA Annex]
-	, output :: OutputType
+	, output :: MessageState
 	, force :: Bool
 	, fast :: Bool
 	, auto :: Bool
@@ -104,7 +102,7 @@ newState gitrepo = AnnexState
 	{ repo = gitrepo
 	, backends = []
 	, remotes = []
-	, output = NormalOutput
+	, output = defaultMessageState
 	, force = False
 	, fast = False
 	, auto = False
diff --git a/Annex/Branch.hs b/Annex/Branch.hs
index ce1dd58cec..706522f3b3 100644
--- a/Annex/Branch.hs
+++ b/Annex/Branch.hs
@@ -330,6 +330,7 @@ setCommitted = void $ do
 {- Stages the journal into the index. -}
 stageJournal :: Annex ()
 stageJournal = do
+	showStoringStateAction
 	fs <- getJournalFiles
 	g <- gitRepo
 	withIndex $ liftIO $ do
diff --git a/Annex/Content.hs b/Annex/Content.hs
index 01ee7d83d9..b5754e15be 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -297,8 +297,8 @@ getKeysPresent = liftIO . traverse (2 :: Int) =<< fromRepo gitAnnexObjectDir
  - especially if performing a short-lived action.
  -}
 saveState :: Bool -> Annex ()
-saveState oneshot = do
-	Annex.Queue.flush False
+saveState oneshot = doSideAction $ do
+	Annex.Queue.flush
 	unless oneshot $
 		ifM alwayscommit
 			( Annex.Branch.commit "update" , Annex.Branch.stage)
diff --git a/Annex/Queue.hs b/Annex/Queue.hs
index f49a220690..728e29645e 100644
--- a/Annex/Queue.hs
+++ b/Annex/Queue.hs
@@ -26,15 +26,14 @@ add command params files = do
 flushWhenFull :: Annex ()
 flushWhenFull = do
 	q <- get
-	when (Git.Queue.full q) $ flush False
+	when (Git.Queue.full q) flush
 
 {- Runs (and empties) the queue. -}
-flush :: Bool -> Annex ()
-flush silent = do
+flush :: Annex ()
+flush = do
 	q <- get
 	unless (0 == Git.Queue.size q) $ do
-		unless silent $
-			showSideAction "Recording state in git"
+		showStoringStateAction
 		q' <- inRepo $ Git.Queue.flush q
 		store q'
 
diff --git a/Messages.hs b/Messages.hs
index af7eb88b43..4330f7c09f 100644
--- a/Messages.hs
+++ b/Messages.hs
@@ -13,6 +13,8 @@ module Messages (
 	metered,
 	MeterUpdate,
 	showSideAction,
+	doSideAction,
+	showStoringStateAction,
 	showOutput,
 	showLongNote,
 	showEndOk,
@@ -37,6 +39,7 @@ import Data.Quantity
 
 import Common
 import Types
+import Types.Messages
 import Types.Key
 import qualified Annex
 import qualified Messages.JSON as JSON
@@ -61,9 +64,9 @@ showProgress = handle q $
  - The action is passed a callback to use to update the meter. -}
 type MeterUpdate = Integer -> IO ()
 metered :: Key -> (MeterUpdate -> Annex a) -> Annex a
-metered key a = Annex.getState Annex.output >>= go (keySize key)
+metered key a = withOutputType $ go (keySize key)
 	where
-		go (Just size) Annex.NormalOutput = do
+		go (Just size) NormalOutput = do
 			progress <- liftIO $ newProgress "" size
 			meter <- liftIO $ newMeter progress "B" 25 (renderNums binaryOpts 1)
 			showOutput
@@ -76,8 +79,27 @@ metered key a = Annex.getState Annex.output >>= go (keySize key)
                 go _ _ = a (const noop)
 
 showSideAction :: String -> Annex ()
-showSideAction s = handle q $
-	putStrLn $ "(" ++ s ++ "...)"
+showSideAction m = Annex.getState Annex.output >>= go
+	where
+		go (MessageState v StartBlock) = do
+			p
+	 		Annex.changeState $ \s -> s { Annex.output = MessageState v InBlock }
+		go (MessageState _ InBlock) = return ()
+		go _ = p
+		p = handle q $ putStrLn $ "(" ++ m ++ "...)"
+			
+showStoringStateAction :: Annex ()
+showStoringStateAction = showSideAction "Recording state in git"
+
+{- Performs an action, that may call showSideAction multiple times.
+ - Only the first will be displayed. -}
+doSideAction :: Annex a -> Annex a
+doSideAction a = do
+	o <- Annex.getState Annex.output
+	set $ o { sideActionBlock = StartBlock }
+	set o `after` a
+	where
+		set o = Annex.changeState $ \s -> s {  Annex.output = o }
 
 showOutput :: Annex ()
 showOutput = handle q $
@@ -122,9 +144,9 @@ maybeShowJSON v = handle (JSON.add v) q
 
 {- Shows a complete JSON value, only when in json mode. -}
 showFullJSON :: JSON a => [(String, a)] -> Annex Bool
-showFullJSON v = Annex.getState Annex.output >>= liftIO . go
+showFullJSON v = withOutputType $ liftIO . go
 	where
-		go Annex.JSONOutput = JSON.complete v >> return True
+		go JSONOutput = JSON.complete v >> return True
 		go _ = return False
 
 {- Performs an action that outputs nonstandard/customized output, and
@@ -153,14 +175,17 @@ setupConsole = do
 	fileEncoding stderr
 
 handle :: IO () -> IO () -> Annex ()
-handle json normal = Annex.getState Annex.output >>= go
+handle json normal = withOutputType $ go
 	where
-		go Annex.NormalOutput = liftIO normal
-		go Annex.QuietOutput = q
-		go Annex.JSONOutput = liftIO $ flushed json
+		go NormalOutput = liftIO normal
+		go QuietOutput = q
+		go JSONOutput = liftIO $ flushed json
 
 q :: Monad m => m ()
 q = noop
 
 flushed :: IO () -> IO ()
 flushed a = a >> hFlush stdout
+
+withOutputType :: (OutputType -> Annex a) -> Annex a
+withOutputType a = outputType <$> Annex.getState Annex.output >>= a
diff --git a/Option.hs b/Option.hs
index 2f0d00744d..7696220554 100644
--- a/Option.hs
+++ b/Option.hs
@@ -20,6 +20,7 @@ import System.Log.Logger
 
 import Common.Annex
 import qualified Annex
+import Types.Messages
 import Limit
 import Usage
 
@@ -31,11 +32,11 @@ common =
 		"avoid slow operations"
 	, Option ['a'] ["auto"] (NoArg (setauto True))
 		"automatic mode"
-	, Option ['q'] ["quiet"] (NoArg (setoutput Annex.QuietOutput))
+	, Option ['q'] ["quiet"] (NoArg (setoutput QuietOutput))
 		"avoid verbose output"
-	, Option ['v'] ["verbose"] (NoArg (setoutput Annex.NormalOutput))
+	, Option ['v'] ["verbose"] (NoArg (setoutput NormalOutput))
 		"allow verbose output (default)"
-	, Option ['j'] ["json"] (NoArg (setoutput Annex.JSONOutput))
+	, Option ['j'] ["json"] (NoArg (setoutput JSONOutput))
 		"enable JSON output"
 	, Option ['d'] ["debug"] (NoArg setdebug)
 		"show debug messages"
@@ -46,7 +47,8 @@ common =
 		setforce v = Annex.changeState $ \s -> s { Annex.force = v }
 		setfast v = Annex.changeState $ \s -> s { Annex.fast = v }
 		setauto v = Annex.changeState $ \s -> s { Annex.auto = v }
-		setoutput v = Annex.changeState $ \s -> s { Annex.output = v }
+		setoutput v = Annex.changeState $ \s ->
+			s { Annex.output = (Annex.output s) { outputType = v } }
 		setforcebackend v = Annex.changeState $ \s -> s { Annex.forcebackend = Just v }
 		setdebug = liftIO $ updateGlobalLogger rootLoggerName $
 			setLevel DEBUG
diff --git a/Types/Messages.hs b/Types/Messages.hs
new file mode 100644
index 0000000000..75653d574c
--- /dev/null
+++ b/Types/Messages.hs
@@ -0,0 +1,20 @@
+{- git-annex Messages data types
+ - 
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Types.Messages where
+
+data OutputType = NormalOutput | QuietOutput | JSONOutput
+
+data SideActionBlock = NoBlock | StartBlock | InBlock
+
+data MessageState = MessageState
+	{ outputType :: OutputType
+	, sideActionBlock :: SideActionBlock
+	}
+
+defaultMessageState :: MessageState
+defaultMessageState = MessageState NormalOutput NoBlock
diff --git a/Upgrade/V1.hs b/Upgrade/V1.hs
index a8005b2644..ddf0728b61 100644
--- a/Upgrade/V1.hs
+++ b/Upgrade/V1.hs
@@ -59,7 +59,7 @@ upgrade = do
 			updateSymlinks
 			moveLocationLogs
 	
-			Annex.Queue.flush True
+			Annex.Queue.flush
 			setVersion
 		)
 	

From 090d50a7f7741e1b8d69a416563e86739f747a0b Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 
Date: Sat, 28 Apr 2012 22:53:38 +0000
Subject: [PATCH 075/220]

---
 doc/forum/wishlist:_simpler_gpg_usage.mdwn | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage.mdwn

diff --git a/doc/forum/wishlist:_simpler_gpg_usage.mdwn b/doc/forum/wishlist:_simpler_gpg_usage.mdwn
new file mode 100644
index 0000000000..f09f817af1
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage.mdwn
@@ -0,0 +1,10 @@
+This is my current understanding on how one must use gpg with git-annex:
+
+ * Generate(or copy around) a gpg key on every machine that needs to access the encrypted remote.
+ * git annex initremote myremote encryption=KEY for each key that you generated
+
+What I'm trying to figure out is if I can generate a no-passphrase gpg key and commit it to the repository, and have git-annex use that. That way any new clones of the annex automatically have access to any encrypted remotes, without having to do any key management.
+
+I think I can generate a no-passphrase key, but then I still have to manually copy it around to each machine.
+
+I'm not a huge gpg user so part of this is me wanting to avoid having to manage and keeping track of the keys.  This would probably be a non-issue if I used gpg on more machines and was more comfortable with it.

From 20a25d2f20f255f434193bab31d0981d74b3aee3 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 
Date: Sun, 29 Apr 2012 01:41:58 +0000
Subject: [PATCH 076/220] Added a comment

---
 ..._6923fa6ebc0bbe7d93edb1d01d7c46c5._comment | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage/comment_1_6923fa6ebc0bbe7d93edb1d01d7c46c5._comment

diff --git a/doc/forum/wishlist:_simpler_gpg_usage/comment_1_6923fa6ebc0bbe7d93edb1d01d7c46c5._comment b/doc/forum/wishlist:_simpler_gpg_usage/comment_1_6923fa6ebc0bbe7d93edb1d01d7c46c5._comment
new file mode 100644
index 0000000000..f96f5c3777
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage/comment_1_6923fa6ebc0bbe7d93edb1d01d7c46c5._comment
@@ -0,0 +1,19 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
+ nickname="Justin"
+ subject="comment 1"
+ date="2012-04-29T01:41:57Z"
+ content="""
+Thinking about this more, I think minimally git-annex could support a
+
+    remote..gpg-options
+
+or
+
+    remote..gpg-keyring
+
+for options to be passed to gpg.  I'm not sure how automatically setting it to $ANNEX_ROOT/.gnupg/.. would work.
+
+
+I need to read the encryption code to fully understand it, but I also wonder if there is not also a way to just bypass gpg entirely and store the remote-encryption keys locally in plain text.
+"""]]

From 44defe8b70a299a66050e8c60fe7f46daeed6661 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sun, 29 Apr 2012 02:39:20 +0000
Subject: [PATCH 077/220] Added a comment

---
 ...comment_2_6fc874b6c391df242bd2592c4a65eae8._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage/comment_2_6fc874b6c391df242bd2592c4a65eae8._comment

diff --git a/doc/forum/wishlist:_simpler_gpg_usage/comment_2_6fc874b6c391df242bd2592c4a65eae8._comment b/doc/forum/wishlist:_simpler_gpg_usage/comment_2_6fc874b6c391df242bd2592c4a65eae8._comment
new file mode 100644
index 0000000000..29eb11622a
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage/comment_2_6fc874b6c391df242bd2592c4a65eae8._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 2"
+ date="2012-04-29T02:39:20Z"
+ content="""
+The encryption uses a symmetric cipher that is stored in the git repository already. It's just stored encrypted to the various gpg keys that have been configured to use it. It would certianly be possible to store the symmetric cipher unencrypted in the git repo. 
+
+I don't see your idea of gpg-options saving any work. It would still require you to do key distribution and run commands in each repo to set it up.
+"""]]

From 0828f31a8bbf6f3bb27b2b2c0b0802269ceb3b58 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sun, 29 Apr 2012 02:41:38 +0000
Subject: [PATCH 078/220] Added a comment

---
 .../comment_3_012f340c8c572fe598fc860c1046dabd._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage/comment_3_012f340c8c572fe598fc860c1046dabd._comment

diff --git a/doc/forum/wishlist:_simpler_gpg_usage/comment_3_012f340c8c572fe598fc860c1046dabd._comment b/doc/forum/wishlist:_simpler_gpg_usage/comment_3_012f340c8c572fe598fc860c1046dabd._comment
new file mode 100644
index 0000000000..051f17a24b
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage/comment_3_012f340c8c572fe598fc860c1046dabd._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 3"
+ date="2012-04-29T02:41:38Z"
+ content="""
+BTW re your Tweet.. I was so happy to be able to use 'c i a' in Crypto.hs. :)
+"""]]

From d7a4a9a66bd51b18a9e5f4427c3492db1adec40d Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
 
Date: Sun, 29 Apr 2012 03:09:04 +0000
Subject: [PATCH 079/220] Added a comment

---
 ...comment_4_e0c2a13217b795964f3b630c001661ef._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage/comment_4_e0c2a13217b795964f3b630c001661ef._comment

diff --git a/doc/forum/wishlist:_simpler_gpg_usage/comment_4_e0c2a13217b795964f3b630c001661ef._comment b/doc/forum/wishlist:_simpler_gpg_usage/comment_4_e0c2a13217b795964f3b630c001661ef._comment
new file mode 100644
index 0000000000..c9e3375414
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage/comment_4_e0c2a13217b795964f3b630c001661ef._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
+ nickname="Justin"
+ subject="comment 4"
+ date="2012-04-29T03:09:03Z"
+ content="""
+I got a good laugh out of it :-)
+
+Storing the key unencrypted would make things easier..  I think at least for my use-cases I don't require another layer of protection on top of the ssh keys that provide access to the encrypted remotes themselves.
+"""]]

From 1c16f616df9a8469d24cefb6007333df3a35a449 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 29 Apr 2012 14:02:18 -0400
Subject: [PATCH 080/220] Added shared cipher mode to encryptable special
 remotes.

This option avoids gpg key distribution, at the expense of flexability, and
with the requirement that all clones of the git repository be equally
trusted.
---
 Annex.hs                     |  2 +-
 Crypto.hs                    | 75 +++++++++++++++---------------------
 Remote/Helper/Encryptable.hs | 25 +++++++-----
 Types/Crypto.hs              |  6 +--
 Utility/Gpg.hs               | 13 ++++++-
 debian/changelog             |  3 ++
 doc/encryption.mdwn          | 13 +++++++
 7 files changed, 79 insertions(+), 58 deletions(-)

diff --git a/Annex.hs b/Annex.hs
index 441e460beb..f2fb1f4019 100644
--- a/Annex.hs
+++ b/Annex.hs
@@ -90,7 +90,7 @@ data AnnexState = AnnexState
 	, shared :: Maybe SharedRepository
 	, forcetrust :: TrustMap
 	, trustmap :: Maybe TrustMap
-	, ciphers :: M.Map EncryptedCipher Cipher
+	, ciphers :: M.Map StorableCipher Cipher
 	, lockpool :: M.Map FilePath Fd
 	, flags :: M.Map String Bool
 	, fields :: M.Map String String
diff --git a/Crypto.hs b/Crypto.hs
index cb1ca40d14..e530bd0e65 100644
--- a/Crypto.hs
+++ b/Crypto.hs
@@ -3,16 +3,17 @@
  - Currently using gpg; could later be modified to support different
  - crypto backends if neccessary.
  -
- - Copyright 2011 Joey Hess 
+ - Copyright 2011-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
 module Crypto (
 	Cipher,
-	EncryptedCipher,
-	genCipher,
-	updateCipher,
+	StorableCipher(..),
+	genEncryptedCipher,
+	genSharedCipher,
+	updateEncryptedCipher,
 	describeCipher,
 	storeCipher,
 	extractCipher,
@@ -60,59 +61,55 @@ cipherPassphrase (Cipher c) = drop cipherHalf c
 cipherHmac :: Cipher -> String
 cipherHmac (Cipher c) = take cipherHalf c
 
-{- Creates a new Cipher, encrypted as specified in the remote's configuration -}
-genCipher :: RemoteConfig -> IO EncryptedCipher
-genCipher c = do
-	ks <- configKeyIds c
-	random <- genrandom
+{- Creates a new Cipher, encrypted to the specificed key id. -}
+genEncryptedCipher :: String -> IO StorableCipher
+genEncryptedCipher keyid = do
+	ks <- Gpg.findPubKeys keyid
+	random <- Gpg.genRandom cipherSize
 	encryptCipher (Cipher random) ks
-	where
-		genrandom = Gpg.readStrict
-			-- Armor the random data, to avoid newlines,
-			-- since gpg only reads ciphers up to the first
-			-- newline.
-			[ Params "--gen-random --armor"
-			, Param $ show randomquality
-			, Param $ show cipherSize
-			]
-		-- 1 is /dev/urandom; 2 is /dev/random
-		randomquality = 1 :: Int
 
-{- Updates an existing Cipher, re-encrypting it to add KeyIds specified in
- - the remote's configuration. -}
-updateCipher :: RemoteConfig -> EncryptedCipher -> IO EncryptedCipher
-updateCipher c encipher@(EncryptedCipher _ ks) = do
-	ks' <- configKeyIds c
-	cipher <- decryptCipher c encipher
+{- Creates a new, shared Cipher. -}
+genSharedCipher :: IO StorableCipher
+genSharedCipher = SharedCipher <$> Gpg.genRandom cipherSize
+
+{- Updates an existing Cipher, re-encrypting it to add a keyid. -}
+updateEncryptedCipher :: String -> StorableCipher -> IO StorableCipher
+updateEncryptedCipher _ (SharedCipher _) = undefined
+updateEncryptedCipher keyid encipher@(EncryptedCipher _ ks) = do
+	ks' <- Gpg.findPubKeys keyid
+	cipher <- decryptCipher encipher
 	encryptCipher cipher (merge ks ks')
 	where
 		merge (KeyIds a) (KeyIds b) = KeyIds $ a ++ b
 
-describeCipher :: EncryptedCipher -> String
+describeCipher :: StorableCipher -> String
+describeCipher (SharedCipher _) = "shared cipher"
 describeCipher (EncryptedCipher _ (KeyIds ks)) =
 	"with gpg " ++ keys ks ++ " " ++ unwords ks
 	where
 		keys [_] = "key"
 		keys _ = "keys"
 
-{- Stores an EncryptedCipher in a remote's configuration. -}
-storeCipher :: RemoteConfig -> EncryptedCipher -> RemoteConfig
+{- Stores an StorableCipher in a remote's configuration. -}
+storeCipher :: RemoteConfig -> StorableCipher -> RemoteConfig
+storeCipher c (SharedCipher t) = M.insert "cipher" (toB64 t) c
 storeCipher c (EncryptedCipher t ks) = 
 	M.insert "cipher" (toB64 t) $ M.insert "cipherkeys" (showkeys ks) c
 	where
 		showkeys (KeyIds l) = join "," l
 
-{- Extracts an EncryptedCipher from a remote's configuration. -}
-extractCipher :: RemoteConfig -> Maybe EncryptedCipher
+{- Extracts an StorableCipher from a remote's configuration. -}
+extractCipher :: RemoteConfig -> Maybe StorableCipher
 extractCipher c = 
 	case (M.lookup "cipher" c, M.lookup "cipherkeys" c) of
 		(Just t, Just ks) -> Just $ EncryptedCipher (fromB64 t) (readkeys ks)
+		(Just t, Nothing) -> Just $ SharedCipher (fromB64 t)
 		_ -> Nothing
 	where
 		readkeys = KeyIds . split ","
 
 {- Encrypts a Cipher to the specified KeyIds. -}
-encryptCipher :: Cipher -> KeyIds -> IO EncryptedCipher
+encryptCipher :: Cipher -> KeyIds -> IO StorableCipher
 encryptCipher (Cipher c) (KeyIds ks) = do
 	let ks' = nub $ sort ks -- gpg complains about duplicate recipient keyids
 	encipher <- Gpg.pipeStrict (encrypt++recipients ks') c
@@ -126,9 +123,9 @@ encryptCipher (Cipher c) (KeyIds ks) = do
 		force_recipients = Params "--no-encrypt-to --no-default-recipient"
 
 {- Decrypting an EncryptedCipher is expensive; the Cipher should be cached. -}
-decryptCipher :: RemoteConfig -> EncryptedCipher -> IO Cipher
-decryptCipher _ (EncryptedCipher encipher _) = 
-	Cipher <$> Gpg.pipeStrict decrypt encipher
+decryptCipher :: StorableCipher -> IO Cipher
+decryptCipher (SharedCipher t) = return $ Cipher t
+decryptCipher (EncryptedCipher t _) = Cipher <$> Gpg.pipeStrict decrypt t
 	where
 		decrypt = [ Param "--decrypt" ]
 
@@ -165,14 +162,6 @@ pass :: (Cipher -> IO L.ByteString -> (Handle -> IO a) -> IO a)
       -> Cipher -> IO L.ByteString -> (L.ByteString -> IO a) -> IO a
 pass to c i a = to c i $ \h -> a =<< L.hGetContents h
 
-configKeyIds :: RemoteConfig -> IO KeyIds
-configKeyIds c = Gpg.findPubKeys $ configGet c "encryption"
-
-configGet :: RemoteConfig -> String -> String
-configGet c key = fromMaybe missing $ M.lookup key c
-	where
-		missing = error $ "missing " ++ key ++ " in remote config"
-
 hmacWithCipher :: Cipher -> String -> String
 hmacWithCipher c = hmacWithCipher' (cipherHmac c) 
 hmacWithCipher' :: String -> String -> String
diff --git a/Remote/Helper/Encryptable.hs b/Remote/Helper/Encryptable.hs
index bcecb30cc4..a44e6e4539 100644
--- a/Remote/Helper/Encryptable.hs
+++ b/Remote/Helper/Encryptable.hs
@@ -17,17 +17,22 @@ import Config
 
 {- Encryption setup for a remote. The user must specify whether to use
  - an encryption key, or not encrypt. An encrypted cipher is created, or is
- - updated to be accessible to an additional encryption key. -}
+ - updated to be accessible to an additional encryption key. Or the user
+ - could opt to use a shared cipher, which is stored unencrypted. -}
 encryptionSetup :: RemoteConfig -> Annex RemoteConfig
-encryptionSetup c =
-	case (M.lookup "encryption" c, extractCipher c) of
-		(Nothing, Nothing) -> error "Specify encryption=key or encryption=none"
-		(Just "none", Nothing) -> return c
-		(Just "none", Just _) -> error "Cannot change encryption type of existing remote."
-		(Nothing, Just _) -> return c
-		(Just _, Nothing) -> use "encryption setup" $ genCipher c
-		(Just _, Just v) -> use "encryption updated" $ updateCipher c v
+encryptionSetup c = case (M.lookup "encryption" c, extractCipher c) of
+	(Nothing, Nothing) -> error "Specify encryption=key or encryption=none or encryption=shared"
+	(Just "none", Nothing) -> return c
+	(Nothing, Just _) -> return c
+	(Just "shared", Just (SharedCipher _)) -> return c
+	(Just "none", Just _) -> cannotchange
+	(Just "shared", Just (EncryptedCipher _ _)) -> cannotchange
+	(Just _, Just (SharedCipher _)) -> cannotchange
+	(Just "shared", Nothing) -> use "encryption setup" $ genSharedCipher
+	(Just keyid, Nothing) -> use "encryption setup" $ genEncryptedCipher keyid
+	(Just keyid, Just v) -> use "encryption updated" $ updateEncryptedCipher keyid v
 	where
+		cannotchange = error "Cannot change encryption type of existing remote."
 		use m a = do
 			cipher <- liftIO a
 			showNote $ m ++ " " ++ describeCipher cipher
@@ -78,7 +83,7 @@ remoteCipher c = go $ extractCipher c
 				Nothing -> decrypt encipher cache
 		decrypt encipher cache = do
 			showNote "gpg"
-			cipher <- liftIO $ decryptCipher c encipher
+			cipher <- liftIO $ decryptCipher encipher
 			Annex.changeState (\s -> s { Annex.ciphers = M.insert encipher cipher cache })
 			return $ Just cipher
 
diff --git a/Types/Crypto.hs b/Types/Crypto.hs
index 686bf5c1a6..135522ba11 100644
--- a/Types/Crypto.hs
+++ b/Types/Crypto.hs
@@ -1,13 +1,13 @@
 {- git-annex crypto types
  -
- - Copyright 2011 Joey Hess 
+ - Copyright 2011-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
 module Types.Crypto (
 	Cipher(..),
-	EncryptedCipher(..),
+	StorableCipher(..),
 	KeyIds(..),
 ) where
 
@@ -16,5 +16,5 @@ import Utility.Gpg (KeyIds(..))
 -- XXX ideally, this would be a locked memory region
 newtype Cipher = Cipher String
 
-data EncryptedCipher = EncryptedCipher String KeyIds
+data StorableCipher = EncryptedCipher String KeyIds | SharedCipher String
 	deriving (Ord, Eq)
diff --git a/Utility/Gpg.hs b/Utility/Gpg.hs
index 4c798f2732..ff6735ba57 100644
--- a/Utility/Gpg.hs
+++ b/Utility/Gpg.hs
@@ -94,7 +94,18 @@ findPubKeys for = KeyIds . parse <$> readStrict params
 		pubKey = isPrefixOf "pub:"
 		keyIdField s = split ":" s !! 4
 
-
+{- Creates a block of high-quality random data suitable to use as a cipher.
+ - It is armored, to avoid newlines, since gpg only reads ciphers up to the
+ - first newline. -}
+genRandom :: Int -> IO String
+genRandom size = readStrict
+	[ Params "--gen-random --armor"
+	, Param $ show randomquality
+	, Param $ show size
+	]
+	where
+		-- 1 is /dev/urandom; 2 is /dev/random
+		randomquality = 1 :: Int
 
 {- A test key. This is provided pre-generated since generating a new gpg
  - key is too much work (requires too much entropy) for a test suite to
diff --git a/debian/changelog b/debian/changelog
index 06990a0f8a..7a9b5c6eab 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -7,6 +7,9 @@ git-annex (3.20120419) UNRELEASED; urgency=low
     settings, to allow custom headers to be sent with all HTTP requests.
     (Requested by the Internet Archive)
   * uninit: Clear annex.uuid from .git/config. Closes: #670639
+  * Added shared cipher mode to encryptable special remotes. This option
+    avoids gpg key distribution, at the expense of flexability, and with
+    the requirement that all clones of the git repository be equally trusted.
 
  -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
 
diff --git a/doc/encryption.mdwn b/doc/encryption.mdwn
index 0f83bb7f90..cc61fea6f7 100644
--- a/doc/encryption.mdwn
+++ b/doc/encryption.mdwn
@@ -33,3 +33,16 @@ Note that once a key has been given access to a remote, it's not
 possible to revoke that access, short of deleting the remote. See
 [[encryption_design|design/encryption]] for other security risks
 associated with encryption.
+
+## shared cipher mode
+
+Alternatively, you can configure git-annex to use a shared cipher to
+encrypt data stored in a remote. This shared cipher is stored,
+**unencrypted** in the git repository. So it's shared amoung every
+clone of the git repository. The advantage is you don't need to set up gpg
+keys. The disadvantage is that this is **insecure** unless you
+trust every clone of the git repository with access to the encrypted data
+stored in the special remote.
+
+To use shared encryption, specify "encryption=shared" when first setting
+up a special remote.

From 9f04367336965fcfc0b78aeacefe70970e77db19 Mon Sep 17 00:00:00 2001
From: "http://joey.kitenet.net/" 
Date: Sun, 29 Apr 2012 18:04:13 +0000
Subject: [PATCH 081/220] Added a comment

---
 .../comment_5_9668b58eb71901e1db8da7db38e068ca._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/wishlist:_simpler_gpg_usage/comment_5_9668b58eb71901e1db8da7db38e068ca._comment

diff --git a/doc/forum/wishlist:_simpler_gpg_usage/comment_5_9668b58eb71901e1db8da7db38e068ca._comment b/doc/forum/wishlist:_simpler_gpg_usage/comment_5_9668b58eb71901e1db8da7db38e068ca._comment
new file mode 100644
index 0000000000..60b98bde5d
--- /dev/null
+++ b/doc/forum/wishlist:_simpler_gpg_usage/comment_5_9668b58eb71901e1db8da7db38e068ca._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 5"
+ date="2012-04-29T18:04:13Z"
+ content="""
+encryption=shared is now supported
+"""]]

From bd592d1450e52a99e7507a211ad1c36414d3d869 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 29 Apr 2012 14:31:34 -0400
Subject: [PATCH 082/220] refactor

---
 Crypto.hs                    | 26 ++------------------------
 Remote/Helper/Encryptable.hs | 19 +++++++++++++++++++
 2 files changed, 21 insertions(+), 24 deletions(-)

diff --git a/Crypto.hs b/Crypto.hs
index e530bd0e65..58c0e6d008 100644
--- a/Crypto.hs
+++ b/Crypto.hs
@@ -10,13 +10,12 @@
 
 module Crypto (
 	Cipher,
+	KeyIds(..),
 	StorableCipher(..),
 	genEncryptedCipher,
 	genSharedCipher,
 	updateEncryptedCipher,
 	describeCipher,
-	storeCipher,
-	extractCipher,
 	decryptCipher,		
 	encryptKey,
 	withEncryptedHandle,
@@ -28,7 +27,6 @@ module Crypto (
 ) where
 
 import qualified Data.ByteString.Lazy.Char8 as L
-import qualified Data.Map as M
 import Data.ByteString.Lazy.UTF8 (fromString)
 import Data.Digest.Pure.SHA
 import Control.Applicative
@@ -36,8 +34,6 @@ import Control.Applicative
 import Common.Annex
 import qualified Utility.Gpg as Gpg
 import Types.Key
-import Types.Remote
-import Utility.Base64
 import Types.Crypto
 
 {- The first half of a Cipher is used for HMAC; the remainder
@@ -90,24 +86,6 @@ describeCipher (EncryptedCipher _ (KeyIds ks)) =
 		keys [_] = "key"
 		keys _ = "keys"
 
-{- Stores an StorableCipher in a remote's configuration. -}
-storeCipher :: RemoteConfig -> StorableCipher -> RemoteConfig
-storeCipher c (SharedCipher t) = M.insert "cipher" (toB64 t) c
-storeCipher c (EncryptedCipher t ks) = 
-	M.insert "cipher" (toB64 t) $ M.insert "cipherkeys" (showkeys ks) c
-	where
-		showkeys (KeyIds l) = join "," l
-
-{- Extracts an StorableCipher from a remote's configuration. -}
-extractCipher :: RemoteConfig -> Maybe StorableCipher
-extractCipher c = 
-	case (M.lookup "cipher" c, M.lookup "cipherkeys" c) of
-		(Just t, Just ks) -> Just $ EncryptedCipher (fromB64 t) (readkeys ks)
-		(Just t, Nothing) -> Just $ SharedCipher (fromB64 t)
-		_ -> Nothing
-	where
-		readkeys = KeyIds . split ","
-
 {- Encrypts a Cipher to the specified KeyIds. -}
 encryptCipher :: Cipher -> KeyIds -> IO StorableCipher
 encryptCipher (Cipher c) (KeyIds ks) = do
@@ -160,7 +138,7 @@ withDecryptedContent = pass withDecryptedHandle
 
 pass :: (Cipher -> IO L.ByteString -> (Handle -> IO a) -> IO a) 
       -> Cipher -> IO L.ByteString -> (L.ByteString -> IO a) -> IO a
-pass to c i a = to c i $ \h -> a =<< L.hGetContents h
+pass to n s a = to n s $ \h -> a =<< L.hGetContents h
 
 hmacWithCipher :: Cipher -> String -> String
 hmacWithCipher c = hmacWithCipher' (cipherHmac c) 
diff --git a/Remote/Helper/Encryptable.hs b/Remote/Helper/Encryptable.hs
index a44e6e4539..789a1d9964 100644
--- a/Remote/Helper/Encryptable.hs
+++ b/Remote/Helper/Encryptable.hs
@@ -14,6 +14,7 @@ import Types.Remote
 import Crypto
 import qualified Annex
 import Config
+import Utility.Base64
 
 {- Encryption setup for a remote. The user must specify whether to use
  - an encryption key, or not encrypt. An encrypted cipher is created, or is
@@ -93,3 +94,21 @@ cipherKey Nothing _ = return Nothing
 cipherKey (Just c) k = maybe Nothing encrypt <$> remoteCipher c
 	where
 		encrypt ciphertext = Just (ciphertext, encryptKey ciphertext k)
+
+{- Stores an StorableCipher in a remote's configuration. -}
+storeCipher :: RemoteConfig -> StorableCipher -> RemoteConfig
+storeCipher c (SharedCipher t) = M.insert "cipher" (toB64 t) c
+storeCipher c (EncryptedCipher t ks) = 
+	M.insert "cipher" (toB64 t) $ M.insert "cipherkeys" (showkeys ks) c
+	where
+		showkeys (KeyIds l) = join "," l
+
+{- Extracts an StorableCipher from a remote's configuration. -}
+extractCipher :: RemoteConfig -> Maybe StorableCipher
+extractCipher c = 
+	case (M.lookup "cipher" c, M.lookup "cipherkeys" c) of
+		(Just t, Just ks) -> Just $ EncryptedCipher (fromB64 t) (readkeys ks)
+		(Just t, Nothing) -> Just $ SharedCipher (fromB64 t)
+		_ -> Nothing
+	where
+		readkeys = KeyIds . split ","

From d2bfba6324ca54253be62716b1bbdc86e0e1aafe Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 29 Apr 2012 16:10:47 -0400
Subject: [PATCH 083/220] show percent the bloom filter is full

---
 Command/Status.hs | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/Command/Status.hs b/Command/Status.hs
index 16bcec7cce..057ab72bfa 100644
--- a/Command/Status.hs
+++ b/Command/Status.hs
@@ -12,6 +12,7 @@ module Command.Status where
 import Control.Monad.State.Strict
 import qualified Data.Map as M
 import Text.JSON
+import Data.Ratio
 
 import Common.Annex
 import qualified Types.Backend as B
@@ -160,7 +161,7 @@ bloom_info = stat "bloom filter size" $ json id $ do
 	let note = aside $
 		if localkeys >= capacity
 		then "appears too small for this repository; adjust annex.bloomcapacity"
-		else "has room for " ++ show (capacity - localkeys) ++ " more local annex keys"
+		else show (floor (percentage capacity localkeys) :: Integer) ++ "% full"
 
 	-- Two bloom filters are used at the same time, so double the size
 	-- of one.
@@ -169,6 +170,10 @@ bloom_info = stat "bloom filter size" $ json id $ do
 
 	return $ size ++ note
 
+	where
+		percentage :: Integer -> Integer -> Double
+		percentage full have = 100 * (fromRational $ have % full)
+
 disk_size :: Stat
 disk_size = stat "available local disk space" $ json id $ lift $
 	calcfree

From 0c9c14b52fcdd9684da70c8dd763661acdbd3843 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 29 Apr 2012 17:48:07 -0400
Subject: [PATCH 084/220] percentage library

---
 Command/Status.hs     |  8 ++------
 Utility/Percentage.hs | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 40 insertions(+), 6 deletions(-)
 create mode 100644 Utility/Percentage.hs

diff --git a/Command/Status.hs b/Command/Status.hs
index 057ab72bfa..2540a92da8 100644
--- a/Command/Status.hs
+++ b/Command/Status.hs
@@ -12,7 +12,6 @@ module Command.Status where
 import Control.Monad.State.Strict
 import qualified Data.Map as M
 import Text.JSON
-import Data.Ratio
 
 import Common.Annex
 import qualified Types.Backend as B
@@ -31,6 +30,7 @@ import Logs.UUID
 import Logs.Trust
 import Remote
 import Config
+import Utility.Percentage
 
 -- a named computation that produces a statistic
 type Stat = StatState (Maybe (String, StatState String))
@@ -161,7 +161,7 @@ bloom_info = stat "bloom filter size" $ json id $ do
 	let note = aside $
 		if localkeys >= capacity
 		then "appears too small for this repository; adjust annex.bloomcapacity"
-		else show (floor (percentage capacity localkeys) :: Integer) ++ "% full"
+		else showPercentage 1 (percentage capacity localkeys) ++ " full"
 
 	-- Two bloom filters are used at the same time, so double the size
 	-- of one.
@@ -170,10 +170,6 @@ bloom_info = stat "bloom filter size" $ json id $ do
 
 	return $ size ++ note
 
-	where
-		percentage :: Integer -> Integer -> Double
-		percentage full have = 100 * (fromRational $ have % full)
-
 disk_size :: Stat
 disk_size = stat "available local disk space" $ json id $ lift $
 	calcfree
diff --git a/Utility/Percentage.hs b/Utility/Percentage.hs
new file mode 100644
index 0000000000..309e00181d
--- /dev/null
+++ b/Utility/Percentage.hs
@@ -0,0 +1,38 @@
+{- percentages
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Utility.Percentage (
+	Percentage,
+	percentage,
+	showPercentage
+) where
+
+import Data.Ratio
+
+newtype Percentage = Percentage (Ratio Integer)
+
+instance Show Percentage where
+	show = showPercentage 0
+
+{- Normally the big number comes first. But 110% is allowed if desired. :) -}
+percentage :: Integer -> Integer -> Percentage
+percentage 0 _ = Percentage 0
+percentage full have = Percentage $ have * 100 % full
+
+{- Pretty-print a Percentage, with a specified level of precision. -}
+showPercentage :: Int -> Percentage -> String
+showPercentage precision (Percentage p)
+	| precision == 0 || remainder == 0 = go $ show int
+	| otherwise = go $ show int ++ "." ++ strip0s (show remainder)
+	where
+		go v = v ++ "%"
+		int :: Integer
+		(int, frac) = properFraction (fromRational p)
+		remainder = floor (frac * multiplier) :: Integer
+		strip0s = reverse . dropWhile (== '0') . reverse
+		multiplier :: Float
+		multiplier = 10 ** (fromIntegral precision)

From bf70bbdbfc875836cf880ba8a03cac74948a30a3 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 30 Apr 2012 13:59:05 -0400
Subject: [PATCH 085/220] fix test suite build

---
 Annex.hs  | 6 ++++++
 Option.hs | 8 +++-----
 test.hs   | 9 +++++----
 3 files changed, 14 insertions(+), 9 deletions(-)

diff --git a/Annex.hs b/Annex.hs
index f2fb1f4019..d1509d4bd2 100644
--- a/Annex.hs
+++ b/Annex.hs
@@ -18,6 +18,7 @@ module Annex (
 	changeState,
 	setFlag,
 	setField,
+	setOutput,
 	getFlag,
 	getField,
 	addCleanup,
@@ -148,6 +149,11 @@ addCleanup :: String -> Annex () -> Annex ()
 addCleanup uid a = changeState $ \s ->
 	s { cleanup = M.insertWith' const uid a $ cleanup s }
 
+{- Sets the type of output to emit. -}
+setOutput :: OutputType -> Annex ()
+setOutput o = changeState $ \s ->
+	s { output = (output s) { outputType = o } }
+
 {- Checks if a flag was set. -}
 getFlag :: String -> Annex Bool
 getFlag flag = fromMaybe False . M.lookup flag <$> getState flags
diff --git a/Option.hs b/Option.hs
index 7696220554..1bac2cd050 100644
--- a/Option.hs
+++ b/Option.hs
@@ -32,11 +32,11 @@ common =
 		"avoid slow operations"
 	, Option ['a'] ["auto"] (NoArg (setauto True))
 		"automatic mode"
-	, Option ['q'] ["quiet"] (NoArg (setoutput QuietOutput))
+	, Option ['q'] ["quiet"] (NoArg (Annex.setOutput QuietOutput))
 		"avoid verbose output"
-	, Option ['v'] ["verbose"] (NoArg (setoutput NormalOutput))
+	, Option ['v'] ["verbose"] (NoArg (Annex.setOutput NormalOutput))
 		"allow verbose output (default)"
-	, Option ['j'] ["json"] (NoArg (setoutput JSONOutput))
+	, Option ['j'] ["json"] (NoArg (Annex.setOutput JSONOutput))
 		"enable JSON output"
 	, Option ['d'] ["debug"] (NoArg setdebug)
 		"show debug messages"
@@ -47,8 +47,6 @@ common =
 		setforce v = Annex.changeState $ \s -> s { Annex.force = v }
 		setfast v = Annex.changeState $ \s -> s { Annex.fast = v }
 		setauto v = Annex.changeState $ \s -> s { Annex.auto = v }
-		setoutput v = Annex.changeState $ \s ->
-			s { Annex.output = (Annex.output s) { outputType = v } }
 		setforcebackend v = Annex.changeState $ \s -> s { Annex.forcebackend = Just v }
 		setdebug = liftIO $ updateGlobalLogger rootLoggerName $
 			setLevel DEBUG
diff --git a/test.hs b/test.hs
index 5f4b0ad40e..8c24c3a23d 100644
--- a/test.hs
+++ b/test.hs
@@ -38,6 +38,7 @@ import qualified Logs.Remote
 import qualified Remote
 import qualified Command.DropUnused
 import qualified Types.Key
+import qualified Types.Messages
 import qualified Config
 import qualified Crypto
 import qualified Utility.Path
@@ -720,10 +721,10 @@ git_annex_expectoutput command params expected = do
 -- are not run; this should only be used for actions that query state.
 annexeval :: Types.Annex a -> IO a
 annexeval a = do
-	g <- Git.Construct.fromCurrent
-	g' <- Git.Config.read g
-	s <- Annex.new g'
-	Annex.eval s { Annex.output = Annex.QuietOutput } a
+	s <- Annex.new =<< Git.Config.read =<< Git.Construct.fromCurrent
+	Annex.eval s $ do
+		Annex.setOutput Types.Messages.QuietOutput
+		a
 
 innewrepo :: Assertion -> Assertion
 innewrepo a = withgitrepo $ \r -> indir r a

From 76b80d6af08f3e758e2180813392860578ff9756 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 30 Apr 2012 13:59:28 -0400
Subject: [PATCH 086/220] releasing version 3.20120430

---
 debian/changelog | 4 ++--
 git-annex.cabal  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 7a9b5c6eab..27a6bab2b8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,4 @@
-git-annex (3.20120419) UNRELEASED; urgency=low
+git-annex (3.20120430) unstable; urgency=low
 
   * Fix use of annex.diskreserve config setting.
   * Directory special remotes now check annex.diskreserve.
@@ -11,7 +11,7 @@ git-annex (3.20120419) UNRELEASED; urgency=low
     avoids gpg key distribution, at the expense of flexability, and with
     the requirement that all clones of the git repository be equally trusted.
 
- -- Joey Hess   Fri, 20 Apr 2012 16:14:08 -0400
+ -- Joey Hess   Mon, 30 Apr 2012 13:16:10 -0400
 
 git-annex (3.20120418) unstable; urgency=low
 
diff --git a/git-annex.cabal b/git-annex.cabal
index da0e9f4de7..6299944698 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -1,5 +1,5 @@
 Name: git-annex
-Version: 3.20120418
+Version: 3.20120430
 Cabal-Version: >= 1.8
 License: GPL
 Maintainer: Joey Hess 

From f377cbc12908c173ba1fde782cae3aa80b9dffef Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 30 Apr 2012 14:10:10 -0400
Subject: [PATCH 087/220] update

---
 .gitignore | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)

diff --git a/.gitignore b/.gitignore
index fe65fc5747..b6b8d606d6 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,12 +1,8 @@
 tmp
-*.hi
-*.o
 test
 configure
 Build/SysConfig.hs
 git-annex
-git-annex-shell
-git-union-merge
 git-annex.1
 git-annex-shell.1
 git-union-merge.1
@@ -15,5 +11,5 @@ html
 *.tix
 .hpc
 Utility/Touch.hs
-Utility/StatFS.hs
+Utility/libdiskfree.o
 dist

From 1f5965fa0d5a869568aa4579bb996b6b04a236f8 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 30 Apr 2012 14:34:08 -0400
Subject: [PATCH 088/220] add news item for git-annex 3.20120430

---
 doc/news/version_3.20120309.mdwn |  5 -----
 doc/news/version_3.20120430.mdwn | 12 ++++++++++++
 2 files changed, 12 insertions(+), 5 deletions(-)
 delete mode 100644 doc/news/version_3.20120309.mdwn
 create mode 100644 doc/news/version_3.20120430.mdwn

diff --git a/doc/news/version_3.20120309.mdwn b/doc/news/version_3.20120309.mdwn
deleted file mode 100644
index 869b96ccec..0000000000
--- a/doc/news/version_3.20120309.mdwn
+++ /dev/null
@@ -1,5 +0,0 @@
-git-annex 3.20120309 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Fix key directory hash calculation code to behave as it did before
-     version 3.20120227 when a key contains non-ascii characters (only
-     WORM backend is likely to have been affected)."""]]
\ No newline at end of file
diff --git a/doc/news/version_3.20120430.mdwn b/doc/news/version_3.20120430.mdwn
new file mode 100644
index 0000000000..07e1e85ad2
--- /dev/null
+++ b/doc/news/version_3.20120430.mdwn
@@ -0,0 +1,12 @@
+git-annex 3.20120430 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Fix use of annex.diskreserve config setting.
+   * Directory special remotes now check annex.diskreserve.
+   * Support git's core.sharedRepository configuration.
+   * Add annex.http-headers and annex.http-headers-command config
+     settings, to allow custom headers to be sent with all HTTP requests.
+     (Requested by the Internet Archive)
+   * uninit: Clear annex.uuid from .git/config. Closes: #[670639](http://bugs.debian.org/670639)
+   * Added shared cipher mode to encryptable special remotes. This option
+     avoids gpg key distribution, at the expense of flexability, and with
+     the requirement that all clones of the git repository be equally trusted."""]]
\ No newline at end of file

From 153912633e3aa1f0f4e5413e1786ea9b5770715f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Wed, 2 May 2012 00:27:52 +0000
Subject: [PATCH 089/220]

---
 doc/forum/Windows_support.mdwn | 6 ++++++
 1 file changed, 6 insertions(+)
 create mode 100644 doc/forum/Windows_support.mdwn

diff --git a/doc/forum/Windows_support.mdwn b/doc/forum/Windows_support.mdwn
new file mode 100644
index 0000000000..0e9e8dcb6e
--- /dev/null
+++ b/doc/forum/Windows_support.mdwn
@@ -0,0 +1,6 @@
+Hi,
+
+Do you have any news about Windows support?
+Is this something you're currently working on?
+
+Thanks!

From 8f8f7ecf6fed3afa189225d961ba6c232745be93 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 1 May 2012 20:34:28 -0400
Subject: [PATCH 090/220] recycled comment

---
 .../comment_1_23fa9aa3b00940a1c1b3876c35eef019._comment  | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 doc/forum/Windows_support/comment_1_23fa9aa3b00940a1c1b3876c35eef019._comment

diff --git a/doc/forum/Windows_support/comment_1_23fa9aa3b00940a1c1b3876c35eef019._comment b/doc/forum/Windows_support/comment_1_23fa9aa3b00940a1c1b3876c35eef019._comment
new file mode 100644
index 0000000000..95323ff997
--- /dev/null
+++ b/doc/forum/Windows_support/comment_1_23fa9aa3b00940a1c1b3876c35eef019._comment
@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="http://joey.kitenet.net/"
+ nickname="joey"
+ subject="comment 1"
+ date="2012-03-12T06:43:02Z"
+ content="""
+[[todo/windows_support]] has everything I know about making a windows port. This badly needs someone who understand Windows to dive into it. The question of how to create a symbolic link (or the relevant Windows equivilant) from haskell on Windows
+is a good starting point..
+"""]]

From 834e73cde5f1669384641a8df3b8c542e240db29 Mon Sep 17 00:00:00 2001
From: "http://id.wgnr.me/" 
Date: Wed, 2 May 2012 14:09:28 +0000
Subject: [PATCH 091/220]

---
 ...___versions_3.20120315_and_3.20120430.mdwn | 49 +++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn

diff --git a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
new file mode 100644
index 0000000000..d8c15ed260
--- /dev/null
+++ b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
@@ -0,0 +1,49 @@
+What steps will reproduce the problem?
+
+	$ git annex initremote rsyncremote type=rsync rsyncurl=myuser@rsync.hidrive.strato.com:/users/myuser/git-annex/Music/ encryption=0xC597DECC177AFD7C
+	$ git annex get --from rsyncremote "file"
+
+What is the expected output? What do you see instead?
+
+I expect that the requested file is copied as for every other remote, but instead I get this error:
+
+----------------------------------------
+	get  (from rsyncremote...) (gpg) 
+	rsync: change_dir "/users/myuser/git-annex/Music/0e5/a5b/'GPGHMACSHA1--3afd32ab8e70ac329262adeb770c330b0845b1e0" failed: No such file or directory (2)
+
+	sent 8 bytes  received 10 bytes  7.20 bytes/sec
+	total size is 0  speedup is 0.00
+	rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1518) [Receiver=3.0.9]
+
+	  rsync failed -- run git annex again to resume file transfer
+
+	rsync: change_dir "/users/myuser/git-annex/Music/8k/QZ/'GPGHMACSHA1--3afd32ab8e70ac329262adeb770c330b0845b1e0" failed: No such file or directory (2)
+
+	sent 8 bytes  received 10 bytes  36.00 bytes/sec
+	total size is 0  speedup is 0.00
+	rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1518) [Receiver=3.0.9]
+
+	  rsync failed -- run git annex again to resume file transfer
+failed
+	git-annex: get: 1 failed
+----------------------------------------
+
+I can verify that the directory /users/myuser/git-annex/Music/0e5/a5b/GPGHMACSHA1--3afd32ab8e70ac329262adeb770c330b0845b1e0 exists in the rsync remote, without the ' character.
+
+What version of git-annex are you using? On what operating system?
+
+I tried versions 3.20120315 and 3.20120430 on Gentoo linux.
+
+	$ uname -a
+	Linux odin 3.3.1-gentoo-odin #1 SMP Sat Apr 7 21:18:11 CEST 2012 x86_64 Intel(R) Core(TM) i5 CPU M 560 @ 2.67GHz GenuineIntel GNU/Linux
+
+	$ ghc --version
+	The Glorious Glasgow Haskell Compilation System, version 7.4.1
+
+Please provide any additional information below.
+
+The rsync remote config in .git/config:
+
+	[remote "rsyncremote"]
+		annex-rsyncurl = myuser@rsync.hidrive.strato.com:/users/myuser/git-annex/Music/
+		annex-uuid = "UUID"

From 5337c4e0c4e9f7a803a77532057a20f7c54fe28a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 11:38:27 -0400
Subject: [PATCH 092/220] rsync protocol?

---
 Utility/RsyncFile.hs                                |  2 +-
 ...te__44___versions_3.20120315_and_3.20120430.mdwn | 13 +++++++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/Utility/RsyncFile.hs b/Utility/RsyncFile.hs
index a691d0a0e6..fae23d4ceb 100644
--- a/Utility/RsyncFile.hs
+++ b/Utility/RsyncFile.hs
@@ -58,7 +58,7 @@ rsyncUrlIsShell s
 	| "rsync://" `isPrefixOf` s = False
 	| otherwise = go s
 	where
-		-- host:dir is rsync protocol, while host:dir is ssh/rsh
+		-- host:dir is rsync protocol, while host/dir is ssh/rsh
 		go [] = False
 		go (c:cs)
 			| c == '/' = False -- got to directory with no colon
diff --git a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
index d8c15ed260..ad97a82b2c 100644
--- a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
+++ b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
@@ -47,3 +47,16 @@ The rsync remote config in .git/config:
 	[remote "rsyncremote"]
 		annex-rsyncurl = myuser@rsync.hidrive.strato.com:/users/myuser/git-annex/Music/
 		annex-uuid = "UUID"
+
+> Here's what the --debug flag shows is being run:
+
+	Running: rsync ["--progress","--inplace","joey@localhost:/tmp/Music/d98/a3c/'GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c/GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c'","/home/joey/tmp/x/.git/annex/tmp/GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c"] 
+
+> But, this works for me, here, despite containing the quoting!
+> That's because here it's using rsync over ssh, which actually requires
+> that quoting. Are you using rsync
+> over the rsync protocol? If so, the workaround is to explicitly make
+> the rsyncurl start with `rsync://` -- and if this is the case, I need
+> to adjust the code in git-annex that determines if it's using ssh or
+> the rsync protocol.
+>  --[[Joey]]

From 913775410bb1b30116c3550c43389747c3b0cf14 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 11:43:30 -0400
Subject: [PATCH 093/220] update

---
 Utility/RsyncFile.hs                                      | 2 +-
 ...c_remote__44___versions_3.20120315_and_3.20120430.mdwn | 8 ++++++--
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/Utility/RsyncFile.hs b/Utility/RsyncFile.hs
index fae23d4ceb..db9057843c 100644
--- a/Utility/RsyncFile.hs
+++ b/Utility/RsyncFile.hs
@@ -58,7 +58,7 @@ rsyncUrlIsShell s
 	| "rsync://" `isPrefixOf` s = False
 	| otherwise = go s
 	where
-		-- host:dir is rsync protocol, while host/dir is ssh/rsh
+		-- host::dir is rsync protocol, while host:dir is ssh/rsh
 		go [] = False
 		go (c:cs)
 			| c == '/' = False -- got to directory with no colon
diff --git a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
index ad97a82b2c..c57c645a01 100644
--- a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
+++ b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
@@ -56,7 +56,11 @@ The rsync remote config in .git/config:
 > That's because here it's using rsync over ssh, which actually requires
 > that quoting. Are you using rsync
 > over the rsync protocol? If so, the workaround is to explicitly make
-> the rsyncurl start with `rsync://` -- and if this is the case, I need
+> the rsyncurl start with `rsync://`
+> 
+> And if this is the case, I need
 > to adjust the code in git-annex that determines if it's using ssh or
-> the rsync protocol.
+> the rsync protocol. It assumes that (and this is what the rsync man 
+> says AFAICS) that the rsync protocol is only used if the url starts
+> with `rsync://` or contains `::`.
 >  --[[Joey]]

From 17fd57bd818931532eda1892f23e0998ad0110d5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 11:50:57 -0400
Subject: [PATCH 094/220] hidrive.strato.com

---
 ...emote__44___versions_3.20120315_and_3.20120430.mdwn | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
index c57c645a01..ab5eb78435 100644
--- a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
+++ b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
@@ -48,7 +48,7 @@ The rsync remote config in .git/config:
 		annex-rsyncurl = myuser@rsync.hidrive.strato.com:/users/myuser/git-annex/Music/
 		annex-uuid = "UUID"
 
-> Here's what the --debug flag shows is being run:
+> Here's what the --debug flag shows is being run: --[[Joey]] 
 
 	Running: rsync ["--progress","--inplace","joey@localhost:/tmp/Music/d98/a3c/'GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c/GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c'","/home/joey/tmp/x/.git/annex/tmp/GPGHMACSHA1--878c3a3f59965bd87b4738ab29562efd215b954c"] 
 
@@ -63,4 +63,12 @@ The rsync remote config in .git/config:
 > the rsync protocol. It assumes that (and this is what the rsync man 
 > says AFAICS) that the rsync protocol is only used if the url starts
 > with `rsync://` or contains `::`.
+> 
+> Hmm, I see that `hidrive.strato.com` is some kind of rsync provider?
+> Perhaps they do something with rsync over ssh that
+> avoids the need for shell quoting. For example, they might pass incoming
+> ssh connections directly into rsync, bypassing the shell
+> -- which avoids the need for this quoting. Any details you can provide
+> about them would probably be useful then. Ie, do they really use rsync
+> over ssh, is it really a `rsync.net` type rsync provider? 
 >  --[[Joey]]

From 6d61067599142af3eb520527589b0adfbf45867d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:08:31 -0400
Subject: [PATCH 095/220] rsync shellescape disable option

Rsync special remotes can be configured with shellescape=no to avoid shell
quoting that is normally done when using rsync over ssh. This is known to
be needed for certian rsync hosting providers (specificially
hidrive.strato.com) that use rsync over ssh but do not pass it through the
shell.
---
 Remote/Rsync.hs                | 21 ++++++++++++---------
 debian/changelog               | 10 ++++++++++
 doc/special_remotes/rsync.mdwn |  8 ++++++++
 3 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/Remote/Rsync.hs b/Remote/Rsync.hs
index 571cd8f5ee..60cbf4595f 100644
--- a/Remote/Rsync.hs
+++ b/Remote/Rsync.hs
@@ -22,9 +22,10 @@ import Utility.RsyncFile
 
 type RsyncUrl = String
 
-data RsyncOpts = RsyncOpts {
-	rsyncUrl :: RsyncUrl,
-	rsyncOptions :: [CommandParam]
+data RsyncOpts = RsyncOpts
+	{ rsyncUrl :: RsyncUrl
+	, rsyncOptions :: [CommandParam]
+	, rsyncShellEscape :: Bool
 }
 
 remote :: RemoteType
@@ -37,7 +38,7 @@ remote = RemoteType {
 
 gen :: Git.Repo -> UUID -> Maybe RemoteConfig -> Annex Remote
 gen r u c = do
-	o <- genRsyncOpts r
+	o <- genRsyncOpts r c
 	cst <- remoteCost r expensiveRemoteCost
 	return $ encryptableRemote c
 		(storeEncrypted o)
@@ -58,11 +59,13 @@ gen r u c = do
 			remotetype = remote
 		}
 
-genRsyncOpts :: Git.Repo -> Annex RsyncOpts
-genRsyncOpts r = do
+genRsyncOpts :: Git.Repo -> Maybe RemoteConfig -> Annex RsyncOpts
+genRsyncOpts r c = do
 	url <- getRemoteConfig r "rsyncurl" (error "missing rsyncurl")
-	opts <- getRemoteConfig r "rsync-options" ""
-	return $ RsyncOpts url $ map Param $ filter safe $ words opts
+	opts <- map Param . filter safe . words
+		<$> getRemoteConfig r "rsync-options" ""
+	let escape = maybe True (\m -> M.lookup "shellescape" m /= Just "no") c
+	return $ RsyncOpts url opts escape
 	where
 		safe o
 			-- Don't allow user to pass --delete to rsync;
@@ -86,7 +89,7 @@ rsyncSetup u c = do
 
 rsyncEscape :: RsyncOpts -> String -> String
 rsyncEscape o s
-	| rsyncUrlIsShell (rsyncUrl o) = shellEscape s
+	| rsyncShellEscape o && rsyncUrlIsShell (rsyncUrl o) = shellEscape s
 	| otherwise = s
 
 rsyncUrls :: RsyncOpts -> Key -> [String]
diff --git a/debian/changelog b/debian/changelog
index 27a6bab2b8..033b739de5 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,13 @@
+git-annex (3.20120431) UNRELEASED; urgency=low
+
+  * Rsync special remotes can be configured with shellescape=no
+    to avoid shell quoting that is normally done when using rsync over ssh.
+    This is known to be needed for certian rsync hosting providers
+    (specificially hidrive.strato.com) that use rsync over ssh but do not
+    pass it through the shell.
+
+ -- Joey Hess   Wed, 02 May 2012 13:06:18 -0400
+
 git-annex (3.20120430) unstable; urgency=low
 
   * Fix use of annex.diskreserve config setting.
diff --git a/doc/special_remotes/rsync.mdwn b/doc/special_remotes/rsync.mdwn
index 90d544a1e1..286615460c 100644
--- a/doc/special_remotes/rsync.mdwn
+++ b/doc/special_remotes/rsync.mdwn
@@ -24,5 +24,13 @@ These parameters can be passed to `git annex initremote` to configure rsync:
 * `rsyncurl` - Required. This is the url or `hostname:/directory` to 
   pass to rsync to tell it where to store content.
 
+* `shellescape` - Optional. Set to "no" to avoid shell escaping normally
+  done when using rsync over ssh. That escaping is needed with typical
+  setups, but not with some hosting providers that do not expose rsynced
+  filenames to the shell. You'll know you need this option if `git annex get`
+  from the special remote fails with an error message containing a single
+  quote (`'`) character. If that happens, you can re-run initremote
+  setting shellescape=no.
+
 The `annex-rsync-options` git configuration setting can be used to pass
 parameters to rsync.

From 8f453004791217e23268728129c197e49c19f67b Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:15:19 -0400
Subject: [PATCH 096/220] dropunused: Allow specifying ranges to drop.

Sort of by popular demand, but the last straw for not using seq
was that it can run into command line length limits.
---
 Command/DropUnused.hs | 40 +++++++++++++++++++++++++++-------------
 debian/changelog      |  1 +
 doc/git-annex.mdwn    |  2 ++
 3 files changed, 30 insertions(+), 13 deletions(-)

diff --git a/Command/DropUnused.hs b/Command/DropUnused.hs
index 0b2a602161..9c9513ca9d 100644
--- a/Command/DropUnused.hs
+++ b/Command/DropUnused.hs
@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2010 Joey Hess 
+ - Copyright 2010,2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -18,7 +18,7 @@ import qualified Git
 import qualified Option
 import Types.Key
 
-type UnusedMap = M.Map String Key
+type UnusedMap = M.Map Integer Key
 
 def :: [Command]
 def = [withOptions [Command.Drop.fromOption] $
@@ -34,10 +34,20 @@ withUnusedMaps params = do
 	unused <- readUnusedLog ""
 	unusedbad <- readUnusedLog "bad"
 	unusedtmp <- readUnusedLog "tmp"
-	return $ map (start (unused, unusedbad, unusedtmp)) params
+	return $ map (start (unused, unusedbad, unusedtmp)) $
+		concatMap unusedSpec params
 
-start :: (UnusedMap, UnusedMap, UnusedMap) -> FilePath -> CommandStart
-start (unused, unusedbad, unusedtmp) s = search
+unusedSpec :: String -> [Integer]
+unusedSpec spec
+	| "-" `isInfixOf` spec = range $ separate (== '-') spec
+	| otherwise = catMaybes [readish spec]
+	where
+		range (a, b) = case (readish a, readish b) of
+			(Just x, Just y) -> [x..y]
+			_ -> []
+
+start :: (UnusedMap, UnusedMap, UnusedMap) -> Integer -> CommandStart
+start (unused, unusedbad, unusedtmp) n = search
 	[ (unused, perform)
 	, (unusedbad, performOther gitAnnexBadLocation)
 	, (unusedtmp, performOther gitAnnexTmpLocation)
@@ -45,10 +55,10 @@ start (unused, unusedbad, unusedtmp) s = search
 	where
 		search [] = stop
 		search ((m, a):rest) =
-			case M.lookup s m of
+			case M.lookup n m of
 				Nothing -> search rest
 				Just key -> do
-					showStart "dropunused" s
+					showStart "dropunused" (show n)
 					next $ a key
 
 perform :: Key -> CommandPerform
@@ -70,11 +80,15 @@ performOther filespec key = do
 readUnusedLog :: FilePath -> Annex UnusedMap
 readUnusedLog prefix = do
 	f <- fromRepo $ gitAnnexUnusedLog prefix
-	e <- liftIO $ doesFileExist f
-	if e
-		then M.fromList . map parse . lines <$> liftIO (readFile f)
-		else return M.empty
+	ifM (liftIO $ doesFileExist f)
+		( M.fromList . catMaybes . map parse . lines
+			<$> liftIO (readFile f)
+		, return M.empty
+		)
 	where
-		parse line = (num, fromJust $ readKey rest)
+		parse line =
+			case (readish tag, readKey rest) of
+				(Just num, Just key) -> Just (num, key)
+				_ -> Nothing
 			where
-				(num, rest) = separate (== ' ') line
+				(tag, rest) = separate (== ' ') line
diff --git a/debian/changelog b/debian/changelog
index 033b739de5..72bce85511 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -5,6 +5,7 @@ git-annex (3.20120431) UNRELEASED; urgency=low
     This is known to be needed for certian rsync hosting providers
     (specificially hidrive.strato.com) that use rsync over ssh but do not
     pass it through the shell.
+  * dropunused: Allow specifying ranges to drop.
 
  -- Joey Hess   Wed, 02 May 2012 13:06:18 -0400
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 098d520010..998e1fa266 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -240,6 +240,8 @@ subdirectories).
   Drops the data corresponding to the numbers, as listed by the last
   `git annex unused`
 
+  You can also specify ranges of numbers, such as "1-1000".
+
   To drop the data from a remote, specify --from.
 
 * merge

From c058cdf46165af3ede7f09155ce99dc2bde9a17f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:17:51 -0400
Subject: [PATCH 097/220] move

---
 doc/{ => install}/NixOS.mdwn | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
 rename doc/{ => install}/NixOS.mdwn (67%)

diff --git a/doc/NixOS.mdwn b/doc/install/NixOS.mdwn
similarity index 67%
rename from doc/NixOS.mdwn
rename to doc/install/NixOS.mdwn
index 864184a23c..115f9fa532 100644
--- a/doc/NixOS.mdwn
+++ b/doc/install/NixOS.mdwn
@@ -2,4 +2,5 @@ Users of the [Nix package manager](http://nixos.org/) can install it by running:
 
     nix-env -i git-annex
 
-The build status of the package within Nix can be seen on the [Hydra Build Farm](http://hydra.nixos.org/job/nixpkgs/trunk/gitAndTools.gitAnnex).
+The build status of the package within Nix can be seen on the [Hydra Build
+Farm](http://hydra.nixos.org/job/nixpkgs/trunk/gitAndTools.gitAnnex).

From ea0c48bfcd1d376a2f25573cca8c7876671dfd2f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:18:21 -0400
Subject: [PATCH 098/220] gentoo

---
 doc/install.mdwn        | 1 +
 doc/install/Gentoo.mdwn | 3 +++
 2 files changed, 4 insertions(+)
 create mode 100644 doc/install/Gentoo.mdwn

diff --git a/doc/install.mdwn b/doc/install.mdwn
index 2590060e69..fe0522aa05 100644
--- a/doc/install.mdwn
+++ b/doc/install.mdwn
@@ -8,6 +8,7 @@
 * [[openSUSE]]
 * [[ArchLinux]]
 * [[NixOS]]
+* [[Gentoo]]
 * Windows: [[sorry, not possible yet|todo/windows_support]]
 
 ## Using cabal
diff --git a/doc/install/Gentoo.mdwn b/doc/install/Gentoo.mdwn
new file mode 100644
index 0000000000..b2061ad7a1
--- /dev/null
+++ b/doc/install/Gentoo.mdwn
@@ -0,0 +1,3 @@
+Gentoo users can:
+
+	`emerge git-annex`

From d8606a327d5b7dc34708405634baab1aade3c96c Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:20:21 -0400
Subject: [PATCH 099/220] portage overlay

---
 doc/install/Gentoo.mdwn | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/install/Gentoo.mdwn b/doc/install/Gentoo.mdwn
index b2061ad7a1..c508309c6e 100644
--- a/doc/install/Gentoo.mdwn
+++ b/doc/install/Gentoo.mdwn
@@ -1,3 +1,5 @@
 Gentoo users can:
 
 	`emerge git-annex`
+
+A possibly more up-to-date version is in the haskell portage overlay.

From 7d6b36dffbb11837a6fcfea3317b7d24ccbeeff7 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 13:22:04 -0400
Subject: [PATCH 100/220] deseq

---
 ...dd_range_argument_to___34__git_annex_dropunused__34___.mdwn | 3 +++
 doc/walkthrough/unused_data.mdwn                               | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/doc/bugs/add_range_argument_to___34__git_annex_dropunused__34___.mdwn b/doc/bugs/add_range_argument_to___34__git_annex_dropunused__34___.mdwn
index bbe6007a87..471a698a0e 100644
--- a/doc/bugs/add_range_argument_to___34__git_annex_dropunused__34___.mdwn
+++ b/doc/bugs/add_range_argument_to___34__git_annex_dropunused__34___.mdwn
@@ -16,3 +16,6 @@ I work around this lack as I want to drop all unused files anyway by something l
 
 > I don't see adding my own range operations to be an improvement worth
 > making; it'd arguably only be a complication. --[[Joey]] [[done]]
+
+>> Actually, this did get implemented, since using seq could fall afoul 
+>> of command-line length limits in extreme cases.
diff --git a/doc/walkthrough/unused_data.mdwn b/doc/walkthrough/unused_data.mdwn
index 518550ac02..63fb9f66d9 100644
--- a/doc/walkthrough/unused_data.mdwn
+++ b/doc/walkthrough/unused_data.mdwn
@@ -27,4 +27,4 @@ data anymore, you can easily remove it:
 
 Hint: To drop a lot of unused data, use a command like this:
 
-	# git annex dropunused `seq 1 1000`
+	# git annex dropunused 1-1000

From 392931eca9191117ae5c9d479fabab1e8ecaf8df Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 14:59:05 -0400
Subject: [PATCH 101/220] addunused: New command, the opposite of dropunused,
 it relinks unused content into the git repository.

---
 Command/AddUnused.hs  | 34 ++++++++++++++++
 Command/DropUnused.hs | 60 +++-------------------------
 Command/Unused.hs     | 10 +----
 GitAnnex.hs           |  2 +
 Logs/Unused.hs        | 91 +++++++++++++++++++++++++++++++++++++++++++
 Usage.hs              |  2 +
 debian/changelog      |  2 +
 doc/git-annex.mdwn    |  8 +++-
 8 files changed, 145 insertions(+), 64 deletions(-)
 create mode 100644 Command/AddUnused.hs
 create mode 100644 Logs/Unused.hs

diff --git a/Command/AddUnused.hs b/Command/AddUnused.hs
new file mode 100644
index 0000000000..c498216dc3
--- /dev/null
+++ b/Command/AddUnused.hs
@@ -0,0 +1,34 @@
+{- git-annex command
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Command.AddUnused where
+
+import Common.Annex
+import Logs.Unused
+import Command
+import qualified Command.Add
+
+def :: [Command]
+def = [command "addunused" (paramRepeating paramNumRange)
+	seek "add back unused files"]
+
+seek :: [CommandSeek]
+seek = [withUnusedMaps start]
+
+start :: UnusedMaps -> Int -> CommandStart
+start = startUnused "addunused" perform (performOther "bad") (performOther "tmp")
+
+perform :: Key -> CommandPerform
+perform key = next $ Command.Add.cleanup file key True
+	where
+		file = "unused." ++ show key
+
+{- The content is not in the annex, but in another directory, and
+ - it seems better to error out, rather than moving bad/tmp content into
+ - the annex. -}
+performOther :: String -> Key -> CommandPerform
+performOther other _ = error $ "cannot addunused " ++ other ++ "content"
diff --git a/Command/DropUnused.hs b/Command/DropUnused.hs
index 9c9513ca9d..a94c2873dd 100644
--- a/Command/DropUnused.hs
+++ b/Command/DropUnused.hs
@@ -7,8 +7,7 @@
 
 module Command.DropUnused where
 
-import qualified Data.Map as M
-
+import Logs.Unused
 import Common.Annex
 import Command
 import qualified Annex
@@ -16,50 +15,17 @@ import qualified Command.Drop
 import qualified Remote
 import qualified Git
 import qualified Option
-import Types.Key
-
-type UnusedMap = M.Map Integer Key
 
 def :: [Command]
 def = [withOptions [Command.Drop.fromOption] $
-	command "dropunused" (paramRepeating paramNumber)
+	command "dropunused" (paramRepeating paramNumRange)
 		seek "drop unused file content"]
 
 seek :: [CommandSeek]
-seek = [withUnusedMaps]
+seek = [withUnusedMaps start]
 
-{- Read unused logs once, and pass the maps to each start action. -}
-withUnusedMaps :: CommandSeek
-withUnusedMaps params = do
-	unused <- readUnusedLog ""
-	unusedbad <- readUnusedLog "bad"
-	unusedtmp <- readUnusedLog "tmp"
-	return $ map (start (unused, unusedbad, unusedtmp)) $
-		concatMap unusedSpec params
-
-unusedSpec :: String -> [Integer]
-unusedSpec spec
-	| "-" `isInfixOf` spec = range $ separate (== '-') spec
-	| otherwise = catMaybes [readish spec]
-	where
-		range (a, b) = case (readish a, readish b) of
-			(Just x, Just y) -> [x..y]
-			_ -> []
-
-start :: (UnusedMap, UnusedMap, UnusedMap) -> Integer -> CommandStart
-start (unused, unusedbad, unusedtmp) n = search
-	[ (unused, perform)
-	, (unusedbad, performOther gitAnnexBadLocation)
-	, (unusedtmp, performOther gitAnnexTmpLocation)
-	]
-	where
-		search [] = stop
-		search ((m, a):rest) =
-			case M.lookup n m of
-				Nothing -> search rest
-				Just key -> do
-					showStart "dropunused" (show n)
-					next $ a key
+start :: UnusedMaps -> Int -> CommandStart
+start = startUnused "dropunused" perform (performOther gitAnnexBadLocation) (performOther gitAnnexTmpLocation)
 
 perform :: Key -> CommandPerform
 perform key = maybe droplocal dropremote =<< Remote.byName =<< from
@@ -76,19 +42,3 @@ performOther filespec key = do
 	f <- fromRepo $ filespec key
 	liftIO $ whenM (doesFileExist f) $ removeFile f
 	next $ return True
-
-readUnusedLog :: FilePath -> Annex UnusedMap
-readUnusedLog prefix = do
-	f <- fromRepo $ gitAnnexUnusedLog prefix
-	ifM (liftIO $ doesFileExist f)
-		( M.fromList . catMaybes . map parse . lines
-			<$> liftIO (readFile f)
-		, return M.empty
-		)
-	where
-		parse line =
-			case (readish tag, readKey rest) of
-				(Just num, Just key) -> Just (num, key)
-				_ -> Nothing
-			where
-				(tag, rest) = separate (== ' ') line
diff --git a/Command/Unused.hs b/Command/Unused.hs
index 5bdadcf44a..6b319ee72c 100644
--- a/Command/Unused.hs
+++ b/Command/Unused.hs
@@ -19,9 +19,9 @@ import Control.Monad.ST
 
 import Common.Annex
 import Command
+import Logs.Unused
 import Annex.Content
 import Utility.FileMode
-import Utility.TempFile
 import Logs.Location
 import Config
 import qualified Annex
@@ -91,19 +91,13 @@ check file msg a c = do
 	l <- a
 	let unusedlist = number c l
 	unless (null l) $ showLongNote $ msg unusedlist
-	writeUnusedFile file unusedlist
+	writeUnusedLog file unusedlist
 	return $ c + length l
 
 number :: Int -> [a] -> [(Int, a)]
 number _ [] = []
 number n (x:xs) = (n+1, x) : number (n+1) xs
 
-writeUnusedFile :: FilePath -> [(Int, Key)] -> Annex ()
-writeUnusedFile prefix l = do
-	logfile <- fromRepo $ gitAnnexUnusedLog prefix
-	liftIO $ viaTmp writeFile logfile $
-		unlines $ map (\(n, k) -> show n ++ " " ++ show k) l
-
 table :: [(Int, Key)] -> [String]
 table l = "  NUMBER  KEY" : map cols l
 	where
diff --git a/GitAnnex.hs b/GitAnnex.hs
index 52886c3087..0e707b1868 100644
--- a/GitAnnex.hs
+++ b/GitAnnex.hs
@@ -37,6 +37,7 @@ import qualified Command.InitRemote
 import qualified Command.Fsck
 import qualified Command.Unused
 import qualified Command.DropUnused
+import qualified Command.AddUnused
 import qualified Command.Unlock
 import qualified Command.Lock
 import qualified Command.PreCommit
@@ -86,6 +87,7 @@ cmds = concat
 	, Command.Fsck.def
 	, Command.Unused.def
 	, Command.DropUnused.def
+	, Command.AddUnused.def
 	, Command.Find.def
 	, Command.Whereis.def
 	, Command.Log.def
diff --git a/Logs/Unused.hs b/Logs/Unused.hs
new file mode 100644
index 0000000000..7d240cfe33
--- /dev/null
+++ b/Logs/Unused.hs
@@ -0,0 +1,91 @@
+{- git-annex unused log file
+ -
+ - Copyright 2010,2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Logs.Unused (
+	UnusedMap,
+	UnusedMaps(..),
+	writeUnusedLog,
+	readUnusedLog,
+	withUnusedMaps,
+	startUnused,
+) where
+
+import qualified Data.Map as M
+
+import Common.Annex
+import Command
+import Types.Key
+import Utility.TempFile
+
+writeUnusedLog :: FilePath -> [(Int, Key)] -> Annex ()
+writeUnusedLog prefix l = do
+	logfile <- fromRepo $ gitAnnexUnusedLog prefix
+	liftIO $ viaTmp writeFile logfile $
+		unlines $ map (\(n, k) -> show n ++ " " ++ show k) l
+
+readUnusedLog :: FilePath -> Annex UnusedMap
+readUnusedLog prefix = do
+	f <- fromRepo $ gitAnnexUnusedLog prefix
+	ifM (liftIO $ doesFileExist f)
+		( M.fromList . catMaybes . map parse . lines
+			<$> liftIO (readFile f)
+		, return M.empty
+		)
+	where
+		parse line =
+			case (readish tag, readKey rest) of
+				(Just num, Just key) -> Just (num, key)
+				_ -> Nothing
+			where
+				(tag, rest) = separate (== ' ') line
+
+type UnusedMap = M.Map Int Key
+
+data UnusedMaps = UnusedMaps
+	{ unusedMap :: UnusedMap
+	, unusedBadMap :: UnusedMap
+	, unusedTmpMap :: UnusedMap
+	}
+
+{- Read unused logs once, and pass the maps to each start action. -}
+withUnusedMaps :: (UnusedMaps -> Int -> CommandStart) -> CommandSeek
+withUnusedMaps a params = do
+	unused <- readUnusedLog ""
+	unusedbad <- readUnusedLog "bad"
+	unusedtmp <- readUnusedLog "tmp"
+	return $ map (a $ UnusedMaps unused unusedbad unusedtmp) $
+		concatMap unusedSpec params
+
+unusedSpec :: String -> [Int]
+unusedSpec spec
+	| "-" `isInfixOf` spec = range $ separate (== '-') spec
+	| otherwise = catMaybes [readish spec]
+	where
+		range (a, b) = case (readish a, readish b) of
+			(Just x, Just y) -> [x..y]
+			_ -> []
+
+{- Start action for unused content. Finds the number in the maps, and
+ - calls either of 3 actions, depending on the type of unused file. -}
+startUnused :: String
+	-> (Key -> CommandPerform)
+	-> (Key -> CommandPerform) 
+	-> (Key -> CommandPerform)
+	-> UnusedMaps -> Int -> CommandStart
+startUnused message unused badunused tmpunused maps n = search
+	[ (unusedMap maps, unused)
+	, (unusedBadMap maps, badunused)
+	, (unusedTmpMap maps, tmpunused)
+	]
+	where
+		search [] = stop
+		search ((m, a):rest) =
+			case M.lookup n m of
+				Nothing -> search rest
+				Just key -> do
+					showStart message (show n)
+					next $ a key
diff --git a/Usage.hs b/Usage.hs
index b1de930ef2..e74c1490d0 100644
--- a/Usage.hs
+++ b/Usage.hs
@@ -61,6 +61,8 @@ paramUrl :: String
 paramUrl = "URL"
 paramNumber :: String
 paramNumber = "NUMBER"
+paramNumRange :: String
+paramNumRange = "NUM|RANGE"
 paramRemote :: String
 paramRemote = "REMOTE"
 paramGlob :: String
diff --git a/debian/changelog b/debian/changelog
index 72bce85511..b419b46227 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -6,6 +6,8 @@ git-annex (3.20120431) UNRELEASED; urgency=low
     (specificially hidrive.strato.com) that use rsync over ssh but do not
     pass it through the shell.
   * dropunused: Allow specifying ranges to drop.
+  * addunused: New command, the opposite of dropunused, it relinks unused
+    content into the git repository.
 
  -- Joey Hess   Wed, 02 May 2012 13:06:18 -0400
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 998e1fa266..5d41f86e9c 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -235,7 +235,7 @@ subdirectories).
 
   To check for annexed data on a remote, specify --from.
 
-* dropunused [number ...]
+* dropunused [number|range ...]
 
   Drops the data corresponding to the numbers, as listed by the last
   `git annex unused`
@@ -244,6 +244,12 @@ subdirectories).
 
   To drop the data from a remote, specify --from.
 
+* addunused [number|range ...]
+
+  Adds back files for the content corresponding to the numbers or ranges,
+  as listed by the last `git annex unused`. The files will have names
+  starting with "unused."
+
 * merge
 
   Automatically merges remote tracking branches */git-annex into

From 441b138592c243beb3627c8aab5aafb12523ad05 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 15:03:44 -0400
Subject: [PATCH 102/220] followup and close

---
 ...sync_remote__44___versions_3.20120315_and_3.20120430.mdwn | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
index ab5eb78435..85e2433829 100644
--- a/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
+++ b/doc/bugs/Error___39__get__39__ting_files_from_rsync_remote__44___versions_3.20120315_and_3.20120430.mdwn
@@ -64,6 +64,8 @@ The rsync remote config in .git/config:
 > says AFAICS) that the rsync protocol is only used if the url starts
 > with `rsync://` or contains `::`.
 > 
+>> Nope, it is indeed using rsync over ssh as git-annex thought.
+> 
 > Hmm, I see that `hidrive.strato.com` is some kind of rsync provider?
 > Perhaps they do something with rsync over ssh that
 > avoids the need for shell quoting. For example, they might pass incoming
@@ -72,3 +74,6 @@ The rsync remote config in .git/config:
 > about them would probably be useful then. Ie, do they really use rsync
 > over ssh, is it really a `rsync.net` type rsync provider? 
 >  --[[Joey]]
+> 
+>> This was the case, and the shellescape=no config option has been added
+>> to rsync special remotes to deal with it. [[done]] --[[Joey]] 

From 32de288c35b5cc728821c1c465c398f9bd8ba8d5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 2 May 2012 19:51:41 -0400
Subject: [PATCH 103/220] syntax tweaks

Although I hate to lose one of the only places I've ever used the list monad..
---
 Logs/Remote.hs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Logs/Remote.hs b/Logs/Remote.hs
index 5c9d67df03..b75573a411 100644
--- a/Logs/Remote.hs
+++ b/Logs/Remote.hs
@@ -36,7 +36,7 @@ configSet u c = do
 
 {- Map of remotes by uuid containing key/value config maps. -}
 readRemoteLog :: Annex (M.Map UUID RemoteConfig)
-readRemoteLog = (simpleMap . parseLog parseConfig) <$> Annex.Branch.get remoteLog
+readRemoteLog = simpleMap . parseLog parseConfig <$> Annex.Branch.get remoteLog
 
 parseConfig :: String -> Maybe RemoteConfig
 parseConfig = Just . keyValToConfig . words
@@ -59,7 +59,7 @@ configToKeyVal m = map toword $ sort $ M.toList m
 		toword (k, v) = k ++ "=" ++ configEscape v
 
 configEscape :: String -> String
-configEscape = (>>= escape)
+configEscape = concatMap escape
 	where
 		escape c
 			| isSpace c || c `elem` "&" = "&" ++ show (ord c) ++ ";"

From 8c09c17f6bcc1d9aaeab38eb4a6f7682139e8c65 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 4 May 2012 00:44:11 -0400
Subject: [PATCH 104/220] use strict insertWith

---
 Logs/UUIDBased.hs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Logs/UUIDBased.hs b/Logs/UUIDBased.hs
index b09d93f903..847d499237 100644
--- a/Logs/UUIDBased.hs
+++ b/Logs/UUIDBased.hs
@@ -83,7 +83,7 @@ changeLog t u v = M.insert u $ LogEntry (Date t) v
 {- Only add an LogEntry if it's newer (or at least as new as) than any
  - existing LogEntry for a UUID. -}
 addLog :: UUID -> LogEntry a -> Log a -> Log a
-addLog = M.insertWith best
+addLog = M.insertWith' best
 
 {- Converts a Log into a simple Map without the timestamp information.
  - This is a one-way trip, but useful for code that never needs to change

From 657d09d49990af85c1a91b1154a195a30438477c Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 5 May 2012 20:11:08 -0400
Subject: [PATCH 105/220] fix build

---
 test.hs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test.hs b/test.hs
index 8c24c3a23d..1952a39a9b 100644
--- a/test.hs
+++ b/test.hs
@@ -35,8 +35,8 @@ import qualified GitAnnex
 import qualified Logs.UUIDBased
 import qualified Logs.Trust
 import qualified Logs.Remote
+import qualified Logs.Unused
 import qualified Remote
-import qualified Command.DropUnused
 import qualified Types.Key
 import qualified Types.Messages
 import qualified Config
@@ -495,7 +495,7 @@ test_unused = "git-annex unused/dropunused" ~: intmpclonerepo $ do
 	where
 		checkunused expectedkeys = do
 			git_annex "unused" [] @? "unused failed"
-			unusedmap <- annexeval $ Command.DropUnused.readUnusedLog ""
+			unusedmap <- annexeval $ Logs.Unused.readUnusedLog ""
 			let unusedkeys = M.elems unusedmap
 			assertEqual "unused keys differ"
 				(sort expectedkeys) (sort unusedkeys)

From f7d8982672fd330a466c2cb22f34388e7cc429c0 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 5 May 2012 20:15:32 -0400
Subject: [PATCH 106/220] Fix use of several config settings

annex.ssh-options, annex.rsync-options, annex.bup-split-options.

And adjust types to avoid the bugs that broke several config settings
recently. Now "annex." prefixing is enforced at the type level.
---
 Annex/Content.hs  |  4 ++--
 Annex/Queue.hs    |  2 +-
 Annex/Ssh.hs      |  2 +-
 Annex/UUID.hs     | 14 +++++++-------
 Annex/Version.hs  |  4 ++--
 Backend.hs        |  2 +-
 Command/Unused.hs |  4 ++--
 Config.hs         | 43 ++++++++++++++++++++++++++++---------------
 Remote/Hook.hs    |  4 ++--
 debian/changelog  |  3 +++
 10 files changed, 49 insertions(+), 33 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index b5754e15be..2142d1f09d 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -304,12 +304,12 @@ saveState oneshot = doSideAction $ do
 			( Annex.Branch.commit "update" , Annex.Branch.stage)
 	where
 		alwayscommit = fromMaybe True . Git.configTrue
-			<$> getConfig "annex.alwayscommit" ""
+			<$> getConfig (annexConfig "alwayscommit") ""
 
 {- Downloads content from any of a list of urls. -}
 downloadUrl :: [Url.URLString] -> FilePath -> Annex Bool
 downloadUrl urls file = do
-	o <- map Param . words <$> getConfig "annex.web-options" ""
+	o <- map Param . words <$> getConfig (annexConfig "web-options") ""
 	headers <- getHttpHeaders
 	liftIO $ anyM (\u -> Url.download u headers o file) urls
 
diff --git a/Annex/Queue.hs b/Annex/Queue.hs
index 728e29645e..24575e9068 100644
--- a/Annex/Queue.hs
+++ b/Annex/Queue.hs
@@ -46,7 +46,7 @@ new = do
 	store q
 	return q
 	where
-		queuesize = readish <$> getConfig "annex.queuesize" ""
+		queuesize = readish <$> getConfig (annexConfig "queuesize") ""
 
 store :: Git.Queue.Queue -> Annex ()
 store q = changeState $ \s -> s { repoqueue = Just q }
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index 6a230312ab..f0824b1191 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -48,7 +48,7 @@ sshInfo (host, port) = ifM caching
 	where
 		caching = fromMaybe SysConfig.sshconnectioncaching 
 			. Git.configTrue
-			<$> getConfig "annex.sshcaching" ""
+			<$> getConfig (annexConfig "sshcaching") ""
 
 cacheParams :: FilePath -> [CommandParam]
 cacheParams socketfile =
diff --git a/Annex/UUID.hs b/Annex/UUID.hs
index 5459cc7fee..517840fbad 100644
--- a/Annex/UUID.hs
+++ b/Annex/UUID.hs
@@ -23,12 +23,11 @@ module Annex.UUID (
 import Common.Annex
 import qualified Git
 import qualified Git.Config
-import qualified Git.Command
 import qualified Build.SysConfig as SysConfig
 import Config
 
-configkey :: String
-configkey = "annex.uuid"
+configkey :: ConfigKey
+configkey = annexConfig "uuid"
 
 {- Generates a UUID. There is a library for this, but it's not packaged,
  - so use the command line tool. -}
@@ -64,16 +63,17 @@ getRepoUUID r = do
 		cachekey = remoteConfig r "uuid"
 
 removeRepoUUID :: Annex ()
-removeRepoUUID = inRepo $ Git.Command.run "config"
-	[Param "--unset", Param configkey]
+removeRepoUUID = unsetConfig configkey
 
 getUncachedUUID :: Git.Repo -> UUID
-getUncachedUUID = toUUID . Git.Config.get configkey ""
+getUncachedUUID = toUUID . Git.Config.get key ""
+	where
+		(ConfigKey key) = configkey
 
 {- Make sure that the repo has an annex.uuid setting. -}
 prepUUID :: Annex ()
 prepUUID = whenM ((==) NoUUID <$> getUUID) $
 	storeUUID configkey =<< liftIO genUUID
 
-storeUUID :: String -> UUID -> Annex ()
+storeUUID :: ConfigKey -> UUID -> Annex ()
 storeUUID configfield = setConfig configfield . fromUUID
diff --git a/Annex/Version.hs b/Annex/Version.hs
index a1d0402445..7c909ae05b 100644
--- a/Annex/Version.hs
+++ b/Annex/Version.hs
@@ -21,8 +21,8 @@ supportedVersions = [defaultVersion]
 upgradableVersions :: [Version]
 upgradableVersions = ["0", "1", "2"]
 
-versionField :: String
-versionField = "annex.version"
+versionField :: ConfigKey
+versionField = annexConfig "version"
 
 getVersion :: Annex (Maybe Version)
 getVersion = handle <$> getConfig versionField ""
diff --git a/Backend.hs b/Backend.hs
index 19562205c8..8071b9b835 100644
--- a/Backend.hs
+++ b/Backend.hs
@@ -46,7 +46,7 @@ orderedList = do
 			l' <- (lookupBackendName name :) <$> standard
 			Annex.changeState $ \s -> s { Annex.backends = l' }
 			return l'
-		standard = parseBackendList <$> getConfig "annex.backends" ""
+		standard = parseBackendList <$> getConfig (annexConfig "backends") ""
 		parseBackendList [] = list
 		parseBackendList s = map lookupBackendName $ words s
 
diff --git a/Command/Unused.hs b/Command/Unused.hs
index 6b319ee72c..f5ee452a80 100644
--- a/Command/Unused.hs
+++ b/Command/Unused.hs
@@ -183,10 +183,10 @@ exclude smaller larger = S.toList $ remove larger $ S.fromList smaller
  -}
 bloomCapacity :: Annex Int
 bloomCapacity = fromMaybe 500000 . readish
-	<$> getConfig "annex.bloomcapacity" ""
+	<$> getConfig (annexConfig "bloomcapacity") ""
 bloomAccuracy :: Annex Int
 bloomAccuracy = fromMaybe 1000 . readish
-	<$> getConfig "annex.bloomaccuracy" ""
+	<$> getConfig (annexConfig "bloomaccuracy") ""
 bloomBitsHashes :: Annex (Int, Int)
 bloomBitsHashes = do
 	capacity <- bloomCapacity
diff --git a/Config.hs b/Config.hs
index 3feb246e53..5f1ac8bb20 100644
--- a/Config.hs
+++ b/Config.hs
@@ -1,6 +1,6 @@
 {- Git configuration
  -
- - Copyright 2011 Joey Hess 
+ - Copyright 2011-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -14,29 +14,40 @@ import qualified Git.Command
 import qualified Annex
 import Utility.DataUnits
 
-type ConfigKey = String
+type UnqualifiedConfigKey = String
+data ConfigKey = ConfigKey String
 
 {- Changes a git config setting in both internal state and .git/config -}
 setConfig :: ConfigKey -> String -> Annex ()
-setConfig k value = do
-	inRepo $ Git.Command.run "config" [Param k, Param value]
+setConfig (ConfigKey key) value = do
+	inRepo $ Git.Command.run "config" [Param key, Param value]
 	-- re-read git config and update the repo's state
 	newg <- inRepo Git.Config.read
 	Annex.changeState $ \s -> s { Annex.repo = newg }
 
-{- Looks up a git config setting in git config. -}
+{- Unsets a git config setting. (Leaves it in state currently.) -}
+unsetConfig :: ConfigKey -> Annex ()
+unsetConfig (ConfigKey key) = inRepo $ Git.Command.run "config"
+        [Param "--unset", Param key]
+
+{- Looks up a setting in git config. -}
 getConfig :: ConfigKey -> String -> Annex String
-getConfig key def = fromRepo $ Git.Config.get key def
+getConfig (ConfigKey key) def = fromRepo $ Git.Config.get key def
 
 {- Looks up a per-remote config setting in git config.
  - Failing that, tries looking for a global config option. -}
-getRemoteConfig :: Git.Repo -> ConfigKey -> String -> Annex String
-getRemoteConfig r key def =
-	getConfig (remoteConfig r key) =<< getConfig key def
+getRemoteConfig :: Git.Repo -> UnqualifiedConfigKey -> String -> Annex String
+getRemoteConfig r key def = 
+	getConfig (remoteConfig r key) =<< getConfig (annexConfig key) def
 
 {- A per-remote config setting in git config. -}
-remoteConfig :: Git.Repo -> ConfigKey -> String
-remoteConfig r key = "remote." ++ fromMaybe "" (Git.remoteName r) ++ ".annex-" ++ key
+remoteConfig :: Git.Repo -> UnqualifiedConfigKey -> ConfigKey
+remoteConfig r key = ConfigKey $
+	"remote." ++ fromMaybe "" (Git.remoteName r) ++ ".annex-" ++ key
+
+{- A global annex setting in git config. -}
+annexConfig :: UnqualifiedConfigKey -> ConfigKey
+annexConfig key = ConfigKey $ "annex." ++ key
 
 {- Calculates cost for a remote. Either the default, or as configured 
  - by remote..annex-cost, or if remote..annex-cost-command
@@ -83,17 +94,19 @@ getNumCopies v = perhaps (use v) =<< Annex.getState Annex.forcenumcopies
 	where
 		use (Just n) = return n
 		use Nothing = perhaps (return 1) =<< 
-			readish <$> getConfig "annex.numcopies" "1"
+			readish <$> getConfig (annexConfig "numcopies") "1"
 		perhaps fallback = maybe fallback (return . id)
 
 {- Gets the trust level set for a remote in git config. -}
 getTrustLevel :: Git.Repo -> Annex (Maybe String)
-getTrustLevel r = fromRepo $ Git.Config.getMaybe $ remoteConfig r "trustlevel"
+getTrustLevel r = fromRepo $ Git.Config.getMaybe key
+	where
+		(ConfigKey key) = remoteConfig r "trustlevel"
 
 {- Gets annex.diskreserve setting. -}
 getDiskReserve :: Annex Integer
 getDiskReserve = fromMaybe megabyte . readSize dataUnits
-	<$> getConfig "annex.diskreserve" ""
+	<$> getConfig (annexConfig "diskreserve") ""
 	where
 		megabyte = 1000000
 
@@ -101,7 +114,7 @@ getDiskReserve = fromMaybe megabyte . readSize dataUnits
  - splitting it into lines. -}
 getHttpHeaders :: Annex [String]
 getHttpHeaders = do
-	cmd <- getConfig "annex.http-headers-command" ""
+	cmd <- getConfig (annexConfig "http-headers-command") ""
 	if (null cmd)
 		then fromRepo $ Git.Config.getList "annex.http-headers"
 		else lines . snd <$> liftIO (pipeFrom "sh" ["-c", cmd])
diff --git a/Remote/Hook.hs b/Remote/Hook.hs
index 1c87823caa..dcac9da889 100644
--- a/Remote/Hook.hs
+++ b/Remote/Hook.hs
@@ -74,14 +74,14 @@ hookEnv k f = Just $ fileenv f ++ keyenv
 
 lookupHook :: String -> String -> Annex (Maybe String)
 lookupHook hooktype hook =do
-	command <- getConfig hookname ""
+	command <- getConfig (annexConfig hookname) ""
 	if null command
 		then do
 			warning $ "missing configuration for " ++ hookname
 			return Nothing
 		else return $ Just command
 	where
-		hookname =  "annex." ++ hooktype ++ "-" ++ hook ++ "-hook"
+		hookname = hooktype ++ "-" ++ hook ++ "-hook"
 
 runHook :: String -> String -> Key -> Maybe FilePath -> Annex Bool -> Annex Bool
 runHook hooktype hook k f a = maybe (return False) run =<< lookupHook hooktype hook
diff --git a/debian/changelog b/debian/changelog
index b419b46227..049531dad0 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -8,6 +8,9 @@ git-annex (3.20120431) UNRELEASED; urgency=low
   * dropunused: Allow specifying ranges to drop.
   * addunused: New command, the opposite of dropunused, it relinks unused
     content into the git repository.
+  * Fix use of several config settings annex.ssh-options,
+    annex.rsync-options, annex.bup-split-options. (And adjust types to avoid
+    the bugs that broke several config settings.)
 
  -- Joey Hess   Wed, 02 May 2012 13:06:18 -0400
 

From bbfa74e7ac365124e5aa8d4797d9f1c957b81eda Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 7 May 2012 13:18:45 -0400
Subject: [PATCH 107/220] format

---
 debian/changelog | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/changelog b/debian/changelog
index 049531dad0..7d10b560b1 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -8,7 +8,7 @@ git-annex (3.20120431) UNRELEASED; urgency=low
   * dropunused: Allow specifying ranges to drop.
   * addunused: New command, the opposite of dropunused, it relinks unused
     content into the git repository.
-  * Fix use of several config settings annex.ssh-options,
+  * Fix use of several config settings: annex.ssh-options,
     annex.rsync-options, annex.bup-split-options. (And adjust types to avoid
     the bugs that broke several config settings.)
 

From 3770bc5d4181e230159b493a2f6bef7a2f19a7b7 Mon Sep 17 00:00:00 2001
From: "https://me.yahoo.com/a/IAg3idYGk.joxsJb2WCxl20gig.0.8hS#d5165"
 
Date: Mon, 7 May 2012 21:57:58 +0000
Subject: [PATCH 108/220]

---
 doc/forum/Making_git-annex_less_necessary.mdwn | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 doc/forum/Making_git-annex_less_necessary.mdwn

diff --git a/doc/forum/Making_git-annex_less_necessary.mdwn b/doc/forum/Making_git-annex_less_necessary.mdwn
new file mode 100644
index 0000000000..086c051398
--- /dev/null
+++ b/doc/forum/Making_git-annex_less_necessary.mdwn
@@ -0,0 +1,5 @@
+http://git-annex.branchable.com/walkthrough/ says "Git wants to first stage the entire contents of the file in its index. That can be slow for big files (sorta why git-annex exists in the first place)."

+What is git doing that git-annex isn't, other than copying the file to .git/objects rather than just moving it to .git/annex/objects, prepending it with "blob"+length, and compressing it? If git were changed to store the "blob"+length as part of the object filename rather than as part of the object file content, have a config option to use uncompressed objects for large files (and not try to pack them when creating pack files), and were used on a filesystem such as zfs or btrfs which does COW so the copy would be as fast as a move, then what speed advantage would git-annex still have over git? I realize git-annex has more features than just big file handling, and has the worm backend for even faster handling, but I'm just talking about the case with the default sha backend.

+Have such changes been proposed for git? It seems that for anybody already familiar with the git codebase, adding the config option for uncompressed objects and moving the storage location for "blob"+length would be easy changes to make, and I see no downside to them. It wouldn't break backwards compatibility because the object filename being hash."blob".length rather than just hash would indicate that the new object format is in use, and a ".raw" filename extension could be used for uncompressed objects (or more sensibly, in the new format, no additional extension for uncompressed, and ".compressed" for compressed).

+This would also eliminate the need for a git-annex object store separate from the git object store, and the complexities involved with having them separate, and the need for symlinks, and the complexities they cause. I don't think that relying on COW for speed is unreasonable once btrfs becomes the default in major Linux distros (the bsds already have zfs and hammerfs); right now part of what git-annex is doing is just working around the functional deficiency of non-COW filesystems.

+P.S. I recommend a "plain" option for the page type when submitting comments on your wiki, so I don't have to put HTML line break markup at the end of my lines.

From ac08c37649e25cd3647d3ca97bda91ffc9232889 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Tue, 8 May 2012 18:22:12 +0000
Subject: [PATCH 109/220] Added a comment

---
 ...mment_1_03faaa3866778d24cd03887b85dc9954._comment | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 doc/forum/Making_git-annex_less_necessary/comment_1_03faaa3866778d24cd03887b85dc9954._comment

diff --git a/doc/forum/Making_git-annex_less_necessary/comment_1_03faaa3866778d24cd03887b85dc9954._comment b/doc/forum/Making_git-annex_less_necessary/comment_1_03faaa3866778d24cd03887b85dc9954._comment
new file mode 100644
index 0000000000..3396643f66
--- /dev/null
+++ b/doc/forum/Making_git-annex_less_necessary/comment_1_03faaa3866778d24cd03887b85dc9954._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.245"
+ subject="comment 1"
+ date="2012-05-08T18:22:12Z"
+ content="""
+git's code base makes lots of assumptions hardcoding the size of the hash, etc. (grep its source for magic numbers 40 and 42...) I'd like to see git get parameratised hashes. SHA1 insecurity may evenutally push it in that direction. However, when I asked the git developers about this at the Gittogether last year, there were several ideas floated that would avoid parameterisation, and a lot of good thoughts about problems parameterised hashes would cause.
+
+Moving data into git proper would still leave the problems unique to large data of not being able to store it all on every clone. Which means a git-annex like thing is needed to track where the data resides and move it around.
+
+(BTW, in markdown, you separate paragraphs with blank lines. Like in email.)
+"""]]

From ef247f3da7f9b767180a2985922cf3d0c391f278 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Thu, 10 May 2012 13:14:50 +0000
Subject: [PATCH 110/220]

---
 doc/forum/Git_Annex_Transfer_Protocols.mdwn | 9 +++++++++
 1 file changed, 9 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols.mdwn

diff --git a/doc/forum/Git_Annex_Transfer_Protocols.mdwn b/doc/forum/Git_Annex_Transfer_Protocols.mdwn
new file mode 100644
index 0000000000..d6a660a2d8
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols.mdwn
@@ -0,0 +1,9 @@
+Hi,
+
+May I know, which processes git-annex is using to move/copy/get file content between repositories?
+Is it using same processes which git uses, Like send-pack, receive-pack.
+I want to use Aspera to move file contents between repositories.
+Is it enough if I customize send-pack, receive-pack of git code to do Aspera file transfer or git-annex uses any other transfer mechanism.
+
+Many Thanks,
+Royal Pinto

From ab84b612ead4bea1c1e5fdf090d7371283a296a7 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 10 May 2012 18:18:01 +0000
Subject: [PATCH 111/220] Added a comment: rsync over ssh

---
 .../comment_1_a870ec991078c95a6bb683d6962ab56e._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_1_a870ec991078c95a6bb683d6962ab56e._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_1_a870ec991078c95a6bb683d6962ab56e._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_1_a870ec991078c95a6bb683d6962ab56e._comment
new file mode 100644
index 0000000000..31c463470c
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_1_a870ec991078c95a6bb683d6962ab56e._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.186"
+ subject="rsync over ssh"
+ date="2012-05-10T18:18:01Z"
+ content="""
+Some other protocols such as S3 for special remotes.
+"""]]

From 32a41f8af18a067f59e43f6fdc0be312ade8b69e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 10 May 2012 14:18:35 -0400
Subject: [PATCH 112/220] add a favicon

---
 debian/copyright |   6 ++++++
 doc/favicon.ico  | Bin 0 -> 405 bytes
 2 files changed, 6 insertions(+)
 create mode 100644 doc/favicon.ico

diff --git a/debian/copyright b/debian/copyright
index 332c1e71d2..de1e08e1cd 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -7,3 +7,9 @@ License: GPL-3+
  The full text of version 3 of the GPL is distributed as doc/GPL in
  this package's source, or in /usr/share/common-licenses/GPL-3 on
  Debian systems.
+
+Files: doc/logo.png doc/logo_small.png doc/favicon.png
+Copyright: 2007 Henrik Nyh 
+           2010 Joey Hess 
+License: other
+  Free to modify and redistribute with due credit, and obviously free to use.
diff --git a/doc/favicon.ico b/doc/favicon.ico
new file mode 100644
index 0000000000000000000000000000000000000000..5bb405931fcc8f8194694e5d6cb728e50658891e
GIT binary patch
literal 405
zcmex=bI)35R>*e`CsU}f<4#mC*t-skh=9nPFAvL)f4AHU1X%imZ2>GMq7
z9Qkz8nG&C%#d}NA9x5}4Y=}=u%=tanUZi)Dv@7qlRhOEXZ#_NRE_mW`HTUk>SB@)n
MY&muAb@%_9027yGs{jB1

literal 0
HcmV?d00001


From 660f5de5512e25654e84802317686b21419a4148 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Thu, 10 May 2012 18:48:41 +0000
Subject: [PATCH 113/220] Added a comment: Protocols to transfer file content

---
 .../comment_2_71419376ef50a679ea8f0f9e16991c17._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_2_71419376ef50a679ea8f0f9e16991c17._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_2_71419376ef50a679ea8f0f9e16991c17._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_2_71419376ef50a679ea8f0f9e16991c17._comment
new file mode 100644
index 0000000000..2e07aef088
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_2_71419376ef50a679ea8f0f9e16991c17._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
+ nickname="Royal"
+ subject="Protocols to transfer file content"
+ date="2012-05-10T18:48:40Z"
+ content="""
+Thanks, Is git annex is using same protocols as normal git to transfer content between normal git repositories?
+"""]]

From daaeba3afb3be5b5c72269d999b7286261fac0c3 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 10 May 2012 18:51:56 +0000
Subject: [PATCH 114/220] Added a comment

---
 .../comment_3_fea43664a500111ca99f4043e0dadb14._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_3_fea43664a500111ca99f4043e0dadb14._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_3_fea43664a500111ca99f4043e0dadb14._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_3_fea43664a500111ca99f4043e0dadb14._comment
new file mode 100644
index 0000000000..6e7b36e311
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_3_fea43664a500111ca99f4043e0dadb14._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.186"
+ subject="comment 3"
+ date="2012-05-10T18:51:56Z"
+ content="""
+git-annex doesn't transfer git content between git repositories. You use git for that. Well, git-annex sync can run a few git commands for you to do it.
+"""]]

From 8734e7cba518034a08f85eeef153bf64bb09ed0c Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Thu, 10 May 2012 19:13:48 +0000
Subject: [PATCH 115/220] Added a comment: Git annex content transfer protocols

---
 .../comment_4_56fb2dab1d4030c9820be32b495afdf0._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_4_56fb2dab1d4030c9820be32b495afdf0._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_4_56fb2dab1d4030c9820be32b495afdf0._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_4_56fb2dab1d4030c9820be32b495afdf0._comment
new file mode 100644
index 0000000000..fef10cd813
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_4_56fb2dab1d4030c9820be32b495afdf0._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
+ nickname="Royal"
+ subject="Git annex content transfer protocols"
+ date="2012-05-10T19:13:48Z"
+ content="""
+Sorry if I am not clear. Actually i meant to ask, if i have 2 git repositories which are not special remotes and I am transferring annexed file content between these repositories using git annex command (move or copy) then, which protocol it uses to transfer content? Is it uses git-send-pack git-recieve-pack or some other protocols. 
+"""]]

From 891684f85dd8691bbea2557d645bf194b76abd24 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 10 May 2012 19:17:22 +0000
Subject: [PATCH 116/220] Added a comment

---
 .../comment_5_a6ec9c5a4a3c0bac1df87f1df9be140b._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_5_a6ec9c5a4a3c0bac1df87f1df9be140b._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_5_a6ec9c5a4a3c0bac1df87f1df9be140b._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_5_a6ec9c5a4a3c0bac1df87f1df9be140b._comment
new file mode 100644
index 0000000000..be25737c1d
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_5_a6ec9c5a4a3c0bac1df87f1df9be140b._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.186"
+ subject="comment 5"
+ date="2012-05-10T19:17:22Z"
+ content="""
+rsync over ssh is used to transfer file contents between repositories. (You can use the -d option to see the commands git-annex runs.)
+"""]]

From f02db7bfce54563de9a42588ba926d22d4fe3ace Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Thu, 10 May 2012 19:21:08 +0000
Subject: [PATCH 117/220] Added a comment

---
 .../comment_6_1678452fb7114aeabcf0cc3d5f6c69b0._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Git_Annex_Transfer_Protocols/comment_6_1678452fb7114aeabcf0cc3d5f6c69b0._comment

diff --git a/doc/forum/Git_Annex_Transfer_Protocols/comment_6_1678452fb7114aeabcf0cc3d5f6c69b0._comment b/doc/forum/Git_Annex_Transfer_Protocols/comment_6_1678452fb7114aeabcf0cc3d5f6c69b0._comment
new file mode 100644
index 0000000000..b7ef8f33c4
--- /dev/null
+++ b/doc/forum/Git_Annex_Transfer_Protocols/comment_6_1678452fb7114aeabcf0cc3d5f6c69b0._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
+ nickname="Royal"
+ subject="comment 6"
+ date="2012-05-10T19:21:08Z"
+ content="""
+Ok. This helped me a lot. Thank you
+"""]]

From 61a5df33d4d24ccbedf8395c2218e1e5a3f241e1 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 11 May 2012 12:37:26 -0400
Subject: [PATCH 118/220] releasing version 3.20120511

---
 debian/changelog | 4 ++--
 git-annex.cabal  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 7d10b560b1..6ebd6fabf0 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,4 @@
-git-annex (3.20120431) UNRELEASED; urgency=low
+git-annex (3.20120511) unstable; urgency=low
 
   * Rsync special remotes can be configured with shellescape=no
     to avoid shell quoting that is normally done when using rsync over ssh.
@@ -12,7 +12,7 @@ git-annex (3.20120431) UNRELEASED; urgency=low
     annex.rsync-options, annex.bup-split-options. (And adjust types to avoid
     the bugs that broke several config settings.)
 
- -- Joey Hess   Wed, 02 May 2012 13:06:18 -0400
+ -- Joey Hess   Fri, 11 May 2012 12:29:30 -0400
 
 git-annex (3.20120430) unstable; urgency=low
 
diff --git a/git-annex.cabal b/git-annex.cabal
index 6299944698..f87e49a200 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -1,5 +1,5 @@
 Name: git-annex
-Version: 3.20120430
+Version: 3.20120511
 Cabal-Version: >= 1.8
 License: GPL
 Maintainer: Joey Hess 

From 1c942c898d8f738652a5f4239fc7cd0b009e4903 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 11 May 2012 12:38:08 -0400
Subject: [PATCH 119/220] add news item for git-annex 3.20120511

---
 doc/news/version_3.20120315.mdwn | 21 ---------------------
 doc/news/version_3.20120511.mdwn | 13 +++++++++++++
 2 files changed, 13 insertions(+), 21 deletions(-)
 delete mode 100644 doc/news/version_3.20120315.mdwn
 create mode 100644 doc/news/version_3.20120511.mdwn

diff --git a/doc/news/version_3.20120315.mdwn b/doc/news/version_3.20120315.mdwn
deleted file mode 100644
index a3ccb4cf47..0000000000
--- a/doc/news/version_3.20120315.mdwn
+++ /dev/null
@@ -1,21 +0,0 @@
-git-annex 3.20120315 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * fsck: Fix up any broken links and misplaced content caused by the
-     directory hash calculation bug fixed in the last release.
-   * sync: Sync to lower cost remotes first.
-   * status: Fixed to run in constant space.
-   * status: More accurate display of sizes of tmp and bad keys.
-   * unused: Now uses a bloom filter, and runs in constant space.
-     Use of a bloom filter does mean it will not notice a small
-     number of unused keys. For repos with up to half a million keys,
-     it will miss one key in 1000.
-   * Added annex.bloomcapacity and annex.bloomaccuracy, which can be
-     adjusted as desired to tune the bloom filter.
-   * status: Display amount of memory used by bloom filter, and
-     detect when it's too small for the number of keys in a repository.
-   * git-annex-shell: Runs hooks/annex-content after content is received
-     or dropped.
-   * Work around a bug in rsync (IMHO) introduced by openSUSE's SIP patch.
-   * git-annex now behaves as git-annex-shell if symlinked to and run by that
-     name. The Makefile sets this up, saving some 8 mb of installed size.
-   * git-union-merge is a demo program, so it is no longer built by default."""]]
\ No newline at end of file
diff --git a/doc/news/version_3.20120511.mdwn b/doc/news/version_3.20120511.mdwn
new file mode 100644
index 0000000000..19e8355224
--- /dev/null
+++ b/doc/news/version_3.20120511.mdwn
@@ -0,0 +1,13 @@
+git-annex 3.20120511 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Rsync special remotes can be configured with shellescape=no
+     to avoid shell quoting that is normally done when using rsync over ssh.
+     This is known to be needed for certian rsync hosting providers
+     (specificially hidrive.strato.com) that use rsync over ssh but do not
+     pass it through the shell.
+   * dropunused: Allow specifying ranges to drop.
+   * addunused: New command, the opposite of dropunused, it relinks unused
+     content into the git repository.
+   * Fix use of several config settings: annex.ssh-options,
+     annex.rsync-options, annex.bup-split-options. (And adjust types to avoid
+     the bugs that broke several config settings.)"""]]
\ No newline at end of file

From 2701800cda086782f40dd615c23d605667b8a81b Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 11 May 2012 12:38:56 -0400
Subject: [PATCH 120/220] cleanup

---
 ..._fb1a3135e2d9f39f2c372ccc2c50c85a._comment |   8 --
 ..._ae292ca7294b6790233e545086c3ac2f._comment |  10 --
 ..._29ccda9ac458fd5cc9ec5508c62df6ea._comment |  12 --
 ..._f8fc894680f2a2e5b5e757a677414b42._comment |   8 --
 ..._ea5075cfecc50d5da2364931ef7a02d1._comment |  10 --
 ..._18158b9be2313f49509d59295c7d3c90._comment |   8 --
 ..._03436ddda42decf8cb1b4d5316d88a75._comment |  14 --
 ..._8f7f8d4758804f1b695925934219745a._comment |  42 ------
 ..._cd90223f78571e5bdd3dfc07ab1369d7._comment |   9 --
 ..._7dbf131ff4611abbfc8fbf1ee0f66dbe._comment |   8 --
 ..._b975cbd3a01ba5c2fa0f24fe739d3433._comment | 128 ------------------
 ..._899de1196cd1ba4a393e4ef574d7aa5e._comment |   8 --
 12 files changed, 265 deletions(-)
 delete mode 100644 doc/news/version_3.20120106/comment_1_fb1a3135e2d9f39f2c372ccc2c50c85a._comment
 delete mode 100644 doc/news/version_3.20120106/comment_2_ae292ca7294b6790233e545086c3ac2f._comment
 delete mode 100644 doc/news/version_3.20120106/comment_3_29ccda9ac458fd5cc9ec5508c62df6ea._comment
 delete mode 100644 doc/news/version_3.20120227/comment_1_f8fc894680f2a2e5b5e757a677414b42._comment
 delete mode 100644 doc/news/version_3.20120227/comment_2_ea5075cfecc50d5da2364931ef7a02d1._comment
 delete mode 100644 doc/news/version_3.20120229/comment_1_18158b9be2313f49509d59295c7d3c90._comment
 delete mode 100644 doc/news/version_3.20120229/comment_2_03436ddda42decf8cb1b4d5316d88a75._comment
 delete mode 100644 doc/news/version_3.20120229/comment_3_8f7f8d4758804f1b695925934219745a._comment
 delete mode 100644 doc/news/version_3.20120229/comment_4_cd90223f78571e5bdd3dfc07ab1369d7._comment
 delete mode 100644 doc/news/version_3.20120229/comment_5_7dbf131ff4611abbfc8fbf1ee0f66dbe._comment
 delete mode 100644 doc/news/version_3.20120230/comment_1_b975cbd3a01ba5c2fa0f24fe739d3433._comment
 delete mode 100644 doc/news/version_3.20120230/comment_2_899de1196cd1ba4a393e4ef574d7aa5e._comment

diff --git a/doc/news/version_3.20120106/comment_1_fb1a3135e2d9f39f2c372ccc2c50c85a._comment b/doc/news/version_3.20120106/comment_1_fb1a3135e2d9f39f2c372ccc2c50c85a._comment
deleted file mode 100644
index effd9c9a8f..0000000000
--- a/doc/news/version_3.20120106/comment_1_fb1a3135e2d9f39f2c372ccc2c50c85a._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.186.134.113"
- subject="Not announced on Hackage?"
- date="2012-01-13T17:37:11Z"
- content="""
-We have the [latest version in NixOS](http://hydra.nixos.org/job/nixpkgs/trunk/gitAndTools.gitAnnex), but we cannot advertise that fact on Hackage because it seems the corresponding Cabal file hasn't been uploaded to . Is there any particular reason why Hackage doesn't know about this release?
-"""]]
diff --git a/doc/news/version_3.20120106/comment_2_ae292ca7294b6790233e545086c3ac2f._comment b/doc/news/version_3.20120106/comment_2_ae292ca7294b6790233e545086c3ac2f._comment
deleted file mode 100644
index 7e6920eb72..0000000000
--- a/doc/news/version_3.20120106/comment_2_ae292ca7294b6790233e545086c3ac2f._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joey.kitenet.net/"
- nickname="joey"
- subject="comment 2"
- date="2012-01-13T17:52:46Z"
- content="""
-Uploading to hackage is a PITA (manual password entry and I am often on a slow link besides) and is not integrated with my regular release process, so I often forget to do it. I will try to upload the next release there again.
-
-You might add a page under [[install]] for your git-annex packages.
-"""]]
diff --git a/doc/news/version_3.20120106/comment_3_29ccda9ac458fd5cc9ec5508c62df6ea._comment b/doc/news/version_3.20120106/comment_3_29ccda9ac458fd5cc9ec5508c62df6ea._comment
deleted file mode 100644
index 1bb0c32cb1..0000000000
--- a/doc/news/version_3.20120106/comment_3_29ccda9ac458fd5cc9ec5508c62df6ea._comment
+++ /dev/null
@@ -1,12 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.186.134.113"
- subject="comment 3"
- date="2012-01-13T18:48:28Z"
- content="""
-For what it's worth, the package `cabal-install` is pretty good for uploading packages to Hackages, among other things. It allows users to configure their username/password, and then making a release to Hackage is as simple as running:
-
-    cabal upload foo-version.tar.gz
-
-I use that to do releases of my stuff, too, and I'm quite happy. `cabal-install` has other features, too, of course.
-"""]]
diff --git a/doc/news/version_3.20120227/comment_1_f8fc894680f2a2e5b5e757a677414b42._comment b/doc/news/version_3.20120227/comment_1_f8fc894680f2a2e5b5e757a677414b42._comment
deleted file mode 100644
index aa3c5ce6d1..0000000000
--- a/doc/news/version_3.20120227/comment_1_f8fc894680f2a2e5b5e757a677414b42._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.184.15.65"
- subject="Test-suite won't compile with GHC 7.4.x"
- date="2012-02-28T17:39:59Z"
- content="""
-The recent version requires GHC 7.4.x, but some dependencies for the test suite don't build with that compiler, i.e. the `testpack` library. Do you have any recommendation how to deal with that situation? I would like to update, but I would very much like to run the regression test suite, too.
-"""]]
diff --git a/doc/news/version_3.20120227/comment_2_ea5075cfecc50d5da2364931ef7a02d1._comment b/doc/news/version_3.20120227/comment_2_ea5075cfecc50d5da2364931ef7a02d1._comment
deleted file mode 100644
index d45e66026f..0000000000
--- a/doc/news/version_3.20120227/comment_2_ea5075cfecc50d5da2364931ef7a02d1._comment
+++ /dev/null
@@ -1,10 +0,0 @@
-[[!comment format=mdwn
- username="http://joey.kitenet.net/"
- nickname="joey"
- subject="comment 2"
- date="2012-02-29T04:27:47Z"
- content="""
-Here's the patch that was used to make testpack build with 7.4 on Debian:
-
-
-"""]]
diff --git a/doc/news/version_3.20120229/comment_1_18158b9be2313f49509d59295c7d3c90._comment b/doc/news/version_3.20120229/comment_1_18158b9be2313f49509d59295c7d3c90._comment
deleted file mode 100644
index c3d2cab59a..0000000000
--- a/doc/news/version_3.20120229/comment_1_18158b9be2313f49509d59295c7d3c90._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.186.152.146"
- subject="How do you build the Crypto library with GHC 7.4.1?"
- date="2012-02-29T19:20:20Z"
- content="""
-`Crypto 4.2.4` doesn't seem to compile with GHC 7.4.1. How did you build that package?
-"""]]
diff --git a/doc/news/version_3.20120229/comment_2_03436ddda42decf8cb1b4d5316d88a75._comment b/doc/news/version_3.20120229/comment_2_03436ddda42decf8cb1b4d5316d88a75._comment
deleted file mode 100644
index b08712ee8f..0000000000
--- a/doc/news/version_3.20120229/comment_2_03436ddda42decf8cb1b4d5316d88a75._comment
+++ /dev/null
@@ -1,14 +0,0 @@
-[[!comment format=mdwn
- username="http://joey.kitenet.net/"
- nickname="joey"
- subject="comment 2"
- date="2012-02-29T22:54:01Z"
- content="""
-Probably this patch will help with Crypto:
-
-
-
-Or, there's the `ghc7.0` branch of git-annex in git, which can be used to build with the older, stable ghc.
-
-BTW, when asking, for help, a log of the build failure is always a good idea..
-"""]]
diff --git a/doc/news/version_3.20120229/comment_3_8f7f8d4758804f1b695925934219745a._comment b/doc/news/version_3.20120229/comment_3_8f7f8d4758804f1b695925934219745a._comment
deleted file mode 100644
index a31a182cca..0000000000
--- a/doc/news/version_3.20120229/comment_3_8f7f8d4758804f1b695925934219745a._comment
+++ /dev/null
@@ -1,42 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.186.165.208"
- subject="comment 3"
- date="2012-03-05T21:10:47Z"
- content="""
-Unfortunately, the patch you mentioned doesn't seem to address the problem. I'm getting the following compile error:
-
-    Data/Digest/SHA2.hs:111:4:
-        Could not deduce (Show a) arising from a use of `showHex'
-        from the context (Integral a)
-          bound by the instance declaration at Data/Digest/SHA2.hs:109:10-39
-        Possible fix:
-          add (Show a) to the context of the instance declaration
-        In the first argument of `(.)', namely `(showHex a)'
-        In the expression:
-          (showHex a)
-          . (' ' :)
-            . (showHex b)
-              . (' ' :)
-                . (showHex c)
-                  . (' ' :)
-                    . (showHex d)
-                      . (' ' :)
-                        . (showHex e)
-                          . (' ' :)
-                            . (showHex f) . (' ' :) . (showHex g) . (' ' :) . (showHex h)
-        In an equation for `showsPrec':
-            showsPrec _ (Hash8 a b c d e f g h)
-              = (showHex a)
-                . (' ' :)
-                  . (showHex b)
-                    . (' ' :)
-                      . (showHex c)
-                        . (' ' :)
-                          . (showHex d)
-                            . (' ' :)
-                              . (showHex e)
-                                . (' ' :)
-                                  . (showHex f) . (' ' :) . (showHex g) . (' ' :) . (showHex h)
-
-"""]]
diff --git a/doc/news/version_3.20120229/comment_4_cd90223f78571e5bdd3dfc07ab1369d7._comment b/doc/news/version_3.20120229/comment_4_cd90223f78571e5bdd3dfc07ab1369d7._comment
deleted file mode 100644
index a931c7874a..0000000000
--- a/doc/news/version_3.20120229/comment_4_cd90223f78571e5bdd3dfc07ab1369d7._comment
+++ /dev/null
@@ -1,9 +0,0 @@
-[[!comment format=mdwn
- username="http://joey.kitenet.net/"
- nickname="joey"
- subject="comment 4"
- date="2012-03-05T21:32:00Z"
- content="""
-Hmm, I was able to produce exactly the same build error, and then I downloaded the patch I linked to before, and did
-`patch -p1 < debian/patches/class-constraints.diff` and that fixed the build nicely.
-"""]]
diff --git a/doc/news/version_3.20120229/comment_5_7dbf131ff4611abbfc8fbf1ee0f66dbe._comment b/doc/news/version_3.20120229/comment_5_7dbf131ff4611abbfc8fbf1ee0f66dbe._comment
deleted file mode 100644
index afddb3fb0a..0000000000
--- a/doc/news/version_3.20120229/comment_5_7dbf131ff4611abbfc8fbf1ee0f66dbe._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://peter-simons.myopenid.com/"
- ip="77.186.165.208"
- subject="comment 5"
- date="2012-03-05T23:29:41Z"
- content="""
-I didn't realize that the patch adds a patch file to the source distribution (instead of, well, patching it). That additional level of indirection surprised me. Anyway, now I figured it out and `Crypto` compiles fine. Thanks!
-"""]]
diff --git a/doc/news/version_3.20120230/comment_1_b975cbd3a01ba5c2fa0f24fe739d3433._comment b/doc/news/version_3.20120230/comment_1_b975cbd3a01ba5c2fa0f24fe739d3433._comment
deleted file mode 100644
index eb5c7d257b..0000000000
--- a/doc/news/version_3.20120230/comment_1_b975cbd3a01ba5c2fa0f24fe739d3433._comment
+++ /dev/null
@@ -1,128 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawk_LOahrm_Cdg7io-_H0CNKkaxsRRQgRFo"
- nickname="Peter"
- subject="Test suite failure"
- date="2012-03-06T11:20:35Z"
- content="""
-I managed to compile this version of git-annex with GHC 7.4.1 on NixOS, but unfortunately the test suite fails during the `addurl` test:
-
-    Testing 0:quickcheck:0:prop_idempotent_deencode_git                           
-    Testing 0:quickcheck:1:prop_idempotent_deencode                               
-    Testing 0:quickcheck:2:prop_idempotent_fileKey                                
-    Testing 0:quickcheck:3:prop_idempotent_key_read_show                          
-    Testing 0:quickcheck:4:prop_idempotent_shellEscape                            
-    Testing 0:quickcheck:5:prop_idempotent_shellEscape_multiword                  
-    Testing 0:quickcheck:6:prop_idempotent_configEscape                           
-    Testing 0:quickcheck:7:prop_parentDir_basics                                  
-    Testing 0:quickcheck:8:prop_relPathDirToFile_basics                           
-    Testing 0:quickcheck:9:prop_relPathDirToFile_regressionTest                   
-    Testing 0:quickcheck:10:prop_cost_sane                                        
-    Testing 0:quickcheck:11:prop_hmacWithCipher_sane                              
-    Testing 0:quickcheck:12:prop_TimeStamp_sane                                   
-    Testing 0:quickcheck:13:prop_addLog_sane                                      
-    Testing 1:blackbox:0:git-annex init                                           
-    Testing 1:blackbox:1:git-annex add:0                                          
-    Testing 1:blackbox:1:git-annex add:1                                          
-    Testing 1:blackbox:1:git-annex add:2                                          
-    Testing 1:blackbox:2:git-annex reinject/fromkey                               
-    Testing 1:blackbox:3:git-annex unannex:0:no content                           
-    Testing 1:blackbox:3:git-annex unannex:1:with content                         
-    Testing 1:blackbox:4:git-annex drop:0:no remotes                              
-    Testing 1:blackbox:4:git-annex drop:1:with remote                             
-    Testing 1:blackbox:4:git-annex drop:2:untrusted remote                        
-    Testing 1:blackbox:5:git-annex get                                            
-    Testing 1:blackbox:6:git-annex move                                           
-    Testing 1:blackbox:7:git-annex copy                                           
-    Testing 1:blackbox:8:git-annex unlock/lock                                    
-    Testing 1:blackbox:9:git-annex edit/commit:0                                  
-    Cases: 55  Tried: 28  Errors: 0  Failures: 0add foo (checksum...) ok
-    ok
-    (Recording state in git...)
-    Testing 1:blackbox:9:git-annex edit/commit:1                                  
-    Testing 1:blackbox:10:git-annex fix                                           
-    Testing 1:blackbox:11:git-annex trust/untrust/semitrust/dead                  
-    Testing 1:blackbox:12:git-annex fsck:0                                        
-    Cases: 55  Tried: 32  Errors: 0  Failures: 0  Only 1 of 2 trustworthy copies exist of foo
-      Back it up with git-annex copy.
-      Only 1 of 2 trustworthy copies exist of sha1foo
-      Back it up with git-annex copy.
-      Bad file size (11 B larger); moved to /tmp/nix-build-jzvhzrdysy619y4vgmafryy9ck8mz7z7-git-annex-3.20120230.drv-0/git-annex/.t/tmprepo/.git/annex/bad/SHA256-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77
-      Bad file size (11 B larger); moved to /tmp/nix-build-jzvhzrdysy619y4vgmafryy9ck8mz7z7-git-annex-3.20120230.drv-0/git-annex/.t/tmprepo/.git/annex/bad/SHA1-s25--ee80d2cec57a3810db83b80e1b320df3a3721ffa
-    Testing 1:blackbox:12:git-annex fsck:1                                        
-    Testing 1:blackbox:12:git-annex fsck:2                                        
-    Cases: 55  Tried: 34  Errors: 0  Failures: 0  Only these untrusted locations may have copies of foo
-            17575c68-d5cc-4e18-bc96-fdafe716d488 -- origin (test repo)
-            17aab099-fcde-413a-8ef1-6acc09d7d081 -- here (.t/tmprepo)
-      Back it up to trusted locations with git-annex copy.
-      Only these untrusted locations may have copies of sha1foo
-            17575c68-d5cc-4e18-bc96-fdafe716d488 -- origin (test repo)
-      Back it up to trusted locations with git-annex copy.
-    Testing 1:blackbox:12:git-annex fsck:3                                        
-    Cases: 55  Tried: 35  Errors: 0  Failures: 0  Only 1 of 2 trustworthy copies exist of foo
-      Back it up with git-annex copy.
-      The following untrusted locations may also have copies: 
-            17575c68-d5cc-4e18-bc96-fdafe716d488 -- origin (test repo)
-      Only 1 of 2 trustworthy copies exist of sha1foo
-      Back it up with git-annex copy.
-      The following untrusted locations may also have copies: 
-            17575c68-d5cc-4e18-bc96-fdafe716d488 -- origin (test repo)
-    Testing 1:blackbox:13:git-annex migrate:0                                     
-    Testing 1:blackbox:13:git-annex migrate:1                                     
-    Testing 1:blackbox:14:git-annex unused/dropunused                             
-    Testing 1:blackbox:15:git-annex addurl                                        
-    Cases: 55  Tried: 39  Errors: 0  Failures: 0git-annex: connect: timeout (Connection timed out)
-    ### Failure in: 1:blackbox:15:git-annex addurl
-    addurl failed
-    Testing 1:blackbox:16:git-annex describe                                      
-    Testing 1:blackbox:17:git-annex find                                          
-    Cases: 55  Tried: 41  Errors: 0  Failures: 1foo
-    foo
-    sha1foo
-    sha1foo
-    Testing 1:blackbox:18:git-annex merge                                         
-    Testing 1:blackbox:19:git-annex status                                        
-    Cases: 55  Tried: 43  Errors: 0  Failures: 1{\"command\":\"status\",\"supported backends\":[\"SHA256\",\"SHA1\",\"SHA512\",\"SHA224\",\"SHA384\",\"SHA256E\",\"SHA1E\",\"SHA512E\",\"SHA224E\",\"SHA384E\",\"WORM\",\"URL\"],\"supported remote types\":[\"git\",\"S3\",\"bup\",\"directory\",\"rsync\",\"web\",\"hook\"],\"trusted repositories\":[],\"semitrusted repositories\":[{\"uuid\":\"00000000-0000-0000-0000-000000000001\",\"description\":\"web\",\"here\":false},{\"uuid\":\"17575c68-d5cc-4e18-bc96-fdafe716d488\",\"description\":\"origin (test repo)\",\"here\":false},{\"uuid\":\"5b9fe416-d6ed-4df7-af67-14fc5f2ea631\",\"description\":\".t/tmprepo\",\"here\":true}],\"untrusted repositories\":[],\"dead repositories\":[],\"local annex keys\":0,\"local annex size\":\"0 bytes\",\"known annex keys\":2,\"known annex size\":\"45 bytes\",\"success\":true}
-    Testing 1:blackbox:20:git-annex version                                       
-    Cases: 55  Tried: 44  Errors: 0  Failures: 1git-annex version: 3.20120230
-    local repository version: 3
-    default repository version: 3
-    supported repository versions: 3
-    upgrade supported from repository versions: 0 1 2
-    Testing 1:blackbox:21:git-annex sync                                          
-    Cases: 55  Tried: 45  Errors: 0  Failures: 1# On branch master
-    nothing to commit (working directory clean)
-    To /tmp/nix-build-jzvhzrdysy619y4vgmafryy9ck8mz7z7-git-annex-3.20120230.drv-0/git-annex/.t/repo
-       34fa270..7242932  git-annex -> git-annex
-     * [new branch]      master -> synced/master
-    Testing 1:blackbox:22:git-annex map                                           
-    Testing 1:blackbox:23:git-annex uninit                                        
-    Cases: 55  Tried: 47  Errors: 0  Failures: 1Switched to branch 'git-annex'
-    Switched to branch 'master'
-    Deleted branch git-annex (was e636789).
-    Testing 1:blackbox:24:git-annex upgrade                                       
-    Testing 1:blackbox:25:git-annex whereis                                       
-    Testing 1:blackbox:26:git-annex hook remote                                   
-    Testing 1:blackbox:27:git-annex directory remote                              
-    Testing 1:blackbox:28:git-annex rsync remote                                  
-    Cases: 55  Tried: 52  Errors: 0  Failures: 1sending incremental file list
-    af4/
-    af4/74c/
-    af4/74c/SHA256-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77/
-    af4/74c/SHA256-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77/SHA256-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77
-              20 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/5)
-
-    sent 300 bytes  received 43 bytes  686.00 bytes/sec
-    total size is 20  speedup is 0.06
-    SHA256-s20--e394a389d787383843decc5d3d99b6d184ffa5fddeec23b911f9ee7fc8b9ea77
-              20 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/1)
-
-    sent 160 bytes  received 31 bytes  382.00 bytes/sec
-    total size is 20  speedup is 0.10
-    Testing 1:blackbox:29:git-annex bup remote                                    
-    Testing 1:blackbox:30:git-annex crypto                                        
-    Cases: 55  Tried: 55  Errors: 0  Failures: 1
-    test: failed
-    ** test suite failed!
-
-Apparently, there is a network timeout? I see from the comments in `test.hs` that the test suite tries to avoid depending on network traffic, but is it possible, maybe, that the test tries to resolve a DNS name?
-"""]]
diff --git a/doc/news/version_3.20120230/comment_2_899de1196cd1ba4a393e4ef574d7aa5e._comment b/doc/news/version_3.20120230/comment_2_899de1196cd1ba4a393e4ef574d7aa5e._comment
deleted file mode 100644
index a4cc3073d7..0000000000
--- a/doc/news/version_3.20120230/comment_2_899de1196cd1ba4a393e4ef574d7aa5e._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://joey.kitenet.net/"
- nickname="joey"
- subject="comment 2"
- date="2012-03-06T17:22:54Z"
- content="""
-My mistake, addurl --fast used to avoid the network, so the test suite ran it, but then it was changed to always look up the file size. Removed from test suite.
-"""]]

From b261d09637a3c128a636a7b6741615b5d47b90f1 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkRITTYYsN0TFKN7G5sZ6BWGZOTQ88Pz4s"
 
Date: Tue, 15 May 2012 00:14:10 +0000
Subject: [PATCH 121/220] Added a comment: cygwin

---
 ...comment_1_3cc26ad8101a22e95a8c60cf0c4dedcc._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/todo/windows_support/comment_1_3cc26ad8101a22e95a8c60cf0c4dedcc._comment

diff --git a/doc/todo/windows_support/comment_1_3cc26ad8101a22e95a8c60cf0c4dedcc._comment b/doc/todo/windows_support/comment_1_3cc26ad8101a22e95a8c60cf0c4dedcc._comment
new file mode 100644
index 0000000000..fd5b6f5cd3
--- /dev/null
+++ b/doc/todo/windows_support/comment_1_3cc26ad8101a22e95a8c60cf0c4dedcc._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkRITTYYsN0TFKN7G5sZ6BWGZOTQ88Pz4s"
+ nickname="Zoltán"
+ subject="cygwin"
+ date="2012-05-15T00:14:08Z"
+ content="""
+What about [Cygwin](http://cygwin.com/)? It emulates POSIX fairly well under Windows (including signals, forking, fs (also things like /dev/null, /proc), unix file permissions), has all standard gnu utilities. It also emulates symlinks, but they are unfortunately incompatible with NTFS symlinks introduced in Vista [due to some stupid restrictions on Windows](http://cygwin.com/ml/cygwin/2009-10/msg00756.html).
+
+If git-annex could be modified to not require symlinks to work, the it would be a pretty neat solution (and you get a real shell, not some command.com on drugs (aka cmd.exe))
+"""]]

From 461367f23be5e5ff7925be96ad1291b794adfda1 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlYu7QmD7wrbHWkoxuriaA9XcijM-g5vrQ"
 
Date: Tue, 15 May 2012 02:42:31 +0000
Subject: [PATCH 122/220]

---
 ...er_version__39__s_file_content_without_doing_checkout.mdwn | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout.mdwn

diff --git a/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout.mdwn b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout.mdwn
new file mode 100644
index 0000000000..3ed022b481
--- /dev/null
+++ b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout.mdwn
@@ -0,0 +1,4 @@
+Hi,
+Is there any way I can move or copy file content of older version without doing checkout to that version, by passing commit hash as parameter in move command itself? 
+
+Thank you

From 1e590dcc47253b48d54807717d0cf747c659d93f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w"
 
Date: Tue, 15 May 2012 07:36:25 +0000
Subject: [PATCH 123/220] Added a comment

---
 .../comment_10_5ec2f965c80cc5dd31ee3c4edb695664._comment  | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/bugs/git_rename_detection_on_file_move/comment_10_5ec2f965c80cc5dd31ee3c4edb695664._comment

diff --git a/doc/bugs/git_rename_detection_on_file_move/comment_10_5ec2f965c80cc5dd31ee3c4edb695664._comment b/doc/bugs/git_rename_detection_on_file_move/comment_10_5ec2f965c80cc5dd31ee3c4edb695664._comment
new file mode 100644
index 0000000000..6ea2677289
--- /dev/null
+++ b/doc/bugs/git_rename_detection_on_file_move/comment_10_5ec2f965c80cc5dd31ee3c4edb695664._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w"
+ nickname="Rafael"
+ subject="comment 10"
+ date="2012-05-15T07:36:25Z"
+ content="""
+Won't git itself be fixed on this issue? It was on my plans to look into that, however I don't know how difficult it will be.
+"""]]

From 300d3cbdef2450b21c9eb22361161af37ca2f266 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w"
 
Date: Tue, 15 May 2012 07:59:58 +0000
Subject: [PATCH 124/220] Added a comment

---
 .../comment_1_f114b75b29123453758b493fae7f5167._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_1_f114b75b29123453758b493fae7f5167._comment

diff --git a/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_1_f114b75b29123453758b493fae7f5167._comment b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_1_f114b75b29123453758b493fae7f5167._comment
new file mode 100644
index 0000000000..a53c6bbd69
--- /dev/null
+++ b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_1_f114b75b29123453758b493fae7f5167._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnpdM9F8VbtQ_H5PaPMpGSxPe_d5L1eJ6w"
+ nickname="Rafael"
+ subject="comment 1"
+ date="2012-05-15T07:59:57Z"
+ content="""
+I had a similiar question in forum/new_microfeatures/. I would like to fetch/copy all the annexed content from a repo, be it on the current branch, another branch, or corresponds to an old version of a file. A command like \"git annex copy --all --from=source [path]\" would then ensure I have access to all the content I need even if I have later no longer access to source. Sure I could use rsync.
+"""]]

From a25237d2055c147d626267a099a70cc1498f7a8e Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Tue, 15 May 2012 17:00:10 +0000
Subject: [PATCH 125/220] Added a comment

---
 ...comment_2_e377b7614c2961b460a10e285f3db274._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_2_e377b7614c2961b460a10e285f3db274._comment

diff --git a/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_2_e377b7614c2961b460a10e285f3db274._comment b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_2_e377b7614c2961b460a10e285f3db274._comment
new file mode 100644
index 0000000000..18ff153ad0
--- /dev/null
+++ b/doc/forum/Moving_older_version__39__s_file_content_without_doing_checkout/comment_2_e377b7614c2961b460a10e285f3db274._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.2.51"
+ subject="comment 2"
+ date="2012-05-15T17:00:10Z"
+ content="""
+Yes, I think that [[todo/add_-all_option]] is the right approach for this. Seems unlikely you'd have some files' hashes handy without having them checked out, but operating on all content makes sense.
+
+That page discusses some problems implementing it for some commands, but should not pose a problem for `move`. It would also be possible to support `get` and `copy`, except `--auto` couldn't be used with `--all`. Even `fsck` could support it.
+"""]]

From e36808e167815eaa70a827ea703ed98ebb36da64 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 15 May 2012 14:18:51 -0400
Subject: [PATCH 126/220] Pass -a to cp even when it supports --reflink=auto,
 to preserve permissions.

Amoung other things, this makes unlocking a WORM backed file and then
re-adding it without making any changes not add a new object, as the
timestamp is preserved.
---
 Utility/CopyFile.hs | 14 +++++++-------
 debian/changelog    |  7 +++++++
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/Utility/CopyFile.hs b/Utility/CopyFile.hs
index 01639ef2a7..66b88e4f0c 100644
--- a/Utility/CopyFile.hs
+++ b/Utility/CopyFile.hs
@@ -1,6 +1,6 @@
 {- git-annex file copying
  -
- - Copyright 2010 Joey Hess 
+ - Copyright 2010,2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -16,10 +16,10 @@ copyFileExternal :: FilePath -> FilePath -> IO Bool
 copyFileExternal src dest = do
 	whenM (doesFileExist dest) $
 		removeFile dest
-	boolSystem "cp" [params, File src, File dest]
+	boolSystem "cp" $ params ++ [File src, File dest]
 	where
-		params
-			| SysConfig.cp_reflink_auto = Params "--reflink=auto"
-			| SysConfig.cp_a = Params "-a"
-			| SysConfig.cp_p = Params "-p"
-			| otherwise = Params ""
+		params = map snd $ filter fst
+			[ (SysConfig.cp_reflink_auto, Param "--reflink=auto")
+			, (SysConfig.cp_a, Param "-a")
+			, (SysConfig.cp_p && not SysConfig.cp_a, Param "-p")
+			]
diff --git a/debian/changelog b/debian/changelog
index 6ebd6fabf0..a4e2b8b3e9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+git-annex (3.20120512) UNRELEASED; urgency=low
+
+  * Pass -a to cp even when it supports --reflink=auto, to preserve
+    permissions.
+
+ -- Joey Hess   Tue, 15 May 2012 14:17:49 -0400
+
 git-annex (3.20120511) unstable; urgency=low
 
   * Rsync special remotes can be configured with shellescape=no

From a2be4265bf8207c785c7e996e2cc563ac91eb82e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 16 May 2012 15:19:39 -0400
Subject: [PATCH 127/220] add

---
 doc/bugs/GIT_DIR_support_incomplete.mdwn | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 doc/bugs/GIT_DIR_support_incomplete.mdwn

diff --git a/doc/bugs/GIT_DIR_support_incomplete.mdwn b/doc/bugs/GIT_DIR_support_incomplete.mdwn
new file mode 100644
index 0000000000..d52871df29
--- /dev/null
+++ b/doc/bugs/GIT_DIR_support_incomplete.mdwn
@@ -0,0 +1,15 @@
+`GIT_DIR` support isn't right. Git does not look for `GIT_DIR/.git`;
+git-annex does.
+
+Also, to support this scenario, support for core.worktree needs to be added
+as well:
+
+	mkdir repo workdir
+	git --work-tree=$PWD/workdir --git-dir=$PWD/repo init
+	export GIT_DIR=$PWD/repo
+	git status
+	# ok
+	git annex init "new repo"
+	# fail
+
+--[[Joey]] 

From bb4f31a0ee496ffb83d31cc56f8827e47605d763 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 18 May 2012 16:38:26 -0400
Subject: [PATCH 128/220] Clean up handling of git directory and git worktree.

Baked into the code was an assumption that a repository's git directory
could be determined by adding ".git" to its work tree (or nothing for bare
repos). That fails when core.worktree, or GIT_DIR and GIT_WORK_TREE are
used to separate the two.

This was attacked at the type level, by storing the gitdir and worktree
separately, so Nothing for the worktree means a bare repo.

A complication arose because we don't learn where a repository is bare
until its configuration is read. So another Location type handles
repositories that have not had their config read yet. I am not entirely
happy with this being a Location type, rather than representing them
entirely separate from the Git type. The new code is not worse than the
old, but better types could enforce more safety.

Added support for core.worktree. Overriding it with -c isn't supported
because it's not really clear what to do if a git repo's config is read, is
not bare, and is then overridden to bare. What is the right git directory
in this case? I will worry about this if/when someone has a use case for
overriding core.worktree with -c. (See Git.Config.updateLocation)

Also removed and renamed some functions like gitDir and workTree that
misused git's terminology.

One minor regression is known: git annex add in a bare repository does not
print a nice error message, but runs git ls-files in a way that fails
earlier with a less nice error message. This is because before --work-tree
was always passed to git commands, even in a bare repo, while now it's not.
---
 Annex/Content.hs     |  3 +-
 Annex/Ssh.hs         |  4 +--
 Command/Log.hs       |  2 +-
 Command/Map.hs       |  8 ++---
 Command/Unused.hs    |  2 +-
 Config.hs            |  2 +-
 Git.hs               | 77 ++++++++++++++++++--------------------------
 Git/Command.hs       | 13 ++++----
 Git/Config.hs        | 62 +++++++++++++++++++++++++----------
 Git/Construct.hs     | 10 +++---
 Git/LsFiles.hs       |  2 +-
 Git/Types.hs         | 20 +++++++++---
 Init.hs              |  2 +-
 Locations.hs         | 14 +++-----
 Remote/Bup.hs        |  2 +-
 Remote/Helper/Ssh.hs |  2 +-
 Upgrade/V1.hs        |  4 +--
 Upgrade/V2.hs        |  2 +-
 Utility/Directory.hs | 14 ++++++++
 debian/changelog     |  1 +
 git-union-merge.hs   |  2 +-
 21 files changed, 144 insertions(+), 104 deletions(-)

diff --git a/Annex/Content.hs b/Annex/Content.hs
index 2142d1f09d..26b332e24c 100644
--- a/Annex/Content.hs
+++ b/Annex/Content.hs
@@ -34,6 +34,7 @@ import Common.Annex
 import Logs.Location
 import Annex.UUID
 import qualified Git
+import qualified Git.Config
 import qualified Annex
 import qualified Annex.Queue
 import qualified Annex.Branch
@@ -303,7 +304,7 @@ saveState oneshot = doSideAction $ do
 		ifM alwayscommit
 			( Annex.Branch.commit "update" , Annex.Branch.stage)
 	where
-		alwayscommit = fromMaybe True . Git.configTrue
+		alwayscommit = fromMaybe True . Git.Config.isTrue
 			<$> getConfig (annexConfig "alwayscommit") ""
 
 {- Downloads content from any of a list of urls. -}
diff --git a/Annex/Ssh.hs b/Annex/Ssh.hs
index f0824b1191..8bd4fe33ab 100644
--- a/Annex/Ssh.hs
+++ b/Annex/Ssh.hs
@@ -14,7 +14,7 @@ import qualified Data.Map as M
 
 import Common.Annex
 import Annex.LockPool
-import qualified Git
+import qualified Git.Config
 import Config
 import qualified Build.SysConfig as SysConfig
 import Annex.Perms
@@ -47,7 +47,7 @@ sshInfo (host, port) = ifM caching
 	)
 	where
 		caching = fromMaybe SysConfig.sshconnectioncaching 
-			. Git.configTrue
+			. Git.Config.isTrue
 			<$> getConfig (annexConfig "sshcaching") ""
 
 cacheParams :: FilePath -> [CommandParam]
diff --git a/Command/Log.hs b/Command/Log.hs
index d78b602067..aa39aea9c7 100644
--- a/Command/Log.hs
+++ b/Command/Log.hs
@@ -133,7 +133,7 @@ compareChanges format changes = concatMap diff $ zip changes (drop 1 changes)
  - *lot* for newish files. -}
 getLog :: Key -> [CommandParam] -> Annex [String]
 getLog key os = do
-	top <- fromRepo Git.workTree
+	top <- fromRepo Git.repoPath
 	p <- liftIO $ relPathCwdToFile top
 	let logfile = p  Logs.Location.logFile key
 	inRepo $ pipeNullSplit $
diff --git a/Command/Map.hs b/Command/Map.hs
index bdb86f95a5..86e9609a7e 100644
--- a/Command/Map.hs
+++ b/Command/Map.hs
@@ -156,14 +156,14 @@ absRepo :: Git.Repo -> Git.Repo -> Annex Git.Repo
 absRepo reference r
 	| Git.repoIsUrl reference = return $ Git.Construct.localToUrl reference r
 	| Git.repoIsUrl r = return r
-	| otherwise = liftIO $ Git.Construct.fromAbsPath =<< absPath (Git.workTree r)
+	| otherwise = liftIO $ Git.Construct.fromAbsPath =<< absPath (Git.repoPath r)
 
 {- Checks if two repos are the same. -}
 same :: Git.Repo -> Git.Repo -> Bool
 same a b
-	| both Git.repoIsSsh = matching Git.Url.authority && matching Git.workTree
+	| both Git.repoIsSsh = matching Git.Url.authority && matching Git.repoPath
 	| both Git.repoIsUrl && neither Git.repoIsSsh = matching show
-	| neither Git.repoIsSsh = matching Git.workTree
+	| neither Git.repoIsSsh = matching Git.repoPath
 	| otherwise = False
 		
 	where
@@ -210,7 +210,7 @@ tryScan r
 			where
 				sshcmd = cddir ++ " && " ++
 					"git config --null --list"
-				dir = Git.workTree r
+				dir = Git.repoPath r
 				cddir
 					| "/~" `isPrefixOf` dir =
 						let (userhome, reldir) = span (/= '/') (drop 1 dir)
diff --git a/Command/Unused.hs b/Command/Unused.hs
index f5ee452a80..1224d05457 100644
--- a/Command/Unused.hs
+++ b/Command/Unused.hs
@@ -231,7 +231,7 @@ withKeysReferenced' :: v -> (Key -> v -> Annex v) -> Annex v
 withKeysReferenced' initial a = go initial =<< files
 	where
 		files = do
-			top <- fromRepo Git.workTree
+			top <- fromRepo Git.repoPath
 			inRepo $ LsFiles.inRepo [top]
 		go v [] = return v
 		go v (f:fs) = do
diff --git a/Config.hs b/Config.hs
index 5f1ac8bb20..bb57ab675e 100644
--- a/Config.hs
+++ b/Config.hs
@@ -84,7 +84,7 @@ prop_cost_sane = False `notElem`
 
 {- Checks if a repo should be ignored. -}
 repoNotIgnored :: Git.Repo -> Annex Bool
-repoNotIgnored r = not . fromMaybe False . Git.configTrue
+repoNotIgnored r = not . fromMaybe False . Git.Config.isTrue
 	<$> getRemoteConfig r "ignore" ""
 
 {- If a value is specified, it is used; otherwise the default is looked up
diff --git a/Git.hs b/Git.hs
index 4278e9fcf2..7d64205634 100644
--- a/Git.hs
+++ b/Git.hs
@@ -3,7 +3,7 @@
  - This is written to be completely independant of git-annex and should be
  - suitable for other uses.
  -
- - Copyright 2010, 2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -17,19 +17,17 @@ module Git (
 	repoIsUrl,
 	repoIsSsh,
 	repoIsHttp,
+	repoIsLocal,
 	repoIsLocalBare,
 	repoDescribe,
 	repoLocation,
-	workTree,
-	gitDir,
-	configTrue,
+	repoPath,
+	localGitDir,
 	attributes,
 	hookPath,
 	assertLocal,
 ) where
 
-import qualified Data.Map as M
-import Data.Char
 import Network.URI (uriPath, uriScheme, unEscapeString)
 import System.Posix.Files
 
@@ -41,15 +39,34 @@ import Utility.FileMode
 repoDescribe :: Repo -> String
 repoDescribe Repo { remoteName = Just name } = name
 repoDescribe Repo { location = Url url } = show url
-repoDescribe Repo { location = Dir dir } = dir
+repoDescribe Repo { location = Local { worktree = Just dir } } = dir
+repoDescribe Repo { location = Local { gitdir = dir } } = dir
+repoDescribe Repo { location = LocalUnknown dir } = dir
 repoDescribe Repo { location = Unknown } = "UNKNOWN"
 
 {- Location of the repo, either as a path or url. -}
 repoLocation :: Repo -> String
 repoLocation Repo { location = Url url } = show url
-repoLocation Repo { location = Dir dir } = dir
+repoLocation Repo { location = Local { worktree = Just dir } } = dir
+repoLocation Repo { location = Local { gitdir = dir } } = dir
+repoLocation Repo { location = LocalUnknown dir } = dir
 repoLocation Repo { location = Unknown } = undefined
 
+{- Path to a repository. For non-bare, this is the worktree, for bare, 
+ - it's the gitdir, and for URL repositories, is the path on the remote
+ - host. -}
+repoPath :: Repo -> FilePath
+repoPath Repo { location = Url u } = unEscapeString $ uriPath u
+repoPath Repo { location = Local { worktree = Just d } } = d
+repoPath Repo { location = Local { gitdir = d } } = d
+repoPath Repo { location = LocalUnknown dir } = dir
+repoPath Repo { location = Unknown } = undefined
+
+{- Path to a local repository's .git directory. -}
+localGitDir :: Repo -> FilePath
+localGitDir Repo { location = Local { gitdir = d } } = d
+localGitDir _ = undefined
+
 {- Some code needs to vary between URL and normal repos,
  - or bare and non-bare, these functions help with that. -}
 repoIsUrl :: Repo -> Bool
@@ -74,11 +91,12 @@ repoIsHttp Repo { location = Url url }
 	| otherwise = False
 repoIsHttp _ = False
 
-configAvail ::Repo -> Bool
-configAvail Repo { config = c } = c /= M.empty
+repoIsLocal :: Repo -> Bool
+repoIsLocal Repo { location = Local { } } = True
+repoIsLocal _ = False
 
 repoIsLocalBare :: Repo -> Bool
-repoIsLocalBare r@(Repo { location = Dir _ }) = configAvail r && configBare r
+repoIsLocalBare Repo { location = Local { worktree = Nothing } } = True
 repoIsLocalBare _ = False
 
 assertLocal :: Repo -> a -> a
@@ -90,49 +108,18 @@ assertLocal repo action
 		]
 	| otherwise = action
 
-configBare :: Repo -> Bool
-configBare repo = maybe unknown (fromMaybe False . configTrue) $
-	M.lookup "core.bare" $ config repo
-	where
-		unknown = error $ "it is not known if git repo " ++
-			repoDescribe repo ++
-			" is a bare repository; config not read"
-
 {- Path to a repository's gitattributes file. -}
 attributes :: Repo -> FilePath
 attributes repo
-	| configBare repo = workTree repo ++ "/info/.gitattributes"
-	| otherwise = workTree repo ++ "/.gitattributes"
-
-{- Path to a repository's .git directory. -}
-gitDir :: Repo -> FilePath
-gitDir repo
-	| configBare repo = workTree repo
-	| otherwise = workTree repo  ".git"
+	| repoIsLocalBare repo = repoPath repo ++ "/info/.gitattributes"
+	| otherwise = repoPath repo ++ "/.gitattributes"
 
 {- Path to a given hook script in a repository, only if the hook exists
  - and is executable. -}
 hookPath :: String -> Repo -> IO (Maybe FilePath)
 hookPath script repo = do
-	let hook = gitDir repo  "hooks"  script
+	let hook = localGitDir repo  "hooks"  script
 	ifM (catchBoolIO $ isexecutable hook)
 		( return $ Just hook , return Nothing )
 	where
 		isexecutable f = isExecutable . fileMode <$> getFileStatus f
-
-{- Path to a repository's --work-tree, that is, its top.
- -
- - Note that for URL repositories, this is the path on the remote host. -}
-workTree :: Repo -> FilePath
-workTree Repo { location = Url u } = unEscapeString $ uriPath u
-workTree Repo { location = Dir d } = d
-workTree Repo { location = Unknown } = undefined
-
-{- Checks if a string from git config is a true value. -}
-configTrue :: String -> Maybe Bool
-configTrue s
-	| s' == "true" = Just True
-	| s' == "false" = Just False
-	| otherwise = Nothing
-	where
-		s' = map toLower s
diff --git a/Git/Command.hs b/Git/Command.hs
index bb82d13395..35f0838ba9 100644
--- a/Git/Command.hs
+++ b/Git/Command.hs
@@ -1,6 +1,6 @@
 {- running git commands
  -
- - Copyright 2010, 2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -18,11 +18,12 @@ import Git.Types
 
 {- Constructs a git command line operating on the specified repo. -}
 gitCommandLine :: [CommandParam] -> Repo -> [CommandParam]
-gitCommandLine params repo@(Repo { location = Dir _ } ) =
-	-- force use of specified repo via --git-dir and --work-tree
-	[ Param ("--git-dir=" ++ gitDir repo)
-	, Param ("--work-tree=" ++ workTree repo)
-	] ++ params
+gitCommandLine params Repo { location = l@(Local _ _ ) } = setdir : settree ++ params
+	where
+		setdir = Param $ "--git-dir=" ++ gitdir l
+		settree = case worktree l of
+			Nothing -> []
+			Just t -> [Param $ "--work-tree=" ++ t]
 gitCommandLine _ repo = assertLocal repo $ error "internal"
 
 {- Runs git in the specified repo. -}
diff --git a/Git/Config.hs b/Git/Config.hs
index 38b9ade455..e37b437071 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -1,15 +1,14 @@
 {- git repository configuration handling
  -
- - Copyright 2010,2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
 
 module Git.Config where
 
-import System.Posix.Directory
-import Control.Exception (bracket_)
 import qualified Data.Map as M
+import Data.Char
 
 import Common
 import Git
@@ -30,17 +29,14 @@ getMaybe key repo = M.lookup key (config repo)
 
 {- Runs git config and populates a repo with its config. -}
 read :: Repo -> IO Repo
-read repo@(Repo { location = Dir d }) = bracketcd d $
-	{- Cannot use pipeRead because it relies on the config having
-	   been already read. Instead, chdir to the repo. -}
+read repo@(Repo { location = Local { gitdir = d } }) = read' repo d
+read repo@(Repo { location = LocalUnknown d }) = read' repo d
+read r = assertLocal r $ error "internal"
+{- Cannot use pipeRead because it relies on the config having
+   been already read. Instead, chdir to the repo. -}
+read' :: Repo -> FilePath -> IO Repo
+read' repo d = bracketCd d $
 	pOpen ReadFromPipe "git" ["config", "--null", "--list"] $ hRead repo
-	where
-		bracketcd to a = bracketcd' to a =<< getCurrentDirectory
-		bracketcd' to a cwd 
-			| dirContains to cwd = a
-			| otherwise = bracket_ (changeWorkingDirectory to) (changeWorkingDirectory cwd) a
-read r = assertLocal r $
-	error $ "internal error; trying to read config of " ++ show r
 
 {- Reads git config from a handle and populates a repo with it. -}
 hRead :: Repo -> Handle -> IO Repo
@@ -48,19 +44,42 @@ hRead repo h = do
 	val <- hGetContentsStrict h
 	store val repo
 
-{- Stores a git config into a repo, returning the new version of the repo.
- - The git config may be multiple lines, or a single line. Config settings
- - can be updated inrementally. -}
+{- Stores a git config into a Repo, returning the new version of the Repo.
+ - The git config may be multiple lines, or a single line.
+ - Config settings can be updated incrementally.
+ -}
 store :: String -> Repo -> IO Repo
 store s repo = do
 	let c = parse s
-	let repo' = repo
+	let repo' = updateLocation $ repo
 		{ config = (M.map Prelude.head c) `M.union` config repo
 		, fullconfig = M.unionWith (++) c (fullconfig repo)
 		}
+	print repo'
 	rs <- Git.Construct.fromRemotes repo'
 	return $ repo' { remotes = rs }
 
+{- Updates the location of a repo, based on its configuration.
+ -
+ - Git.Construct makes LocalUknown repos, of which only a directory is
+ - known. Once the config is read, this can be fixed up to a Local repo, 
+ - based on the core.bare and core.worktree settings.
+ -}
+updateLocation :: Repo -> Repo
+updateLocation r = go $ location r
+	where
+		go (LocalUnknown d)
+			| isbare = ret $ Local d Nothing
+			| otherwise = ret $ Local (d  ".git") (Just d)
+		go l@(Local {}) = ret l
+		go _ = r
+		isbare = fromMaybe False $ isTrue =<< getMaybe "core.bare" r
+		ret l = r { location = l' }
+			where
+				l' = maybe l (setworktree l) $
+					getMaybe "core.worktree" r
+		setworktree l t = l { worktree = Just t }
+
 {- Parses git config --list or git config --null --list output into a
  - config map. -}
 parse :: String -> M.Map String [String]
@@ -74,3 +93,12 @@ parse s
 		ls = lines s
 		sep c = M.fromListWith (++) . map (\(k,v) -> (k, [v])) .
 			map (separate (== c))
+
+{- Checks if a string from git config is a true value. -}
+isTrue :: String -> Maybe Bool
+isTrue s
+	| s' == "true" = Just True
+	| s' == "false" = Just False
+	| otherwise = Nothing
+	where
+		s' = map toLower s
diff --git a/Git/Construct.hs b/Git/Construct.hs
index 3f3ea97476..45ea0f64d8 100644
--- a/Git/Construct.hs
+++ b/Git/Construct.hs
@@ -1,6 +1,6 @@
 {- Construction of Git Repo objects
  -
- - Copyright 2010,2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -58,7 +58,7 @@ fromCurrent = do
 fromCwd :: IO Repo
 fromCwd = getCurrentDirectory >>= seekUp isRepoTop >>= maybe norepo makerepo
 	where
-		makerepo = newFrom . Dir
+		makerepo = newFrom . LocalUnknown
 		norepo = error "Not in a git repository."
 
 {- Local Repo constructor, accepts a relative or absolute path. -}
@@ -74,7 +74,7 @@ fromAbsPath dir
 	| otherwise =
 		error $ "internal error, " ++ dir ++ " is not absolute"
 	where
-		ret = newFrom . Dir
+		ret = newFrom . LocalUnknown
  		{- Git always looks for "dir.git" in preference to
 		 - to "dir", even if dir ends in a "/". -}
 		canondir = dropTrailingPathSeparator dir
@@ -122,7 +122,7 @@ localToUrl reference r
 		absurl =
 			Url.scheme reference ++ "//" ++
 			Url.authority reference ++
-			workTree r
+			repoPath r
 
 {- Calculates a list of a repo's configured remotes, by parsing its config. -}
 fromRemotes :: Repo -> IO [Repo]
@@ -191,7 +191,7 @@ fromRemoteLocation s repo = gen $ calcloc s
 fromRemotePath :: FilePath -> Repo -> IO Repo
 fromRemotePath dir repo = do
 	dir' <- expandTilde dir
-	fromAbsPath $ workTree repo  dir'
+	fromAbsPath $ repoPath repo  dir'
 
 {- Git remotes can have a directory that is specified relative
  - to the user's home directory, or that contains tilde expansions.
diff --git a/Git/LsFiles.hs b/Git/LsFiles.hs
index 201d76d1d4..06d4b9f44f 100644
--- a/Git/LsFiles.hs
+++ b/Git/LsFiles.hs
@@ -69,7 +69,7 @@ typeChanged' ps l repo = do
 	fs <- pipeNullSplit (prefix ++ ps ++ suffix) repo
 	-- git diff returns filenames relative to the top of the git repo;
 	-- convert to filenames relative to the cwd, like git ls-files.
-	let top = workTree repo
+	let top = repoPath repo
 	cwd <- getCurrentDirectory
 	return $ map (\f -> relPathDirToFile cwd $ top  f) fs
 	where
diff --git a/Git/Types.hs b/Git/Types.hs
index 6063ad213f..deb14ebd48 100644
--- a/Git/Types.hs
+++ b/Git/Types.hs
@@ -1,6 +1,6 @@
 {- git data types
  -
- - Copyright 2010,2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -10,9 +10,21 @@ module Git.Types where
 import Network.URI
 import qualified Data.Map as M
 
-{- There are two types of repositories; those on local disk and those
- - accessed via an URL. -}
-data RepoLocation = Dir FilePath | Url URI | Unknown
+{- Support repositories on local disk, and repositories accessed via an URL.
+ -
+ - Repos on local disk have a git directory, and unless bare, a worktree.
+ -
+ - A local repo may not have had its config read yet, in which case all
+ - that's known about it is its path.
+ -
+ - Finally, an Unknown repository may be known to exist, but nothing
+ - else known about it.
+ -}
+data RepoLocation
+	= Local { gitdir :: FilePath, worktree :: Maybe FilePath }
+	| LocalUnknown FilePath
+	| Url URI
+	| Unknown
 	deriving (Show, Eq)
 
 data Repo = Repo {
diff --git a/Init.hs b/Init.hs
index a0e16e8815..bddcc696e0 100644
--- a/Init.hs
+++ b/Init.hs
@@ -72,7 +72,7 @@ unlessBare :: Annex () -> Annex ()
 unlessBare = unlessM $ fromRepo Git.repoIsLocalBare
 
 preCommitHook :: Annex FilePath
-preCommitHook = () <$> fromRepo Git.gitDir <*> pure "hooks/pre-commit"
+preCommitHook = () <$> fromRepo Git.localGitDir <*> pure "hooks/pre-commit"
 
 preCommitScript :: String
 preCommitScript = 
diff --git a/Locations.hs b/Locations.hs
index 67abf2166a..46a85e0ee1 100644
--- a/Locations.hs
+++ b/Locations.hs
@@ -85,28 +85,24 @@ gitAnnexLocation key r
 	| Git.repoIsLocalBare r =
 		{- Bare repositories default to hashDirLower for new
 		 - content, as it's more portable. -}
-		check (map inrepo $ annexLocations key)
+		check $ map inrepo $ annexLocations key
 	| otherwise =
 		{- Non-bare repositories only use hashDirMixed, so
 		 - don't need to do any work to check if the file is
 		 - present. -}
-		return $ inrepo ".git"  annexLocation key hashDirMixed
+		return $ inrepo $ annexLocation key hashDirMixed
 	where
-		inrepo d = Git.workTree r  d
+		inrepo d = Git.localGitDir r  d
 		check locs@(l:_) = fromMaybe l <$> firstM doesFileExist locs
 		check [] = error "internal"
 
 {- The annex directory of a repository. -}
 gitAnnexDir :: Git.Repo -> FilePath
-gitAnnexDir r
-	| Git.repoIsLocalBare r = addTrailingPathSeparator $ Git.workTree r  annexDir
-	| otherwise = addTrailingPathSeparator $ Git.workTree r  ".git"  annexDir
+gitAnnexDir r = addTrailingPathSeparator $ Git.localGitDir r  annexDir
 
 {- The part of the annex directory where file contents are stored. -}
 gitAnnexObjectDir :: Git.Repo -> FilePath
-gitAnnexObjectDir r
-	| Git.repoIsLocalBare r = addTrailingPathSeparator $ Git.workTree r  objectDir
-	| otherwise = addTrailingPathSeparator $ Git.workTree r  ".git"  objectDir
+gitAnnexObjectDir r = addTrailingPathSeparator $ Git.localGitDir r  objectDir
 
 {- .git/annex/tmp/ is used for temp files -}
 gitAnnexTmpDir :: Git.Repo -> FilePath
diff --git a/Remote/Bup.hs b/Remote/Bup.hs
index 1081815944..3e7e9211f9 100644
--- a/Remote/Bup.hs
+++ b/Remote/Bup.hs
@@ -184,7 +184,7 @@ storeBupUUID u buprepo = do
 
 onBupRemote :: Git.Repo -> (FilePath -> [CommandParam] -> IO a) -> FilePath -> [CommandParam] -> Annex a
 onBupRemote r a command params = do
-	let dir = shellEscape (Git.workTree r)
+	let dir = shellEscape (Git.repoPath r)
 	sshparams <- sshToRepo r [Param $
 			"cd " ++ dir ++ " && " ++ unwords (command : toCommand params)]
 	liftIO $ a "ssh" sshparams
diff --git a/Remote/Helper/Ssh.hs b/Remote/Helper/Ssh.hs
index 4c5eef0e6c..f6742b89f6 100644
--- a/Remote/Helper/Ssh.hs
+++ b/Remote/Helper/Ssh.hs
@@ -34,7 +34,7 @@ git_annex_shell r command params
 		return $ Just ("ssh", sshparams)
 	| otherwise = return Nothing
 	where
-		dir = Git.workTree r
+		dir = Git.repoPath r
 		shellcmd = "git-annex-shell"
 		shellopts = Param command : File dir : params
 		sshcmd uuid = unwords $
diff --git a/Upgrade/V1.hs b/Upgrade/V1.hs
index ddf0728b61..280742f062 100644
--- a/Upgrade/V1.hs
+++ b/Upgrade/V1.hs
@@ -82,7 +82,7 @@ moveContent = do
 updateSymlinks :: Annex ()
 updateSymlinks = do
 	showAction "updating symlinks"
-	top <- fromRepo Git.workTree
+	top <- fromRepo Git.repoPath
 	files <- inRepo $ LsFiles.inRepo [top]
 	forM_ files fixlink
 	where
@@ -236,4 +236,4 @@ stateDir :: FilePath
 stateDir = addTrailingPathSeparator ".git-annex"
 
 gitStateDir :: Git.Repo -> FilePath
-gitStateDir repo = addTrailingPathSeparator $ Git.workTree repo  stateDir
+gitStateDir repo = addTrailingPathSeparator $ Git.repoPath repo  stateDir
diff --git a/Upgrade/V2.hs b/Upgrade/V2.hs
index c57b0bf685..202ba5b167 100644
--- a/Upgrade/V2.hs
+++ b/Upgrade/V2.hs
@@ -134,4 +134,4 @@ gitAttributesUnWrite repo = do
 stateDir :: FilePath
 stateDir = addTrailingPathSeparator ".git-annex"
 gitStateDir :: Git.Repo -> FilePath
-gitStateDir repo = addTrailingPathSeparator $ Git.workTree repo  stateDir
+gitStateDir repo = addTrailingPathSeparator $ Git.repoPath repo  stateDir
diff --git a/Utility/Directory.hs b/Utility/Directory.hs
index e6622d31ee..3041361dfd 100644
--- a/Utility/Directory.hs
+++ b/Utility/Directory.hs
@@ -15,11 +15,14 @@ import Control.Monad
 import Control.Monad.IfElse
 import System.FilePath
 import Control.Applicative
+import Control.Exception (bracket_)
+import System.Posix.Directory
 
 import Utility.SafeCommand
 import Utility.TempFile
 import Utility.Exception
 import Utility.Monad
+import Utility.Path
 
 {- Lists the contents of a directory.
  - Unlike getDirectoryContents, paths are not relative to the directory. -}
@@ -60,3 +63,14 @@ moveFile src dest = tryIO (rename src dest) >>= onrename
 			case r of
 				(Left _) -> return False
 				(Right s) -> return $ isDirectory s
+
+{- Runs an action in another directory. -}
+bracketCd :: FilePath -> IO a -> IO a
+bracketCd dir a = go =<< getCurrentDirectory
+	where
+		go cwd
+			| dirContains dir cwd = a
+			| otherwise = bracket_
+				(changeWorkingDirectory dir)
+				(changeWorkingDirectory cwd)
+				a
diff --git a/debian/changelog b/debian/changelog
index a4e2b8b3e9..4e61445c8d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -2,6 +2,7 @@ git-annex (3.20120512) UNRELEASED; urgency=low
 
   * Pass -a to cp even when it supports --reflink=auto, to preserve
     permissions.
+  * Clean up handling of git directory and git worktree.
 
  -- Joey Hess   Tue, 15 May 2012 14:17:49 -0400
 
diff --git a/git-union-merge.hs b/git-union-merge.hs
index f44136bfc1..182d8cf790 100644
--- a/git-union-merge.hs
+++ b/git-union-merge.hs
@@ -22,7 +22,7 @@ usage :: IO a
 usage = error $ "bad parameters\n\n" ++ header
 
 tmpIndex :: Git.Repo -> FilePath
-tmpIndex g = Git.gitDir g  "index.git-union-merge"
+tmpIndex g = Git.localGitDir g  "index.git-union-merge"
 
 setup :: Git.Repo -> IO ()
 setup = cleanup -- idempotency

From eb6cb1b87f2d7016ddd4386e2a3bb20d8ea3c036 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 18 May 2012 18:20:53 -0400
Subject: [PATCH 129/220] Add support for core.worktree, and fix support for
 GIT_WORK_TREE and GIT_DIR.

The environment needs to override git-config. Changed when git config is
read, and avoid rereading it once it's been read.

chdir for both worktree settings.
---
 Annex.hs                                 |  3 +-
 Git/Config.hs                            | 15 ++++---
 Git/Construct.hs                         | 27 +-----------
 Git/CurrentRepo.hs                       | 54 ++++++++++++++++++++++++
 GitAnnex.hs                              |  4 +-
 Remote/Git.hs                            |  8 +---
 debian/changelog                         |  2 +
 doc/bugs/GIT_DIR_support_incomplete.mdwn |  2 +
 git-union-merge.hs                       |  4 +-
 test.hs                                  |  4 +-
 10 files changed, 79 insertions(+), 44 deletions(-)
 create mode 100644 Git/CurrentRepo.hs

diff --git a/Annex.hs b/Annex.hs
index d1509d4bd2..a9cc680125 100644
--- a/Annex.hs
+++ b/Annex.hs
@@ -124,7 +124,8 @@ newState gitrepo = AnnexState
 	, cleanup = M.empty
 	}
 
-{- Create and returns an Annex state object for the specified git repo. -}
+{- Makes an Annex state object for the specified git repo.
+ - Ensures the config is read, if it was not already. -}
 new :: Git.Repo -> IO AnnexState
 new gitrepo = newState <$> Git.Config.read gitrepo
 
diff --git a/Git/Config.hs b/Git/Config.hs
index e37b437071..2fa685a11d 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -27,16 +27,20 @@ getList key repo = M.findWithDefault [] key (fullconfig repo)
 getMaybe :: String -> Repo -> Maybe String
 getMaybe key repo = M.lookup key (config repo)
 
-{- Runs git config and populates a repo with its config. -}
+{- Runs git config and populates a repo with its config.
+ - Cannot use pipeRead because it relies on the config having been already
+ - read. Instead, chdir to the repo.
+ -}
 read :: Repo -> IO Repo
 read repo@(Repo { location = Local { gitdir = d } }) = read' repo d
 read repo@(Repo { location = LocalUnknown d }) = read' repo d
 read r = assertLocal r $ error "internal"
-{- Cannot use pipeRead because it relies on the config having
-   been already read. Instead, chdir to the repo. -}
 read' :: Repo -> FilePath -> IO Repo
-read' repo d = bracketCd d $
-	pOpen ReadFromPipe "git" ["config", "--null", "--list"] $ hRead repo
+read' repo@(Repo { config = c}) d
+	| c == M.empty = bracketCd d $
+		pOpen ReadFromPipe "git" ["config", "--null", "--list"] $
+			hRead repo
+	| otherwise = return repo -- config already read
 
 {- Reads git config from a handle and populates a repo with it. -}
 hRead :: Repo -> Handle -> IO Repo
@@ -55,7 +59,6 @@ store s repo = do
 		{ config = (M.map Prelude.head c) `M.union` config repo
 		, fullconfig = M.unionWith (++) c (fullconfig repo)
 		}
-	print repo'
 	rs <- Git.Construct.fromRemotes repo'
 	return $ repo' { remotes = rs }
 
diff --git a/Git/Construct.hs b/Git/Construct.hs
index 45ea0f64d8..b809d7318a 100644
--- a/Git/Construct.hs
+++ b/Git/Construct.hs
@@ -6,7 +6,6 @@
  -}
 
 module Git.Construct (
-	fromCurrent,
 	fromCwd,
 	fromAbsPath,
 	fromPath,
@@ -21,8 +20,6 @@ module Git.Construct (
 ) where
 
 import System.Posix.User
-import System.Posix.Env (getEnv, unsetEnv)
-import System.Posix.Directory (changeWorkingDirectory)
 import qualified Data.Map as M hiding (map, split)
 import Network.URI
 
@@ -31,28 +28,6 @@ import Git.Types
 import Git
 import qualified Git.Url as Url
 
-{- Finds the current git repository.
- -
- - GIT_DIR can override the location of the .git directory.
- -
- - When GIT_WORK_TREE is set, chdir to it, so that anything using
- - this repository runs in the right location. However, this chdir is
- - done after determining GIT_DIR; git does not let GIT_WORK_TREE
- - influence the git directory.
- -
- - Both environment variables are unset, to avoid confusing other git
- - commands that also look at them. This would particularly be a problem
- - when GIT_DIR is relative and we chdir for GIT_WORK_TREE. Instead,
- - the Git module passes --work-tree and --git-dir to git commands it runs.
- -}
-fromCurrent :: IO Repo
-fromCurrent = do
-	r <- maybe fromCwd fromPath =<< getEnv "GIT_DIR"
-	maybe noop changeWorkingDirectory =<< getEnv "GIT_WORK_TREE"
-	unsetEnv "GIT_DIR"
-	unsetEnv "GIT_WORK_TREE"
-	return r
-
 {- Finds the git repository used for the Cwd, which may be in a parent
  - directory. -}
 fromCwd :: IO Repo
@@ -251,3 +226,5 @@ newFrom l = return Repo
 	, remotes = []
 	, remoteName = Nothing
 	}
+
+
diff --git a/Git/CurrentRepo.hs b/Git/CurrentRepo.hs
new file mode 100644
index 0000000000..4325f452c0
--- /dev/null
+++ b/Git/CurrentRepo.hs
@@ -0,0 +1,54 @@
+{- The current git repository.
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Git.CurrentRepo where
+
+import System.Posix.Directory (changeWorkingDirectory)
+import System.Posix.Env (getEnv, unsetEnv)
+
+import Common
+import Git.Types
+import Git.Construct
+import qualified Git.Config
+
+{- Gets the current git repository.
+ -
+ - Honors GIT_DIR and GIT_WORK_TREE.
+ - Both environment variables are unset, to avoid confusing other git
+ - commands that also look at them. Instead, the Git module passes
+ - --work-tree and --git-dir to git commands it runs.
+ -
+ - When GIT_WORK_TREE or core.worktree are set, changes the working
+ - directory if necessary to ensure it is within the repository's work
+ - tree. While not needed for git commands, this is useful for anything
+ - else that looks for files in the worktree.
+ -}
+get :: IO Repo
+get = do
+	gd <- takeenv "GIT_DIR"
+	r <- configure gd =<< maybe fromCwd fromPath gd
+	wt <- maybe (worktree $ location r) Just <$> takeenv "GIT_WORK_TREE"
+	case wt of
+		Nothing -> return r
+		Just d -> do
+			changeWorkingDirectory d
+			return $ addworktree wt r
+	where
+		takeenv s = do
+			v <- getEnv s
+			when (isJust v) $
+				unsetEnv s
+			return v
+		configure Nothing r = Git.Config.read r
+		configure (Just d) r = do
+			r' <- Git.Config.read r
+			-- Let GIT_DIR override the default gitdir.
+			return $ changelocation r' $
+				Local { gitdir = d, worktree = worktree (location r') }
+		addworktree w r = changelocation r $
+			Local { gitdir = gitdir (location r), worktree = w }
+		changelocation r l = r { location = l }
diff --git a/GitAnnex.hs b/GitAnnex.hs
index 0e707b1868..9910e33d21 100644
--- a/GitAnnex.hs
+++ b/GitAnnex.hs
@@ -11,7 +11,7 @@ import System.Console.GetOpt
 
 import Common.Annex
 import qualified Git.Config
-import qualified Git.Construct
+import qualified Git.CurrentRepo
 import CmdLine
 import Command
 import Types.TrustLevel
@@ -133,4 +133,4 @@ header :: String
 header = "Usage: git-annex command [option ..]"
 
 run :: [String] -> IO ()
-run args = dispatch True args cmds options header Git.Construct.fromCurrent
+run args = dispatch True args cmds options header Git.CurrentRepo.get
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 35928b96cb..79439b784f 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -179,12 +179,8 @@ repoAvail r
  - monad using that repository. -}
 onLocal :: Git.Repo -> Annex a -> IO a
 onLocal r a = do
-	-- Avoid re-reading the repository's configuration if it was
-	-- already read.
-	state <- if M.null $ Git.config r
-		then Annex.new r
-		else return $ Annex.newState r
-	Annex.eval state $ do
+	s <- Annex.new r
+	Annex.eval s $ do
 		-- No need to update the branch; its data is not used
 		-- for anything onLocal is used to do.
 		Annex.BranchState.disableUpdate
diff --git a/debian/changelog b/debian/changelog
index 4e61445c8d..586077878b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -3,6 +3,8 @@ git-annex (3.20120512) UNRELEASED; urgency=low
   * Pass -a to cp even when it supports --reflink=auto, to preserve
     permissions.
   * Clean up handling of git directory and git worktree.
+  * Add support for core.worktree, and fix support for GIT_WORK_TREE and
+    GIT_DIR.
 
  -- Joey Hess   Tue, 15 May 2012 14:17:49 -0400
 
diff --git a/doc/bugs/GIT_DIR_support_incomplete.mdwn b/doc/bugs/GIT_DIR_support_incomplete.mdwn
index d52871df29..1b9738c4f7 100644
--- a/doc/bugs/GIT_DIR_support_incomplete.mdwn
+++ b/doc/bugs/GIT_DIR_support_incomplete.mdwn
@@ -13,3 +13,5 @@ as well:
 	# fail
 
 --[[Joey]] 
+
+> [[fixed|done]] --[[Joey]] 
diff --git a/git-union-merge.hs b/git-union-merge.hs
index 182d8cf790..2c2e7a46bd 100644
--- a/git-union-merge.hs
+++ b/git-union-merge.hs
@@ -10,7 +10,7 @@ import System.Environment
 import Common
 import qualified Git.UnionMerge
 import qualified Git.Config
-import qualified Git.Construct
+import qualified Git.CurrentRepo
 import qualified Git.Branch
 import qualified Git.Index
 import qualified Git
@@ -40,7 +40,7 @@ parseArgs = do
 main :: IO ()
 main = do
 	[aref, bref, newref] <- map Git.Ref <$> parseArgs
-	g <- Git.Config.read =<< Git.Construct.fromCurrent
+	g <- Git.Config.read =<< Git.CurrentRepo.get
 	_ <- Git.Index.override $ tmpIndex g
 	setup g
 	Git.UnionMerge.merge aref bref g
diff --git a/test.hs b/test.hs
index 1952a39a9b..c52a88d66f 100644
--- a/test.hs
+++ b/test.hs
@@ -26,7 +26,7 @@ import qualified Annex
 import qualified Annex.UUID
 import qualified Backend
 import qualified Git.Config
-import qualified Git.Construct
+import qualified Git.CurrentRepo
 import qualified Git.Filename
 import qualified Locations
 import qualified Types.Backend
@@ -721,7 +721,7 @@ git_annex_expectoutput command params expected = do
 -- are not run; this should only be used for actions that query state.
 annexeval :: Types.Annex a -> IO a
 annexeval a = do
-	s <- Annex.new =<< Git.Config.read =<< Git.Construct.fromCurrent
+	s <- Annex.new =<< Git.CurrentRepo.get
 	Annex.eval s $ do
 		Annex.setOutput Types.Messages.QuietOutput
 		a

From a1885bd11607d8668d70f81eaafa25e5341e8e8c Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 18 May 2012 18:30:50 -0400
Subject: [PATCH 130/220] make GIT_DIR, GIT_WORK_TREE absolute

GIT_DIR is set to something relative, like ".git" in the pre-commit hook.
But internally all the directories are assumed to be absolute.
---
 Git/CurrentRepo.hs | 10 ++++++----
 test.hs            |  1 -
 2 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/Git/CurrentRepo.hs b/Git/CurrentRepo.hs
index 4325f452c0..746904caf2 100644
--- a/Git/CurrentRepo.hs
+++ b/Git/CurrentRepo.hs
@@ -29,20 +29,22 @@ import qualified Git.Config
  -}
 get :: IO Repo
 get = do
-	gd <- takeenv "GIT_DIR"
+	gd <- pathenv "GIT_DIR"
 	r <- configure gd =<< maybe fromCwd fromPath gd
-	wt <- maybe (worktree $ location r) Just <$> takeenv "GIT_WORK_TREE"
+	wt <- maybe (worktree $ location r) Just <$> pathenv "GIT_WORK_TREE"
 	case wt of
 		Nothing -> return r
 		Just d -> do
 			changeWorkingDirectory d
 			return $ addworktree wt r
 	where
-		takeenv s = do
+		pathenv s = do
 			v <- getEnv s
 			when (isJust v) $
 				unsetEnv s
-			return v
+			case v of
+				Nothing -> return Nothing
+				Just d -> Just <$> absPath d
 		configure Nothing r = Git.Config.read r
 		configure (Just d) r = do
 			r' <- Git.Config.read r
diff --git a/test.hs b/test.hs
index c52a88d66f..9a0fce873e 100644
--- a/test.hs
+++ b/test.hs
@@ -25,7 +25,6 @@ import qualified Utility.SafeCommand
 import qualified Annex
 import qualified Annex.UUID
 import qualified Backend
-import qualified Git.Config
 import qualified Git.CurrentRepo
 import qualified Git.Filename
 import qualified Locations

From 0093a456e869b5250735ef4ec790faac115d9142 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 19 May 2012 10:22:43 -0400
Subject: [PATCH 131/220] test suite saved my bacon

git config reading memoization shouldn't be used when changing config
---
 Config.hs     |  3 +--
 Git/Config.hs | 31 ++++++++++++++++++++-----------
 2 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/Config.hs b/Config.hs
index bb57ab675e..f579e40b21 100644
--- a/Config.hs
+++ b/Config.hs
@@ -21,8 +21,7 @@ data ConfigKey = ConfigKey String
 setConfig :: ConfigKey -> String -> Annex ()
 setConfig (ConfigKey key) value = do
 	inRepo $ Git.Command.run "config" [Param key, Param value]
-	-- re-read git config and update the repo's state
-	newg <- inRepo Git.Config.read
+	newg <- inRepo Git.Config.reRead
 	Annex.changeState $ \s -> s { Annex.repo = newg }
 
 {- Unsets a git config setting. (Leaves it in state currently.) -}
diff --git a/Git/Config.hs b/Git/Config.hs
index 2fa685a11d..38e1ca4be7 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -28,19 +28,28 @@ getMaybe :: String -> Repo -> Maybe String
 getMaybe key repo = M.lookup key (config repo)
 
 {- Runs git config and populates a repo with its config.
- - Cannot use pipeRead because it relies on the config having been already
+ - Avoids re-reading config when run repeatedly. -}
+read :: Repo -> IO Repo
+read repo@(Repo { config = c })
+	| c == M.empty = read' repo
+	| otherwise = return repo
+
+{- Reads config even if it was read before. -}
+reRead :: Repo -> IO Repo
+reRead = read'
+
+{- Cannot use pipeRead because it relies on the config having been already
  - read. Instead, chdir to the repo.
  -}
-read :: Repo -> IO Repo
-read repo@(Repo { location = Local { gitdir = d } }) = read' repo d
-read repo@(Repo { location = LocalUnknown d }) = read' repo d
-read r = assertLocal r $ error "internal"
-read' :: Repo -> FilePath -> IO Repo
-read' repo@(Repo { config = c}) d
-	| c == M.empty = bracketCd d $
-		pOpen ReadFromPipe "git" ["config", "--null", "--list"] $
-			hRead repo
-	| otherwise = return repo -- config already read
+read' :: Repo -> IO Repo
+read' repo = go repo
+	where
+		go Repo { location = Local { gitdir = d } } = git_config d
+		go Repo { location = LocalUnknown d } = git_config d
+		go _ = assertLocal repo $ error "internal"
+		git_config d = bracketCd d $
+			pOpen ReadFromPipe "git" ["config", "--null", "--list"] $
+				hRead repo
 
 {- Reads git config from a handle and populates a repo with it. -}
 hRead :: Repo -> Handle -> IO Repo

From 9d9814477601217c8d39d75cc03e27ee4d734de3 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 19 May 2012 10:37:28 -0400
Subject: [PATCH 132/220] avoid chdir when already inside worktree

---
 Git/CurrentRepo.hs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Git/CurrentRepo.hs b/Git/CurrentRepo.hs
index 746904caf2..a40b412f5f 100644
--- a/Git/CurrentRepo.hs
+++ b/Git/CurrentRepo.hs
@@ -35,7 +35,9 @@ get = do
 	case wt of
 		Nothing -> return r
 		Just d -> do
-			changeWorkingDirectory d
+			cwd <- getCurrentDirectory
+			unless (d `dirContains` cwd) $
+				changeWorkingDirectory d
 			return $ addworktree wt r
 	where
 		pathenv s = do

From ebbd24e5ed6bbb41305d8105d7fac355dcf15bbc Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 19 May 2012 10:51:22 -0400
Subject: [PATCH 133/220] more worktree improvements

Avoid more expensive code path when no core.worktree is configured.

Don't change worktree when reading config if one is already set.
This could happen if GIT_CORE_WORKTREE is set, and the repo also has
core.worktree, and the config is reread. Now GIT_CORE_WORKTREE will
prevail.
---
 Git/Config.hs      | 26 ++++++++++++++------------
 Git/CurrentRepo.hs |  2 +-
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/Git/Config.hs b/Git/Config.hs
index 38e1ca4be7..dab1cdf5ea 100644
--- a/Git/Config.hs
+++ b/Git/Config.hs
@@ -78,19 +78,15 @@ store s repo = do
  - based on the core.bare and core.worktree settings.
  -}
 updateLocation :: Repo -> Repo
-updateLocation r = go $ location r
+updateLocation r@(Repo { location = LocalUnknown d })
+	| isBare r = newloc $ Local d Nothing
+	| otherwise = newloc $ Local (d  ".git") (Just d)
 	where
-		go (LocalUnknown d)
-			| isbare = ret $ Local d Nothing
-			| otherwise = ret $ Local (d  ".git") (Just d)
-		go l@(Local {}) = ret l
-		go _ = r
-		isbare = fromMaybe False $ isTrue =<< getMaybe "core.bare" r
-		ret l = r { location = l' }
-			where
-				l' = maybe l (setworktree l) $
-					getMaybe "core.worktree" r
-		setworktree l t = l { worktree = Just t }
+		newloc l = r { location = getworktree l }
+		getworktree l = case workTree r of
+			Nothing -> l
+			wt -> l { worktree = wt }
+updateLocation r = r
 
 {- Parses git config --list or git config --null --list output into a
  - config map. -}
@@ -114,3 +110,9 @@ isTrue s
 	| otherwise = Nothing
 	where
 		s' = map toLower s
+
+isBare :: Repo -> Bool
+isBare r = fromMaybe False $ isTrue =<< getMaybe "core.bare" r
+
+workTree :: Repo -> Maybe FilePath
+workTree = getMaybe "core.worktree"
diff --git a/Git/CurrentRepo.hs b/Git/CurrentRepo.hs
index a40b412f5f..de11ce217c 100644
--- a/Git/CurrentRepo.hs
+++ b/Git/CurrentRepo.hs
@@ -31,7 +31,7 @@ get :: IO Repo
 get = do
 	gd <- pathenv "GIT_DIR"
 	r <- configure gd =<< maybe fromCwd fromPath gd
-	wt <- maybe (worktree $ location r) Just <$> pathenv "GIT_WORK_TREE"
+	wt <- maybe (Git.Config.workTree r) Just <$> pathenv "GIT_WORK_TREE"
 	case wt of
 		Nothing -> return r
 		Just d -> do

From 37ef39c9295fd5f036264791e201262b0e641f85 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 20 May 2012 00:14:56 -0400
Subject: [PATCH 134/220] suppress "(Recording state in git)" message when
 committing change to remote state

This was shown redundantly for a tricky reason -- while it runs
inside a doSideAction block that would appear to supress it,
the action being run is in a different state monad; for the remote,
and so the suppression doesn't work.

Always suppressing the message when committing to a local remote is
ok do to though -- it mirrors the /dev/nulling of the git annex shell commit
output. And it turns out that any time there is a git-annex branch state
change to commit on the remote, the local repo has also had a similar
change made, and so the message has been shown already.
---
 Messages.hs   | 12 ++++++++++--
 Remote/Git.hs |  3 ++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/Messages.hs b/Messages.hs
index 4330f7c09f..96bf3ae4b5 100644
--- a/Messages.hs
+++ b/Messages.hs
@@ -14,6 +14,7 @@ module Messages (
 	MeterUpdate,
 	showSideAction,
 	doSideAction,
+	doQuietSideAction,
 	showStoringStateAction,
 	showOutput,
 	showLongNote,
@@ -91,12 +92,19 @@ showSideAction m = Annex.getState Annex.output >>= go
 showStoringStateAction :: Annex ()
 showStoringStateAction = showSideAction "Recording state in git"
 
+{- Performs an action, supressing showSideAction messages. -}
+doQuietSideAction :: Annex a -> Annex a
+doQuietSideAction = doSideAction' InBlock
+
 {- Performs an action, that may call showSideAction multiple times.
  - Only the first will be displayed. -}
 doSideAction :: Annex a -> Annex a
-doSideAction a = do
+doSideAction = doSideAction' StartBlock
+
+doSideAction' :: SideActionBlock -> Annex a -> Annex a
+doSideAction' b a = do
 	o <- Annex.getState Annex.output
-	set $ o { sideActionBlock = StartBlock }
+	set $ o { sideActionBlock = b }
 	set o `after` a
 	where
 		set o = Annex.changeState $ \s -> s {  Annex.output = o }
diff --git a/Remote/Git.hs b/Remote/Git.hs
index 79439b784f..cf7542d74e 100644
--- a/Remote/Git.hs
+++ b/Remote/Git.hs
@@ -310,7 +310,8 @@ commitOnCleanup r a = go `after` a
 		go = Annex.addCleanup (Git.repoLocation r) cleanup
 		cleanup
 			| not $ Git.repoIsUrl r = liftIO $ onLocal r $
-				Annex.Branch.commit "update"
+				doQuietSideAction $
+					Annex.Branch.commit "update"
 			| otherwise = void $ do
 				Just (shellcmd, shellparams) <-
 					git_annex_shell r "commit" []

From ab07762ddba62a2fcf04585a3549b503bc4bf1a9 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 22 May 2012 11:27:22 -0400
Subject: [PATCH 135/220] releasing version 3.20120522

---
 debian/changelog | 4 ++--
 git-annex.cabal  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 586077878b..99594c351c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,4 +1,4 @@
-git-annex (3.20120512) UNRELEASED; urgency=low
+git-annex (3.20120522) unstable; urgency=low
 
   * Pass -a to cp even when it supports --reflink=auto, to preserve
     permissions.
@@ -6,7 +6,7 @@ git-annex (3.20120512) UNRELEASED; urgency=low
   * Add support for core.worktree, and fix support for GIT_WORK_TREE and
     GIT_DIR.
 
- -- Joey Hess   Tue, 15 May 2012 14:17:49 -0400
+ -- Joey Hess   Tue, 22 May 2012 11:16:13 -0400
 
 git-annex (3.20120511) unstable; urgency=low
 
diff --git a/git-annex.cabal b/git-annex.cabal
index f87e49a200..e12cbb1777 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -1,5 +1,5 @@
 Name: git-annex
-Version: 3.20120511
+Version: 3.20120522
 Cabal-Version: >= 1.8
 License: GPL
 Maintainer: Joey Hess 

From d0da2ce5eaf35a9895a80a08359df6bb9cbb451a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 22 May 2012 11:28:09 -0400
Subject: [PATCH 136/220] add news item for git-annex 3.20120522

---
 doc/news/version_3.20120405.mdwn | 5 -----
 doc/news/version_3.20120522.mdwn | 7 +++++++
 2 files changed, 7 insertions(+), 5 deletions(-)
 delete mode 100644 doc/news/version_3.20120405.mdwn
 create mode 100644 doc/news/version_3.20120522.mdwn

diff --git a/doc/news/version_3.20120405.mdwn b/doc/news/version_3.20120405.mdwn
deleted file mode 100644
index c5a7ecc22a..0000000000
--- a/doc/news/version_3.20120405.mdwn
+++ /dev/null
@@ -1,5 +0,0 @@
-git-annex 3.20120405 released with [[!toggle text="these changes"]]
-[[!toggleable text="""
-   * Rewrote free disk space checking code, moving the portability
-     handling into a small C library.
-   * status: Display amount of free disk space."""]]
\ No newline at end of file
diff --git a/doc/news/version_3.20120522.mdwn b/doc/news/version_3.20120522.mdwn
new file mode 100644
index 0000000000..55c45900c1
--- /dev/null
+++ b/doc/news/version_3.20120522.mdwn
@@ -0,0 +1,7 @@
+git-annex 3.20120522 released with [[!toggle text="these changes"]]
+[[!toggleable text="""
+   * Pass -a to cp even when it supports --reflink=auto, to preserve
+     permissions.
+   * Clean up handling of git directory and git worktree.
+   * Add support for core.worktree, and fix support for GIT\_WORK\_TREE and
+     GIT\_DIR."""]]
\ No newline at end of file

From 1b1703c84cd3ee9c58bff6a7f586938670ac3b5f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawk5cj-itfFHq_yhJHdzk3QOPp-PNW_MjPU"
 
Date: Wed, 23 May 2012 19:30:22 +0000
Subject: [PATCH 137/220] Added a comment: +1 Cygwin

---
 ...mment_2_8acae818ce468967499050bbe3c532ea._comment | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 doc/todo/windows_support/comment_2_8acae818ce468967499050bbe3c532ea._comment

diff --git a/doc/todo/windows_support/comment_2_8acae818ce468967499050bbe3c532ea._comment b/doc/todo/windows_support/comment_2_8acae818ce468967499050bbe3c532ea._comment
new file mode 100644
index 0000000000..e37a555756
--- /dev/null
+++ b/doc/todo/windows_support/comment_2_8acae818ce468967499050bbe3c532ea._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawk5cj-itfFHq_yhJHdzk3QOPp-PNW_MjPU"
+ nickname="Michael"
+ subject="+1 Cygwin"
+ date="2012-05-23T19:30:21Z"
+ content="""
+Windows support is a must. In my experience, binary file means proprietary editor, which means Windows.
+
+Unfortunately, there's not much overlap between people who use graphical editors in Windows all day vs. people who are willing to tolerate Cygwin's setup.exe, compile a Haskell program, learn git and git-annex's 90-odd subcommands, and use a mintty terminal to manage their repository, especially now that there's a sexy GitHub app for Windows.
+
+That aside, I think Windows-based content producers are still *the* audience for git-annex. First Windows support, then a GUI, then the world.
+"""]]

From 66580a8b7ab4aad30a3d615063ddd0efbd35df65 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 23 May 2012 19:36:14 -0400
Subject: [PATCH 138/220] fix link

---
 doc/design/encryption.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/encryption.mdwn b/doc/design/encryption.mdwn
index 647683bd9f..b7acbb732a 100644
--- a/doc/design/encryption.mdwn
+++ b/doc/design/encryption.mdwn
@@ -20,7 +20,7 @@ with more than one encryption backend in mind helps future-proofing.
 
 [[!template id=note text="""
 The basis of this scheme was originally developed by Lars Wirzenius et al
-[for Obnam](http://braawi.org/obnam/encryption/).
+[for Obnam](http://liw.fi/obnam/encryption/).
 """]]
 
 Data is encrypted by gpg, using a symmetric cipher.

From 09fbc215e8f7cf7aec85c3a68e8ad8c5c3b84b66 Mon Sep 17 00:00:00 2001
From: "https://launchpad.net/~ojwb" 
Date: Thu, 24 May 2012 05:07:07 +0000
Subject: [PATCH 139/220] typo fixes

---
 doc/todo/automatic_bookkeeping_watch_command.mdwn | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/todo/automatic_bookkeeping_watch_command.mdwn b/doc/todo/automatic_bookkeeping_watch_command.mdwn
index d7a3517a19..4b688b839e 100644
--- a/doc/todo/automatic_bookkeeping_watch_command.mdwn
+++ b/doc/todo/automatic_bookkeeping_watch_command.mdwn
@@ -4,10 +4,10 @@ It would run, in the background, watching via inotify for changes, and
 automatically annexing new files, etc.
 
 The blue sky goal would be something automated like dropbox, except fully
-distributed. All files put into the repository would propigate out
+distributed. All files put into the repository would propagate out
 to all the other clones of it, as network links allow. Note that while
 dropbox allows modifying files, git-annex freezes them upon creation,
-so this would not be 100% equivilant to dropbox. --[[Joey]]
+so this would not be 100% equivalent to dropbox. --[[Joey]]
 
 ----
 
@@ -25,7 +25,7 @@ Also nice to have would be:
     
 - Somehow sync remotes, possibly using a push sync like dvcs-autosync
   does, so they are immediately updated.
-- Somehow get content that is unavilable. This is problimatic with inotify,
+- Somehow get content that is unavailable. This is problematic with inotify,
   since we only get an event once the user has tried (and failed) to read
   from the file. Perhaps instead, automatically copy content that is added
   out to remotes, with the goal of all repos eventually getting a copy,
@@ -35,7 +35,7 @@ Also nice to have would be:
 - Perhaps automatically dropunused files that have been deleted,
   although I cannot see a way to do that, since by the time the inotify
   deletion event arrives, the file is deleted, and we cannot see what
-  its symlink pointed to! Alternatievely, perhaps automatically
+  its symlink pointed to! Alternatively, perhaps automatically
   do an expensive unused/dropunused cleanup process.
 - Support OSes other than Linux; it only uses inotify currently.
 

From c270ca5453382e2ccaf70d96bf7ddff4a5a014b8 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 
Date: Thu, 24 May 2012 07:52:54 +0000
Subject: [PATCH 140/220] fix typo

---
 doc/forum/cloud_services_to_support.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/forum/cloud_services_to_support.mdwn b/doc/forum/cloud_services_to_support.mdwn
index e268bc1d86..4d6bde1b54 100644
--- a/doc/forum/cloud_services_to_support.mdwn
+++ b/doc/forum/cloud_services_to_support.mdwn
@@ -1,5 +1,5 @@
 git-annex can already be used to store data in several cloud services:
-Amazon S3, rsync.net, Tahoe-LAFFS, The Internet Archive.
+Amazon S3, rsync.net, Tahoe-LAFS, The Internet Archive.
 
 I would like to support as many other cloud services as possible/reasonable.
 

From 3d208ee569470f816ab6def94ee2b13ad9fe6423 Mon Sep 17 00:00:00 2001
From: "https://rmunn.myopenid.com/" 
Date: Thu, 24 May 2012 18:14:13 +0000
Subject: [PATCH 141/220]

---
 doc/forum/What_happened_to_the_walkthrough__63__.mdwn | 1 +
 1 file changed, 1 insertion(+)
 create mode 100644 doc/forum/What_happened_to_the_walkthrough__63__.mdwn

diff --git a/doc/forum/What_happened_to_the_walkthrough__63__.mdwn b/doc/forum/What_happened_to_the_walkthrough__63__.mdwn
new file mode 100644
index 0000000000..e8098d29a1
--- /dev/null
+++ b/doc/forum/What_happened_to_the_walkthrough__63__.mdwn
@@ -0,0 +1 @@
+As of right now (2012-05-24 at 18:00 UTC), the [[Walkthrough]] page is basically empty. Its entire contents are "A walkthrough of the basic features of git-annex." No links (other than the autogenerated "what links to this page" list at the bottom) and no contents. Any idea what happened?

From 164b55e99209bfb9ec0d92064027ab6e604232d0 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
 
Date: Thu, 24 May 2012 19:18:55 +0000
Subject: [PATCH 142/220] Added a comment

---
 .../comment_1_70db0e3cfb1318e95671c23726e5541d._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/What_happened_to_the_walkthrough__63__/comment_1_70db0e3cfb1318e95671c23726e5541d._comment

diff --git a/doc/forum/What_happened_to_the_walkthrough__63__/comment_1_70db0e3cfb1318e95671c23726e5541d._comment b/doc/forum/What_happened_to_the_walkthrough__63__/comment_1_70db0e3cfb1318e95671c23726e5541d._comment
new file mode 100644
index 0000000000..cbf852dbfb
--- /dev/null
+++ b/doc/forum/What_happened_to_the_walkthrough__63__/comment_1_70db0e3cfb1318e95671c23726e5541d._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawl9sYlePmv1xK-VvjBdN-5doOa_Xw-jH4U"
+ nickname="Richard"
+ subject="comment 1"
+ date="2012-05-24T19:18:55Z"
+ content="""
+It seems the pages that are supposed to be inlined are not being found even though they are in `doc/walkthrough/`.
+"""]]

From b8f12b0dbe4d0304fab0c3eecbb38e52815e2f3e Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 24 May 2012 16:10:21 -0400
Subject: [PATCH 143/220] fix inline pagenames in walkthrough

I should read my own ikiwiki changelog when upgrading, apparently.
---
 doc/walkthrough.mdwn | 36 ++++++++++++++++++------------------
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/doc/walkthrough.mdwn b/doc/walkthrough.mdwn
index f93e28393e..a64b175f95 100644
--- a/doc/walkthrough.mdwn
+++ b/doc/walkthrough.mdwn
@@ -3,22 +3,22 @@ A walkthrough of the basic features of git-annex.
 [[!toc]]
 
 [[!inline feeds=no show=0 template=walkthrough pagenames="""
-	creating_a_repository
-	adding_a_remote
-	adding_files
-	renaming_files
-	getting_file_content
-	syncing
-	transferring_files:_When_things_go_wrong
-	removing_files
-	removing_files:_When_things_go_wrong
-	modifying_annexed_files
-	using_ssh_remotes
-	moving_file_content_between_repositories
-	unused_data
-	fsck:_verifying_your_data
-	fsck:_when_things_go_wrong
-	backups
-	automatically_managing_content
-	more
+	walkthrough/creating_a_repository
+	walkthrough/adding_a_remote
+	walkthrough/adding_files
+	walkthrough/renaming_files
+	walkthrough/getting_file_content
+	walkthrough/syncing
+	walkthrough/transferring_files:_When_things_go_wrong
+	walkthrough/removing_files
+	walkthrough/removing_files:_When_things_go_wrong
+	walkthrough/modifying_annexed_files
+	walkthrough/using_ssh_remotes
+	walkthrough/moving_file_content_between_repositories
+	walkthrough/unused_data
+	walkthrough/fsck:_verifying_your_data
+	walkthrough/fsck:_when_things_go_wrong
+	walkthrough/backups
+	walkthrough/automatically_managing_content
+	walkthrough/more
 """]]

From 570fe8af6f385503c3aab4ef4c1ef606685c194c Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 24 May 2012 20:15:19 +0000
Subject: [PATCH 144/220] Added a comment

---
 .../comment_2_f9305dd19b9b5f35e66d915b8c30374b._comment    | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 doc/forum/What_happened_to_the_walkthrough__63__/comment_2_f9305dd19b9b5f35e66d915b8c30374b._comment

diff --git a/doc/forum/What_happened_to_the_walkthrough__63__/comment_2_f9305dd19b9b5f35e66d915b8c30374b._comment b/doc/forum/What_happened_to_the_walkthrough__63__/comment_2_f9305dd19b9b5f35e66d915b8c30374b._comment
new file mode 100644
index 0000000000..9720adfbc6
--- /dev/null
+++ b/doc/forum/What_happened_to_the_walkthrough__63__/comment_2_f9305dd19b9b5f35e66d915b8c30374b._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 2"
+ date="2012-05-24T20:15:19Z"
+ content="""
+Broken last night during upgrade, fixed now, thanks for noticing.
+"""]]

From f7524811e2e47ca8455d3038b6d0a8102e5a5044 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Fri, 25 May 2012 08:52:55 -0400
Subject: [PATCH 145/220] bug

---
 doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn

diff --git a/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn b/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn
new file mode 100644
index 0000000000..69704bdd07
--- /dev/null
+++ b/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn
@@ -0,0 +1,5 @@
+Add a file, then unlock it, and then lock it. There is an error and the
+symlink gets deleted. 
+
+The file will still be staged in the index, and the file content is still
+in the annex. --[[Joey]] 

From 45a01db6add3399ff6ca93f2e7c7d83dbf59992d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:11:19 -0400
Subject: [PATCH 146/220] add preliminary design

---
 doc/design/assistant.mdwn               | 19 +++++++++++++++
 doc/design/assistant/android.mdwn       | 31 +++++++++++++++++++++++++
 doc/design/assistant/configurators.mdwn | 21 +++++++++++++++++
 doc/design/assistant/inotify.mdwn       | 18 ++++++++++++++
 doc/design/assistant/progressbars.mdwn  | 14 +++++++++++
 doc/design/assistant/syncing.mdwn       | 31 +++++++++++++++++++++++++
 doc/design/assistant/webapp.mdwn        | 27 +++++++++++++++++++++
 doc/design/assistant/windows.mdwn       | 17 ++++++++++++++
 8 files changed, 178 insertions(+)
 create mode 100644 doc/design/assistant.mdwn
 create mode 100644 doc/design/assistant/android.mdwn
 create mode 100644 doc/design/assistant/configurators.mdwn
 create mode 100644 doc/design/assistant/inotify.mdwn
 create mode 100644 doc/design/assistant/progressbars.mdwn
 create mode 100644 doc/design/assistant/syncing.mdwn
 create mode 100644 doc/design/assistant/webapp.mdwn
 create mode 100644 doc/design/assistant/windows.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
new file mode 100644
index 0000000000..a9444c3673
--- /dev/null
+++ b/doc/design/assistant.mdwn
@@ -0,0 +1,19 @@
+The git-annex assistant is being
+[crowd funded on Kickstarter](http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own/).
+
+This is my design and plan for developing it.
+Still being fleshed out, still many ideas and use cases to add. --[[Joey]]
+
+## roadmap
+
+### Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
+
+### Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars||
+
+### Month 3 "easy setup": [[!traillink configurators]]
+
+### Month 4: polishing and overflow
+
+### Month 5 & 6 "9k bonus round": [[!traillink Android]]
+
+### In my overfunded nighmares: [[!traillink Windows]]
diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
new file mode 100644
index 0000000000..24791501da
--- /dev/null
+++ b/doc/design/assistant/android.mdwn
@@ -0,0 +1,31 @@
+Porting git-annex to Android will use the Android native SDK.
+
+A hopefully small Java app will be developed, which runs the webapp
+daemon, and a web browser to display it.
+
+### programs to port
+
+These will probably need to be bundled into the Android app, unless already
+available in the App Store.
+
+* ssh (native ssh needed for scp, not a client like ConnectBot)
+* rsync
+* gpg
+* git (not all git commands are needed, but a core plumbing and a few like `git-add` are.)
+
+### FAT sucks
+
+The main media partition will use some awful FAT filesystem format from
+1982 that cannot support git-annex's symlinks. Hopefully it can at least
+handle all of git's filenames. Possible approaches to this:
+
+* Keep only a bare git repo on Android. The app would then need to include
+  a file browser to access the files in there, and adding a file would move
+  it into the repo. Not ideal.
+* Implement [[smudge]] filters to avoid needing symlinks. Difficult.
+* Use a bare git repo but don't keep files in `annex/objects`, instead
+  leave them outside the repo, and add some local mapping to find them.
+  Seems best?
+* Use a `LD_PRELOAD` wrapper to do Something Crazy.
+
+(May want to consider which of these would make a Windows port easier too.)
diff --git a/doc/design/assistant/configurators.mdwn b/doc/design/assistant/configurators.mdwn
new file mode 100644
index 0000000000..e03723e946
--- /dev/null
+++ b/doc/design/assistant/configurators.mdwn
@@ -0,0 +1,21 @@
+Add to the [[webapp]] some configuration of git-annex.
+
+There are some basic settings that pass through to `git config`, things
+like how much disk space to leave free, how many copies to ensure are kept
+of files, etc.
+
+The meat of the configuration will be in configuration assistants that walk
+through setting up common use cases.
+
+* Clone this repo to a USB drive.
+* Clone this repo to another host:
+  1. Prompt for the hostname (or do avahi local machine discovery). 
+  2. Enable the two hosts to ssh to one-another and run git-annex shell.
+     (A tricky problem, if ssh keys need to be added to do that.)
+  3. Push over a clone of the repository. (Using git-annex-shell?)
+  4. Start [[syncing]].
+* Set up Amazon S3.
+* Set up rsync remote.
+* Set up encryption.
+* I lost my USB drive!
+* etc -- many more possibilities
diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn
new file mode 100644
index 0000000000..be38ef13cd
--- /dev/null
+++ b/doc/design/assistant/inotify.mdwn
@@ -0,0 +1,18 @@
+Finish "git annex watch" command, which runs, in the background, watching via
+inotify for changes, and automatically annexing new files, etc.
+
+There is a `watch` branch in git that adds such a command, although currently
+it only handles adding new files, and nothing else. To make this really
+useful, it needs to:
+
+- notice deleted files and stage the deletion
+  (tricky; there's a race with add..)
+- notice renamed files, auto-fix the symlink, and stage the new file location
+- periodically auto-commit staged changes
+- honor .gitignore, not adding files it excludesa
+
+Also to do:
+
+- Support OSes other than Linux; it only uses inotify currently.
+  OSX and FreeBSD use the same mechanism, and there is a Haskell interface
+  for it,
diff --git a/doc/design/assistant/progressbars.mdwn b/doc/design/assistant/progressbars.mdwn
new file mode 100644
index 0000000000..f76b42d73a
--- /dev/null
+++ b/doc/design/assistant/progressbars.mdwn
@@ -0,0 +1,14 @@
+Currently, git-annex takes a very lazy approch to displaying
+progress into. It just lets rsync or whatever display the progress
+for it, in the terminal.
+
+Something better is needed for the [[webapp]]. There needs to be a
+way for the web app to know what the current progress is of all transfers.
+
+To get this info for downloads, git-annex can watch the file as it arrives
+and use its size.
+
+TODO: What about uploads? Will i have to parse rsync's progresss output?
+Ugh.
+
+This is one of those potentially hidden but time consuming problems.
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
new file mode 100644
index 0000000000..0a081c1010
--- /dev/null
+++ b/doc/design/assistant/syncing.mdwn
@@ -0,0 +1,31 @@
+Once files are added (or removed or moved), need to send those changes to
+all the other git clones, at both the git level and the key/value level.
+
+## git syncing
+
+1. At regular intervals, just run `git annex sync`, which already handles
+   bidirectional syncing.
+2. Investigate the XMPP approach like dvcs-autosync does, or other ways of
+   signaling a change out of band.
+3. Add a hook, so when there's a change to sync, a program can be run.
+
+## data syncing
+
+There are two parts to data syncing. First, map the network and second,
+decide what to sync when.
+
+Mapping the network can reuse code in `git annex map`. Once the map is
+built, we want to find paths through the network that reach all nodes
+eventually, with the least cost. This is a minimum spanning tree problem,
+except with a directed graph, so really a Arborescence problem.
+
+With the map, we can determine which nodes to push new content to. Then we
+need to control those data transfers, sending to the cheapest nodes first,
+and with appropriate rate limiting and control facilities.
+
+This probably will need lots of refinements to get working well.
+
+## other considerations
+
+This assumes the network is connected. It's often not, so the
+cloud needs to be used to bridge between LANs.
diff --git a/doc/design/assistant/webapp.mdwn b/doc/design/assistant/webapp.mdwn
new file mode 100644
index 0000000000..c0f6b893e1
--- /dev/null
+++ b/doc/design/assistant/webapp.mdwn
@@ -0,0 +1,27 @@
+The webapp is a web server that displays a shiny interface.
+
+## security
+
+* Listen only to localhost.
+* Instruct the user's web browser to open an url that contains a secret
+  token. This guards against other users on the same system.
+* I would like to avoid passwords or other authentication methods,
+  it's your local system.
+
+## interface
+
+* list of files uploading and downloading
+* progress bars for each file
+* drag and drop to reorder
+* cancel and pause
+
+## implementation
+
+Hope to use Yesod.
+
+TODO: Ensure that Yesod will work on arm. Necessary for later Android port.
+Will its template haskell cause a problem? Does new GHC support TH on ARM?
+Will it use too much memory or be too slow?
+
+Hopefully Yesod comes with some good UI widgets. Otherwise, need to use
+Jquery or similar.
diff --git a/doc/design/assistant/windows.mdwn b/doc/design/assistant/windows.mdwn
new file mode 100644
index 0000000000..0b176934b9
--- /dev/null
+++ b/doc/design/assistant/windows.mdwn
@@ -0,0 +1,17 @@
+See [[todo/windows_support]]..
+
+## symlinks
+
+Apparently new versions of Windows have something very like symlinks.
+(Or really, 3 or so things not entirely unlike symlinks and all different.)
+Stackoverflow has some details.
+
+Make git use them, as it (apparently) does not yet.
+
+(What **does** git do on Windows when it clones a repo with symlinks?)
+
+## POSIX
+
+Lots of ifdefs and pain to deal with POSIX calls in the code base.
+
+Or I could try to use Cywin.

From 931b85c4817d12ad84f3be2b3649d4b550b9214b Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:14:27 -0400
Subject: [PATCH 147/220] update

---
 doc/design/assistant.mdwn | 20 ++++++++------------
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index a9444c3673..316f62ba1d 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -2,18 +2,14 @@ The git-annex assistant is being
 [crowd funded on Kickstarter](http://www.kickstarter.com/projects/joeyh/git-annex-assistant-like-dropbox-but-with-your-own/).
 
 This is my design and plan for developing it.
-Still being fleshed out, still many ideas and use cases to add. --[[Joey]]
+Still being fleshed out, still many ideas and use cases to add.
+Feel free to chip in with comments! --[[Joey]]
 
 ## roadmap
 
-### Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
-
-### Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars||
-
-### Month 3 "easy setup": [[!traillink configurators]]
-
-### Month 4: polishing and overflow
-
-### Month 5 & 6 "9k bonus round": [[!traillink Android]]
-
-### In my overfunded nighmares: [[!traillink Windows]]
+* Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
+* Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars||
+* Month 3 "easy setup": [[!traillink configurators]]
+* Month 4: polishing and overflow
+* Month 5 & 6 "9k bonus round": [[!traillink Android]]
+* In my overfunded nighmares: [[!traillink Windows]]

From 573f1c03a84ccc285490f6964b2bbfef7205bf09 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:15:07 -0400
Subject: [PATCH 148/220] fix

---
 doc/design/assistant.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 316f62ba1d..6320925e89 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -8,7 +8,7 @@ Feel free to chip in with comments! --[[Joey]]
 ## roadmap
 
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
-* Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars||
+* Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
 * Month 4: polishing and overflow
 * Month 5 & 6 "9k bonus round": [[!traillink Android]]

From 147f63ccabc12b95d28f995b776eb61bde141b37 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:15:40 -0400
Subject: [PATCH 149/220] tweak

---
 doc/design/assistant.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 6320925e89..3ab18e0a3f 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -11,5 +11,5 @@ Feel free to chip in with comments! --[[Joey]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
 * Month 4: polishing and overflow
-* Month 5 & 6 "9k bonus round": [[!traillink Android]]
+* Months 5-6 "9k bonus round": [[!traillink Android]]
 * In my overfunded nighmares: [[!traillink Windows]]

From 08aeec8f1e3d30435177b8f0756630e488ec3ede Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:17:21 -0400
Subject: [PATCH 150/220] alternative

---
 doc/design/assistant/progressbars.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant/progressbars.mdwn b/doc/design/assistant/progressbars.mdwn
index f76b42d73a..2ade05aa57 100644
--- a/doc/design/assistant/progressbars.mdwn
+++ b/doc/design/assistant/progressbars.mdwn
@@ -9,6 +9,6 @@ To get this info for downloads, git-annex can watch the file as it arrives
 and use its size.
 
 TODO: What about uploads? Will i have to parse rsync's progresss output?
-Ugh.
+Feed it via a named pipe? Ugh.
 
 This is one of those potentially hidden but time consuming problems.

From ec9f793db1426126673ce4a5e9380e089628bc7d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:18:27 -0400
Subject: [PATCH 151/220] update

---
 doc/design/assistant.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 3ab18e0a3f..f0805594e7 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -10,6 +10,6 @@ Feel free to chip in with comments! --[[Joey]]
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
-* Month 4: polishing and overflow
+* Month 4: polishing and overflow; release
 * Months 5-6 "9k bonus round": [[!traillink Android]]
 * In my overfunded nighmares: [[!traillink Windows]]

From 6703892e83c89a27a2e2766e5984618e7449bdb8 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:24:43 -0400
Subject: [PATCH 152/220] reord

---
 doc/design/assistant.mdwn                     | 10 +++++-
 doc/design/assistant/leftovers.mdwn           | 14 ++++++++
 .../automatic_bookkeeping_watch_command.mdwn  | 32 +------------------
 3 files changed, 24 insertions(+), 32 deletions(-)
 create mode 100644 doc/design/assistant/leftovers.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index f0805594e7..5cbdb20d19 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -5,11 +5,19 @@ This is my design and plan for developing it.
 Still being fleshed out, still many ideas and use cases to add.
 Feel free to chip in with comments! --[[Joey]]
 
+## the pitch
+
+The blue sky goal would be something automated like dropbox, except fully
+distributed. All files put into the repository would propagate out
+to all the other clones of it, as network links allow. Note that while
+dropbox allows modifying files, git-annex freezes them upon creation,
+so this would not be 100% equivalent to dropbox.
+
 ## roadmap
 
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
-* Month 4: polishing and overflow; release
+* Month 4 "release": [[!traillink leftovers]]; release
 * Months 5-6 "9k bonus round": [[!traillink Android]]
 * In my overfunded nighmares: [[!traillink Windows]]
diff --git a/doc/design/assistant/leftovers.mdwn b/doc/design/assistant/leftovers.mdwn
new file mode 100644
index 0000000000..313544d384
--- /dev/null
+++ b/doc/design/assistant/leftovers.mdwn
@@ -0,0 +1,14 @@
+Things that don't fit anywhere else:
+
+* Somehow get content that is unavailable. This is problematic with inotify,
+  since we only get an event once the user has tried (and failed) to read
+  from the file. This is only needed if all the files in the directory 
+  are not kept synced, but in some situations (ie, low disk space phones),
+  that is likely.
+* Drop files that have not been used lately, or meet some other criteria
+  (as long as there's a copy elsewhere).
+* Perhaps automatically dropunused files that have been deleted,
+  although I cannot see a way to do that, since by the time the inotify
+  deletion event arrives, the file is deleted, and we cannot see what
+  its symlink pointed to! Alternatively, perhaps automatically
+  do an expensive unused/dropunused cleanup process.
diff --git a/doc/todo/automatic_bookkeeping_watch_command.mdwn b/doc/todo/automatic_bookkeeping_watch_command.mdwn
index 4b688b839e..0bb86e4a13 100644
--- a/doc/todo/automatic_bookkeeping_watch_command.mdwn
+++ b/doc/todo/automatic_bookkeeping_watch_command.mdwn
@@ -9,34 +9,4 @@ to all the other clones of it, as network links allow. Note that while
 dropbox allows modifying files, git-annex freezes them upon creation,
 so this would not be 100% equivalent to dropbox. --[[Joey]]
 
-----
-
-There is a `watch` branch in git that adds such a command, although currently
-it only handles adding new files, and nothing else. To make this really
-useful, it needs to:
-
-- notice deleted files and stage the deletion
-  (tricky; there's a race with add..)
-- notice renamed files, auto-fix the symlink, and stage the new file location
-- periodically auto-commit staged changes
-- honor .gitignore, not adding files it excludes
-    
-Also nice to have would be:
-    
-- Somehow sync remotes, possibly using a push sync like dvcs-autosync
-  does, so they are immediately updated.
-- Somehow get content that is unavailable. This is problematic with inotify,
-  since we only get an event once the user has tried (and failed) to read
-  from the file. Perhaps instead, automatically copy content that is added
-  out to remotes, with the goal of all repos eventually getting a copy,
-  if df allows.
-- Drop files that have not been used lately, or meet some other criteria
-  (as long as there's a copy elsewhere).
-- Perhaps automatically dropunused files that have been deleted,
-  although I cannot see a way to do that, since by the time the inotify
-  deletion event arrives, the file is deleted, and we cannot see what
-  its symlink pointed to! Alternatively, perhaps automatically
-  do an expensive unused/dropunused cleanup process.
-- Support OSes other than Linux; it only uses inotify currently.
-
-
+This is a big project with its own [[design pages|design/assistant]].

From 76720a6d0df1dd8f3d4405bf4e7cd1a6ce31005d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 21:38:25 -0400
Subject: [PATCH 153/220] update

---
 doc/design/assistant.mdwn           | 15 ++++++---------
 doc/design/assistant/deltas.mdwn    |  9 +++++++++
 doc/design/assistant/desymlink.mdwn |  5 +++++
 3 files changed, 20 insertions(+), 9 deletions(-)
 create mode 100644 doc/design/assistant/deltas.mdwn
 create mode 100644 doc/design/assistant/desymlink.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 5cbdb20d19..3bbd27c581 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -5,14 +5,6 @@ This is my design and plan for developing it.
 Still being fleshed out, still many ideas and use cases to add.
 Feel free to chip in with comments! --[[Joey]]
 
-## the pitch
-
-The blue sky goal would be something automated like dropbox, except fully
-distributed. All files put into the repository would propagate out
-to all the other clones of it, as network links allow. Note that while
-dropbox allows modifying files, git-annex freezes them upon creation,
-so this would not be 100% equivalent to dropbox.
-
 ## roadmap
 
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
@@ -20,4 +12,9 @@ so this would not be 100% equivalent to dropbox.
 * Month 3 "easy setup": [[!traillink configurators]]
 * Month 4 "release": [[!traillink leftovers]]; release
 * Months 5-6 "9k bonus round": [[!traillink Android]]
-* In my overfunded nighmares: [[!traillink Windows]]
+
+## not yet on the map:
+
+* [[desymlink]]
+* [[deltas]]
+* In my overfunded nighmares: [[Windows]]
diff --git a/doc/design/assistant/deltas.mdwn b/doc/design/assistant/deltas.mdwn
new file mode 100644
index 0000000000..cf2d9f6c34
--- /dev/null
+++ b/doc/design/assistant/deltas.mdwn
@@ -0,0 +1,9 @@
+Speed up syncing of modified versions of to existing files. 
+
+One simple way is to find the key of the old version of a file that's
+being transferred, so it can be used as the basis for rsync, or any
+other similar transfer protocol.
+
+For remotes that don't use rsync, a poor man's version could be had by
+chunking each object into multiple parts. Only modified parts need be
+transferred. Sort of sub-keys to the main key being stored.
diff --git a/doc/design/assistant/desymlink.mdwn b/doc/design/assistant/desymlink.mdwn
new file mode 100644
index 0000000000..e12cd52dc7
--- /dev/null
+++ b/doc/design/assistant/desymlink.mdwn
@@ -0,0 +1,5 @@
+While dropbox allows modifying files in the folder, git-annex freezes
+them upon creation.
+
+To allow editing files in its folder, something like [[todo/smudge]] is
+needed, to get rid of the symlinks that stand in for the files.

From 697298718cbc45f1a521584e6757acc69d0f612d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 22:25:25 -0400
Subject: [PATCH 154/220] update

---
 doc/design/assistant/syncing.mdwn | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
index 0a081c1010..9cbdddcb96 100644
--- a/doc/design/assistant/syncing.mdwn
+++ b/doc/design/assistant/syncing.mdwn
@@ -5,9 +5,11 @@ all the other git clones, at both the git level and the key/value level.
 
 1. At regular intervals, just run `git annex sync`, which already handles
    bidirectional syncing.
-2. Investigate the XMPP approach like dvcs-autosync does, or other ways of
+2. Use a git merge driver that adds both conflicting files,
+   so conflicts never break a sync.
+3. Investigate the XMPP approach like dvcs-autosync does, or other ways of
    signaling a change out of band.
-3. Add a hook, so when there's a change to sync, a program can be run.
+4. Add a hook, so when there's a change to sync, a program can be run.
 
 ## data syncing
 
@@ -29,3 +31,6 @@ This probably will need lots of refinements to get working well.
 
 This assumes the network is connected. It's often not, so the
 cloud needs to be used to bridge between LANs.
+
+It would be nice if, when a USB drive is connected,
+syncing starts automatically.

From 8bf2d1902b5bff01ee9343d6fdcd09254f960d79 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sat, 26 May 2012 23:47:55 -0400
Subject: [PATCH 155/220] format

---
 doc/install/Gentoo.mdwn | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/doc/install/Gentoo.mdwn b/doc/install/Gentoo.mdwn
index c508309c6e..feeaad739b 100644
--- a/doc/install/Gentoo.mdwn
+++ b/doc/install/Gentoo.mdwn
@@ -1,5 +1,3 @@
-Gentoo users can:
-
-	`emerge git-annex`
+Gentoo users can: `emerge git-annex`
 
 A possibly more up-to-date version is in the haskell portage overlay.

From 737a273ea7da564bdae8feaae2385766307e68b4 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 20:33:01 +0000
Subject: [PATCH 156/220] Added a comment: cannot get files

---
 ...t_1_95cadaad486dc32b4503cfddf2500bc1._comment | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment b/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment
new file mode 100644
index 0000000000..604c1fb6d7
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="cannot get files"
+ date="2012-05-27T20:33:00Z"
+ content="""
+Hi,
+
+I could successfully clone my ssh repo's annex to my laptop, following these instructions.
+I'm also able to sync the repositories (laptop and ssh) when I commit new files in the ssh repo.
+
+However, every time I try to get files from the ssh repo (using 'git annex get some_file'), nothing happens.
+Do you know what can be happening?
+
+Thanks!
+"""]]

From 085e02012fd029bcf6709cc3d30d8c0886d4eb52 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 20:33:10 +0000
Subject: [PATCH 157/220] Added a comment: cannot get files

---
 ...t_2_451fd0c6a25ee61ef137e8e5be0c286b._comment | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_2_451fd0c6a25ee61ef137e8e5be0c286b._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_2_451fd0c6a25ee61ef137e8e5be0c286b._comment b/doc/walkthrough/using_ssh_remotes/comment_2_451fd0c6a25ee61ef137e8e5be0c286b._comment
new file mode 100644
index 0000000000..2121401968
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_2_451fd0c6a25ee61ef137e8e5be0c286b._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="cannot get files"
+ date="2012-05-27T20:33:09Z"
+ content="""
+Hi,
+
+I could successfully clone my ssh repo's annex to my laptop, following these instructions.
+I'm also able to sync the repositories (laptop and ssh) when I commit new files in the ssh repo.
+
+However, every time I try to get files from the ssh repo (using 'git annex get some_file'), nothing happens.
+Do you know what can be happening?
+
+Thanks!
+"""]]

From d39c0d542005c01c6653d7a9ab2cc748f1e0206f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 20:33:22 +0000
Subject: [PATCH 158/220] removed

---
 ...t_1_95cadaad486dc32b4503cfddf2500bc1._comment | 16 ----------------
 1 file changed, 16 deletions(-)
 delete mode 100644 doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment b/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment
deleted file mode 100644
index 604c1fb6d7..0000000000
--- a/doc/walkthrough/using_ssh_remotes/comment_1_95cadaad486dc32b4503cfddf2500bc1._comment
+++ /dev/null
@@ -1,16 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
- nickname="Fernando Seabra"
- subject="cannot get files"
- date="2012-05-27T20:33:00Z"
- content="""
-Hi,
-
-I could successfully clone my ssh repo's annex to my laptop, following these instructions.
-I'm also able to sync the repositories (laptop and ssh) when I commit new files in the ssh repo.
-
-However, every time I try to get files from the ssh repo (using 'git annex get some_file'), nothing happens.
-Do you know what can be happening?
-
-Thanks!
-"""]]

From 9ccf933ffa7d11e7abefb51f0ffd8c652847899f Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Sun, 27 May 2012 20:53:06 +0000
Subject: [PATCH 159/220] Added a comment

---
 ...ment_2_365db5820d96d5daa62c19fd76fcdf1e._comment | 13 +++++++++++++
 1 file changed, 13 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_2_365db5820d96d5daa62c19fd76fcdf1e._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_2_365db5820d96d5daa62c19fd76fcdf1e._comment b/doc/walkthrough/using_ssh_remotes/comment_2_365db5820d96d5daa62c19fd76fcdf1e._comment
new file mode 100644
index 0000000000..8973978ad8
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_2_365db5820d96d5daa62c19fd76fcdf1e._comment
@@ -0,0 +1,13 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 2"
+ date="2012-05-27T20:53:05Z"
+ content="""
+When `git annex get` does nothing, it's because it doesn't know a place to get the file from.
+
+This can happen if the `git-annex` branch has not propigated from the place where the file was added.
+For example, if on the laptop you had run `git pull ssh master`, that would only pull the master branch, not the git-annex branch. 
+
+An easy way to ensure the git-annex branch is kept in sync is to run `git annex sync`
+"""]]

From 402b4c81ba2eed3b5fd9a9f9db6800fa7a128b0b Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 16:56:20 -0400
Subject: [PATCH 160/220] add trail

---
 doc/walkthrough.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/walkthrough.mdwn b/doc/walkthrough.mdwn
index a64b175f95..c288b71ded 100644
--- a/doc/walkthrough.mdwn
+++ b/doc/walkthrough.mdwn
@@ -2,7 +2,7 @@ A walkthrough of the basic features of git-annex.
 
 [[!toc]]
 
-[[!inline feeds=no show=0 template=walkthrough pagenames="""
+[[!inline feeds=no trail=yes show=0 template=walkthrough pagenames="""
 	walkthrough/creating_a_repository
 	walkthrough/adding_a_remote
 	walkthrough/adding_files

From e96975eb3af2cb5da49099a4e87013af0534749f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 16:56:28 -0400
Subject: [PATCH 161/220] update

---
 doc/design/assistant/leftovers.mdwn | 1 +
 1 file changed, 1 insertion(+)

diff --git a/doc/design/assistant/leftovers.mdwn b/doc/design/assistant/leftovers.mdwn
index 313544d384..c322a27812 100644
--- a/doc/design/assistant/leftovers.mdwn
+++ b/doc/design/assistant/leftovers.mdwn
@@ -1,5 +1,6 @@
 Things that don't fit anywhere else:
 
+* Automatically start daemon on boot or when user logs in.
 * Somehow get content that is unavailable. This is problematic with inotify,
   since we only get an event once the user has tried (and failed) to read
   from the file. This is only needed if all the files in the directory 

From 4127e92b3aedd265e5f9676a9775e637c3f79f6d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 16:57:32 -0400
Subject: [PATCH 162/220] update

---
 doc/design/assistant/configurators.mdwn | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/design/assistant/configurators.mdwn b/doc/design/assistant/configurators.mdwn
index e03723e946..b9e3a66328 100644
--- a/doc/design/assistant/configurators.mdwn
+++ b/doc/design/assistant/configurators.mdwn
@@ -7,6 +7,8 @@ of files, etc.
 The meat of the configuration will be in configuration assistants that walk
 through setting up common use cases.
 
+* Create a repository (run when the web app is started without a configured
+  repository too).
 * Clone this repo to a USB drive.
 * Clone this repo to another host:
   1. Prompt for the hostname (or do avahi local machine discovery). 

From 94649c12704d02314e89e792df90547ed6e96cae Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 21:13:50 +0000
Subject: [PATCH 163/220] Added a comment

---
 ...nt_3_b2f15a46620385da26d5fe8f11ebfc1a._comment | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_3_b2f15a46620385da26d5fe8f11ebfc1a._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_3_b2f15a46620385da26d5fe8f11ebfc1a._comment b/doc/walkthrough/using_ssh_remotes/comment_3_b2f15a46620385da26d5fe8f11ebfc1a._comment
new file mode 100644
index 0000000000..75a133d840
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_3_b2f15a46620385da26d5fe8f11ebfc1a._comment
@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 3"
+ date="2012-05-27T21:13:50Z"
+ content="""
+Thanks for the quick replay!
+
+I already did 'git annex sync', but it didn't work. The steps were: 'git clone ssh...', then 'cd annex', then 'git annex init \"laptop\"'
+
+After that, I did a 'git annex sync', and tried to get the file, but nothing happens. That's why I found it weird.
+Any other thing that might have happened?
+
+Thanks again!
+"""]]

From dd6f92bf6374457c4e371766c0cccfe9573abf77 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Sun, 27 May 2012 21:33:11 +0000
Subject: [PATCH 164/220] Added a comment

---
 .../comment_4_433ccc87fbb0a13e32d59d77f0b4e56c._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_4_433ccc87fbb0a13e32d59d77f0b4e56c._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_4_433ccc87fbb0a13e32d59d77f0b4e56c._comment b/doc/walkthrough/using_ssh_remotes/comment_4_433ccc87fbb0a13e32d59d77f0b4e56c._comment
new file mode 100644
index 0000000000..3df03abc2c
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_4_433ccc87fbb0a13e32d59d77f0b4e56c._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 4"
+ date="2012-05-27T21:33:11Z"
+ content="""
+Try running `git annex whereis` on the file and see where it says it is.
+"""]]

From 7b09d83db2db5b67c98854902dc5a639d0202a2c Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 21:41:58 +0000
Subject: [PATCH 165/220] Added a comment

---
 ...5_da860d0f8c8772062d28d063942e1af7._comment | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment b/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment
new file mode 100644
index 0000000000..d1091eb8c8
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 5"
+ date="2012-05-27T21:41:57Z"
+ content="""
+Hi,
+
+I guess the problem is with git-annex-shell. I tried to do 'git annex get file --from name_ssh_repo', and I got the following:
+
+bash: git-annex-shell: command not found; failed; exit code 127
+
+The same thing happens if I try to do 'git annex whereis'
+
+git-annex-shell it is indeed installed. How can I make my shell recognize this command?
+
+Thanks a lot!
+"""]]

From 3f5b0fc6cd89db6e2bf44b8bc1f44c1b1fb1346e Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 21:42:40 +0000
Subject: [PATCH 166/220] removed

---
 ...5_da860d0f8c8772062d28d063942e1af7._comment | 18 ------------------
 1 file changed, 18 deletions(-)
 delete mode 100644 doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment b/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment
deleted file mode 100644
index d1091eb8c8..0000000000
--- a/doc/walkthrough/using_ssh_remotes/comment_5_da860d0f8c8772062d28d063942e1af7._comment
+++ /dev/null
@@ -1,18 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
- nickname="Fernando Seabra"
- subject="comment 5"
- date="2012-05-27T21:41:57Z"
- content="""
-Hi,
-
-I guess the problem is with git-annex-shell. I tried to do 'git annex get file --from name_ssh_repo', and I got the following:
-
-bash: git-annex-shell: command not found; failed; exit code 127
-
-The same thing happens if I try to do 'git annex whereis'
-
-git-annex-shell it is indeed installed. How can I make my shell recognize this command?
-
-Thanks a lot!
-"""]]

From b8d36b66678672958382d19d83754a4d65691e57 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 21:42:57 +0000
Subject: [PATCH 167/220] Added a comment

---
 ...5_a9805c7965da0b88a1c9f7f207c450a1._comment | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_5_a9805c7965da0b88a1c9f7f207c450a1._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_5_a9805c7965da0b88a1c9f7f207c450a1._comment b/doc/walkthrough/using_ssh_remotes/comment_5_a9805c7965da0b88a1c9f7f207c450a1._comment
new file mode 100644
index 0000000000..703b89ebfa
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_5_a9805c7965da0b88a1c9f7f207c450a1._comment
@@ -0,0 +1,18 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 5"
+ date="2012-05-27T21:42:56Z"
+ content="""
+Hi,
+
+I guess the problem is with git-annex-shell. I tried to do 'git annex get file --from name_ssh_repo', and I got the following:
+
+bash: git-annex-shell: command not found; failed; exit code 127
+
+The same thing happens if I try to do 'git annex whereis'
+
+git-annex-shell is indeed installed. How can I make my shell recognize this command?
+
+Thanks a lot!
+"""]]

From 6de910312cf8a9f089cc7dda89e0ebc95b8a87a2 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Sun, 27 May 2012 22:08:50 +0000
Subject: [PATCH 168/220] Added a comment

---
 ...mment_6_9d5c12c056892b706cf100ea01866685._comment | 12 ++++++++++++
 1 file changed, 12 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_6_9d5c12c056892b706cf100ea01866685._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_6_9d5c12c056892b706cf100ea01866685._comment b/doc/walkthrough/using_ssh_remotes/comment_6_9d5c12c056892b706cf100ea01866685._comment
new file mode 100644
index 0000000000..4d5961ca90
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_6_9d5c12c056892b706cf100ea01866685._comment
@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 6"
+ date="2012-05-27T22:08:50Z"
+ content="""
+git-annex-shell needs to be installed in the `PATH` on any host that will hold annexed files. 
+
+If you installed with cabal, it might be `.cabal/bin/`. Whereever it was installed to is apparently not on the PATH that is set when you ssh into that host. 
+
+
+"""]]

From 56468d41f292779dbdaac6c21a0b1938dcda1a68 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
 
Date: Sun, 27 May 2012 23:35:19 +0000
Subject: [PATCH 169/220] Added a comment

---
 ...t_7_725e7dbb2d0a74a035127cb01ee0442c._comment | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 doc/walkthrough/using_ssh_remotes/comment_7_725e7dbb2d0a74a035127cb01ee0442c._comment

diff --git a/doc/walkthrough/using_ssh_remotes/comment_7_725e7dbb2d0a74a035127cb01ee0442c._comment b/doc/walkthrough/using_ssh_remotes/comment_7_725e7dbb2d0a74a035127cb01ee0442c._comment
new file mode 100644
index 0000000000..700b170ad6
--- /dev/null
+++ b/doc/walkthrough/using_ssh_remotes/comment_7_725e7dbb2d0a74a035127cb01ee0442c._comment
@@ -0,0 +1,16 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkaT0B6s9jQuMzQUYRVBgWqtO7BhT_ZSaE"
+ nickname="Fernando Seabra"
+ subject="comment 7"
+ date="2012-05-27T23:35:17Z"
+ content="""
+Hi,
+
+It was already installed in PATH. In fact, I can call it from the command line, and it is recognized (e.g. calling 'git-annex-shell' gives me 'git-annex-shell: bad parameters'). However, every time I do a 'git annex whereis' or 'git annex get file --from repo', it gives me the following error:
+
+bash: git-annex-shell: command not found
+Command ssh [\"-S\",\"/Users/username/annex/.git/annex/ssh/username@example.edu\",\"-o\",\"ControlMaster=auto\",\"-o\",\"ControlPersist=yes\",\"username@example.edu\",\"git-annex-shell 'configlist' '/~/annex'\"] failed; exit code 127
+
+I tried to run this ssh command, but it gives me the same 'command not found' error. It seems that the problem is with the ssh repo?
+The ssh repo has a git-annex-shell working and installed in PATH.
+"""]]

From f945c5231d0281c7161323d1d8905a0568f07da4 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 20:06:36 -0400
Subject: [PATCH 170/220] format

---
 doc/special_remotes/rsync.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/special_remotes/rsync.mdwn b/doc/special_remotes/rsync.mdwn
index 286615460c..273469258b 100644
--- a/doc/special_remotes/rsync.mdwn
+++ b/doc/special_remotes/rsync.mdwn
@@ -16,7 +16,7 @@ These parameters can be passed to `git annex initremote` to configure rsync:
 
 * `encryption` - Required. Either "none" to disable encryption of content
   stored in rsync, 
-  or a value that can be looked up (using gpg -k) to find a gpg encryption
+  or a value that can be looked up (using `gpg -k`) to find a gpg encryption
   key that will be given access to the remote. Note that additional gpg
   keys can be given access to a remote by rerunning initremote with
   the new key id. See [[encryption]].

From f120dbc73587bbff42910dce9eb26b79cebe424f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 20:36:01 -0400
Subject: [PATCH 171/220] add manual cabal instuctions, I'm told the AUR may
 have dependency issues

---
 doc/install/ArchLinux.mdwn | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/doc/install/ArchLinux.mdwn b/doc/install/ArchLinux.mdwn
index e531fc968e..68e8b81f1c 100644
--- a/doc/install/ArchLinux.mdwn
+++ b/doc/install/ArchLinux.mdwn
@@ -7,3 +7,13 @@ such as yaourt:
 
 $ yaourt -Sy git-annex
 

+
+----
+
+I'm told the AUR has some dependency problems currently.
+If it doesn't work, you can just use cabal:
+
+
+pacman -S git rsync curl wget gpg openssh cabal-install
+cabal install git-annex --bindir=$HOME/bin
+


From 6e213d04f1233af3faffcd49acf479be086c71f4 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 27 May 2012 20:55:56 -0400
Subject: [PATCH 172/220] sync: Show a nicer message if a user tries to sync to
 a special remote.

---
 Command/Sync.hs  | 9 ++++++++-
 debian/changelog | 6 ++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/Command/Sync.hs b/Command/Sync.hs
index b9ef0bc979..5fb49d30c5 100644
--- a/Command/Sync.hs
+++ b/Command/Sync.hs
@@ -57,10 +57,17 @@ syncRemotes rs = ifM (Annex.getState Annex.fast) ( nub <$> pickfast , wanted )
 		wanted
 			| null rs = good =<< concat . byspeed <$> available
 			| otherwise = listed
-		listed = catMaybes <$> mapM (Remote.byName . Just) rs
+		listed = do
+			l <- catMaybes <$> mapM (Remote.byName . Just) rs
+			let s = filter special l
+			unless (null s) $
+				error $ "cannot sync special remotes: " ++
+					unwords (map Types.Remote.name s)
+			return l
 		available = filter nonspecial <$> Remote.enabledRemoteList
 		good = filterM $ Remote.Git.repoAvail . Types.Remote.repo
 		nonspecial r = Types.Remote.remotetype r == Remote.Git.remote
+		special = not . nonspecial
 		fastest = fromMaybe [] . headMaybe . byspeed
 		byspeed = map snd . sort . M.toList . costmap
 		costmap = M.fromListWith (++) . map costpair
diff --git a/debian/changelog b/debian/changelog
index 99594c351c..9c38d1727f 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+git-annex (3.20120523) UNRELEASED; urgency=low
+
+  * sync: Show a nicer message if a user tries to sync to a special remote.
+
+ -- Joey Hess   Sun, 27 May 2012 20:55:29 -0400
+
 git-annex (3.20120522) unstable; urgency=low
 
   * Pass -a to cp even when it supports --reflink=auto, to preserve

From 7a505a807b2f05046bc8d0093066a915af44279b Mon Sep 17 00:00:00 2001
From: "https://launchpad.net/~gdr-go2" 
Date: Mon, 28 May 2012 18:12:49 +0000
Subject: [PATCH 173/220] Added a comment: FAT symlinks

---
 .../comment_1_8be9a74e5fc4641c2bf2e1bb7673dd59._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/design/assistant/android/comment_1_8be9a74e5fc4641c2bf2e1bb7673dd59._comment

diff --git a/doc/design/assistant/android/comment_1_8be9a74e5fc4641c2bf2e1bb7673dd59._comment b/doc/design/assistant/android/comment_1_8be9a74e5fc4641c2bf2e1bb7673dd59._comment
new file mode 100644
index 0000000000..389eac026d
--- /dev/null
+++ b/doc/design/assistant/android/comment_1_8be9a74e5fc4641c2bf2e1bb7673dd59._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://launchpad.net/~gdr-go2"
+ nickname="gdr-go2"
+ subject="FAT symlinks"
+ date="2012-05-28T18:12:10Z"
+ content="""
+It's a linux kernel so perhaps another option would be to create a big file and mount -o loop
+"""]]

From 5a270415ef8ce94fee3ed1eb6ddc00f6e579dfcc Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 14:25:33 -0400
Subject: [PATCH 174/220] update

---
 doc/design/assistant/android.mdwn | 64 +++++++++++++++++++++++++------
 1 file changed, 52 insertions(+), 12 deletions(-)

diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
index 24791501da..e8f192748c 100644
--- a/doc/design/assistant/android.mdwn
+++ b/doc/design/assistant/android.mdwn
@@ -3,6 +3,8 @@ Porting git-annex to Android will use the Android native SDK.
 A hopefully small Java app will be developed, which runs the webapp
 daemon, and a web browser to display it.
 
+[[!toc]]
+
 ### programs to port
 
 These will probably need to be bundled into the Android app, unless already
@@ -11,21 +13,59 @@ available in the App Store.
 * ssh (native ssh needed for scp, not a client like ConnectBot)
 * rsync
 * gpg
-* git (not all git commands are needed, but a core plumbing and a few like `git-add` are.)
+* git (not all git commands are needed,
+  but core plumbing and a few like `git-add` are.)
 
 ### FAT sucks
 
 The main media partition will use some awful FAT filesystem format from
-1982 that cannot support git-annex's symlinks. Hopefully it can at least
-handle all of git's filenames. Possible approaches to this:
-
-* Keep only a bare git repo on Android. The app would then need to include
-  a file browser to access the files in there, and adding a file would move
-  it into the repo. Not ideal.
-* Implement [[smudge]] filters to avoid needing symlinks. Difficult.
-* Use a bare git repo but don't keep files in `annex/objects`, instead
-  leave them outside the repo, and add some local mapping to find them.
-  Seems best?
-* Use a `LD_PRELOAD` wrapper to do Something Crazy.
+1982 that cannot support git-annex's symlinks. (Hopefully it can at least
+handle all of git's filenames.) Possible approaches to this follow.
 
 (May want to consider which of these would make a Windows port easier too.)
+
+#### bare git repo with file browser
+
+Keep only a bare git repo on Android. The app would then need to include
+a file browser to access the files in there, and adding a file would move
+it into the repo. 
+
+Not ideal.
+
+#### implement git smudge filters
+
+See [[smudge]].
+
+Difficult. Would make git-annex generally better.
+
+#### keep files outside bare git repo
+
+Use a bare git repo but don't keep files in `annex/objects`, instead
+leave them outside the repo, and add some local mapping to find them.
+
+Problem: Would leave files unlocked to modification, which might lose a
+version git-annex dependend upon existing on the phone. (Maybe the phone
+would have to be always considered an untrusted repo, which probably
+makes sense anyway.)
+
+Problem: 
+
+#### crazy `LD_PRELOAD` wrapper
+
+Need I say more? (Also, Android's linker may not even support it.)
+
+### partial content
+
+On a regular system, a reasonable simplifying assumption is that all the
+files in the folder will be synced to the system. A user might want to
+disable syncing of some subdirectories, for eg, archived files. But in
+general, things are simpler to understand and implement if all files sync.
+
+But, a phone probably cannot hold all a user's files. Indeed, it's likely
+that old files will be aggressively dropped from the phone after syncing to
+elsewhere, in order to keep enough free space on it for new files. 
+
+There needs to be a way for the user to browse files not on the phone and
+request they be transferred to it. This could be done as a browser in the
+web app, or using a subdirectory full of placeholder files (not symlinks;
+see above) that start transfer of the real file when accessed.

From 5d178d2aca257347fa0d402a933a45a350bd8ad5 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 14:29:21 -0400
Subject: [PATCH 175/220] reorg

---
 doc/design/assistant.mdwn                 |  2 +-
 doc/design/assistant/android.mdwn         | 18 ------------------
 doc/design/assistant/partial_content.mdwn | 14 ++++++++++++++
 3 files changed, 15 insertions(+), 19 deletions(-)
 create mode 100644 doc/design/assistant/partial_content.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 3bbd27c581..9a9e4fd9fa 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -11,7 +11,7 @@ Feel free to chip in with comments! --[[Joey]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
 * Month 4 "release": [[!traillink leftovers]]; release
-* Months 5-6 "9k bonus round": [[!traillink Android]]
+* Months 5-6 "9k bonus round": [[!traillink Android]] [[!traillink partial_content]]
 
 ## not yet on the map:
 
diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
index e8f192748c..d936d53cc1 100644
--- a/doc/design/assistant/android.mdwn
+++ b/doc/design/assistant/android.mdwn
@@ -3,8 +3,6 @@ Porting git-annex to Android will use the Android native SDK.
 A hopefully small Java app will be developed, which runs the webapp
 daemon, and a web browser to display it.
 
-[[!toc]]
-
 ### programs to port
 
 These will probably need to be bundled into the Android app, unless already
@@ -53,19 +51,3 @@ Problem:
 #### crazy `LD_PRELOAD` wrapper
 
 Need I say more? (Also, Android's linker may not even support it.)
-
-### partial content
-
-On a regular system, a reasonable simplifying assumption is that all the
-files in the folder will be synced to the system. A user might want to
-disable syncing of some subdirectories, for eg, archived files. But in
-general, things are simpler to understand and implement if all files sync.
-
-But, a phone probably cannot hold all a user's files. Indeed, it's likely
-that old files will be aggressively dropped from the phone after syncing to
-elsewhere, in order to keep enough free space on it for new files. 
-
-There needs to be a way for the user to browse files not on the phone and
-request they be transferred to it. This could be done as a browser in the
-web app, or using a subdirectory full of placeholder files (not symlinks;
-see above) that start transfer of the real file when accessed.
diff --git a/doc/design/assistant/partial_content.mdwn b/doc/design/assistant/partial_content.mdwn
new file mode 100644
index 0000000000..5572811d48
--- /dev/null
+++ b/doc/design/assistant/partial_content.mdwn
@@ -0,0 +1,14 @@
+On a regular system, a reasonable simplifying assumption is that all the
+files in the folder will be synced to the system. A user might want to
+disable syncing of some subdirectories, for eg, archived files. But in
+general, things are simpler to understand and implement if all files sync.
+
+But, an Android gadget probably cannot hold all a user's files. Indeed,
+it's likely that old files will be aggressively dropped from the Android
+after syncing to elsewhere, in order to keep enough free space on it for
+new files. 
+
+There needs to be a way for the user to browse files not on the gadget and
+request they be transferred to it. This could be done as a browser in the
+web app, or using a subdirectory full of placeholder files (not symlinks;
+see [[Android]]) that start transfer of the real file when accessed.

From a5da677b9e37747dfb86eda998ce92bd5ff80d03 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 14:41:23 -0400
Subject: [PATCH 176/220] update

---
 doc/design/assistant.mdwn         |  2 +-
 doc/design/assistant/cloud.mdwn   | 28 ++++++++++++++++++++++++++++
 doc/design/assistant/syncing.mdwn |  6 +++---
 3 files changed, 32 insertions(+), 4 deletions(-)
 create mode 100644 doc/design/assistant/cloud.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 9a9e4fd9fa..63f3c56d6f 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -10,7 +10,7 @@ Feel free to chip in with comments! --[[Joey]]
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
 * Month 3 "easy setup": [[!traillink configurators]]
-* Month 4 "release": [[!traillink leftovers]]; release
+* Month 4 "polishing": [[!traillink cloud]] [[!traillink leftovers]]
 * Months 5-6 "9k bonus round": [[!traillink Android]] [[!traillink partial_content]]
 
 ## not yet on the map:
diff --git a/doc/design/assistant/cloud.mdwn b/doc/design/assistant/cloud.mdwn
new file mode 100644
index 0000000000..1d612feac3
--- /dev/null
+++ b/doc/design/assistant/cloud.mdwn
@@ -0,0 +1,28 @@
+The [[syncing]] design assumes the network is connected. But it's often
+not in these pre-IPV6 days, so the cloud needs to be used to bridge between
+LANS.
+
+## more cloud providers
+
+Git-annex already supports several cloud providers via [[special_remotes].
+More should be added, such as:
+
+* Google drive (attractive because it's free)
+* OpenStack Swift
+* Box.com (it's free, and current method is hard to set up and a sorta
+  shakey)
+* Dropbox? That would be ironic..
+
+## limited space
+
+When syncing via the cloud, space there is probably limited, so
+users with more files than cloud space will want to be able to use the
+cloud as a temporary transfer point, which files are removed from after
+they've propigated out.
+
+Other users will want to use the cloud as the canonical or backup location
+of their data, and would want a copy of all their files to be kept there.
+That's also ok.
+
+git-annex will need a way to tell the difference between these, either
+heuristically, or via configuration.
diff --git a/doc/design/assistant/syncing.mdwn b/doc/design/assistant/syncing.mdwn
index 9cbdddcb96..0813b8b70b 100644
--- a/doc/design/assistant/syncing.mdwn
+++ b/doc/design/assistant/syncing.mdwn
@@ -29,8 +29,8 @@ This probably will need lots of refinements to get working well.
 
 ## other considerations
 
-This assumes the network is connected. It's often not, so the
-cloud needs to be used to bridge between LANs.
-
 It would be nice if, when a USB drive is connected,
 syncing starts automatically.
+
+This assumes the network is connected. It's often not, so the
+[[cloud]] needs to be used to bridge between LANs.

From 98adee4369b97036dd69a9c145a10c631fbae69d Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 14:43:27 -0400
Subject: [PATCH 177/220] typos

---
 doc/design/assistant/cloud.mdwn  | 2 +-
 doc/design/assistant/deltas.mdwn | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/design/assistant/cloud.mdwn b/doc/design/assistant/cloud.mdwn
index 1d612feac3..7759bb7993 100644
--- a/doc/design/assistant/cloud.mdwn
+++ b/doc/design/assistant/cloud.mdwn
@@ -4,7 +4,7 @@ LANS.
 
 ## more cloud providers
 
-Git-annex already supports several cloud providers via [[special_remotes].
+Git-annex already supports several cloud providers via [[special_remotes]].
 More should be added, such as:
 
 * Google drive (attractive because it's free)
diff --git a/doc/design/assistant/deltas.mdwn b/doc/design/assistant/deltas.mdwn
index cf2d9f6c34..ff4185a18f 100644
--- a/doc/design/assistant/deltas.mdwn
+++ b/doc/design/assistant/deltas.mdwn
@@ -1,4 +1,4 @@
-Speed up syncing of modified versions of to existing files. 
+Speed up syncing of modified versions of existing files. 
 
 One simple way is to find the key of the old version of a file that's
 being transferred, so it can be used as the basis for rsync, or any

From 8139731708290a516a844dc2cdbeab031e185119 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 14:47:16 -0400
Subject: [PATCH 178/220] update

---
 doc/design/assistant/cloud.mdwn | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/design/assistant/cloud.mdwn b/doc/design/assistant/cloud.mdwn
index 7759bb7993..d8e7f09204 100644
--- a/doc/design/assistant/cloud.mdwn
+++ b/doc/design/assistant/cloud.mdwn
@@ -7,11 +7,11 @@ LANS.
 Git-annex already supports several cloud providers via [[special_remotes]].
 More should be added, such as:
 
-* Google drive (attractive because it's free)
-* OpenStack Swift
+* Google drive (attractive because it's free, only 5 gb tho)
+* OpenStack Swift (teh future)
 * Box.com (it's free, and current method is hard to set up and a sorta
-  shakey)
-* Dropbox? That would be ironic..
+  shakey; a better method would be to use its API)
+* Dropbox? That would be ironic.. Via its API, presumably.
 
 ## limited space
 

From 71c3196356539bccb533828b6cb01fdffa2bedc6 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 15:07:32 -0400
Subject: [PATCH 179/220] update OSX instructions to use ghc7.0 branch

Needed until the next Haskell Platform provides a new ghc.
---
 doc/install/OSX.mdwn | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/doc/install/OSX.mdwn b/doc/install/OSX.mdwn
index f65e0bb4fa..ffd9424372 100644
--- a/doc/install/OSX.mdwn
+++ b/doc/install/OSX.mdwn
@@ -1,4 +1,6 @@
-Install Haskel Platform from [[http://hackage.haskell.org/platform/mac.html]]. The version provided by Macports is too old to work with current versions of git-annex. Then execute
+Install Haskel Platform from [[http://hackage.haskell.org/platform/mac.html]].
+The version provided by Macports is too old to work with current versions of git-annex.
+Then execute
 
 
 sudo port install git-core ossp-uuid md5sha1sum coreutils pcre
@@ -8,8 +10,15 @@ sudo ln -s /opt/local/include/pcre.h  /usr/include/pcre.h # This is hack that al
 # optional: this will enable the gnu tools, (to give sha224sum etc..., it does not override the BSD userland)
 export PATH=$PATH:/opt/local/libexec/gnubin
 
+git clone git://git-annex.branchable.com/ git-annex
+cd git-annex
+git checkout ghc7.0
+
 sudo cabal update
-cabal install git-annex --bindir=$HOME/bin
+cabal install --only-dependencies
+cabal configure
+cabal build
+cabal install --bindir=$HOME/bin
 

 
 Originally posted by Jon at  --[[Joey]], modified by [[kristianrumberg]]

From 4ad541b1c8f0051690b81a01e64c3c696a5b1aea Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Mon, 28 May 2012 18:25:54 -0400
Subject: [PATCH 180/220] note

---
 doc/design/assistant/cloud.mdwn | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/doc/design/assistant/cloud.mdwn b/doc/design/assistant/cloud.mdwn
index d8e7f09204..ae850c7203 100644
--- a/doc/design/assistant/cloud.mdwn
+++ b/doc/design/assistant/cloud.mdwn
@@ -26,3 +26,5 @@ That's also ok.
 
 git-annex will need a way to tell the difference between these, either
 heuristically, or via configuration.
+
+Also needed for USB keys and Android gadgets.

From a1195e92a33ce535502168eaa7ee44c95d20b544 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Tue, 29 May 2012 12:24:27 +0000
Subject: [PATCH 181/220] Added a comment: ANNEX_S3 vs AWS for keys

---
 .../comment_1_666a26f95024760c99c627eed37b1966._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/tips/using_Amazon_S3/comment_1_666a26f95024760c99c627eed37b1966._comment

diff --git a/doc/tips/using_Amazon_S3/comment_1_666a26f95024760c99c627eed37b1966._comment b/doc/tips/using_Amazon_S3/comment_1_666a26f95024760c99c627eed37b1966._comment
new file mode 100644
index 0000000000..60d96cb44e
--- /dev/null
+++ b/doc/tips/using_Amazon_S3/comment_1_666a26f95024760c99c627eed37b1966._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
+ nickname="Matt"
+ subject="ANNEX_S3 vs AWS for keys"
+ date="2012-05-29T12:24:25Z"
+ content="""
+The instructions state ANNEX_S3_ACCESS_KEY_ID and ANNEX_SECRET_ACCESS_KEY but git-annex cannot connect with those constants. git-annex tells me to set both \"AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY\" instead, which works. This is with Xubuntu 12.04.
+"""]]

From f1bcd7c218899ec455b454e2affc0235ab969e6f Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Tue, 29 May 2012 12:40:25 +0000
Subject: [PATCH 182/220] Added a comment: environment variables

---
 .../comment_1_4a1f7a230dad6caa84831685b236fd73._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/special_remotes/S3/comment_1_4a1f7a230dad6caa84831685b236fd73._comment

diff --git a/doc/special_remotes/S3/comment_1_4a1f7a230dad6caa84831685b236fd73._comment b/doc/special_remotes/S3/comment_1_4a1f7a230dad6caa84831685b236fd73._comment
new file mode 100644
index 0000000000..17e35e7d99
--- /dev/null
+++ b/doc/special_remotes/S3/comment_1_4a1f7a230dad6caa84831685b236fd73._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
+ nickname="Matt"
+ subject="environment variables"
+ date="2012-05-29T12:40:25Z"
+ content="""
+Just noting that the environment variables `ANNEX_S3_ACCESS_KEY_ID` and `ANNEX_S3_SECRET_ACCESS_KEY` seem to have been changed to `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
+"""]]

From 21ca84cebf825f98d5521bbec2288f04e75e1505 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Tue, 29 May 2012 13:23:57 +0000
Subject: [PATCH 183/220]

---
 doc/forum/Getting_started_with_Amazon_S3.mdwn | 28 +++++++++++++++++++
 1 file changed, 28 insertions(+)
 create mode 100644 doc/forum/Getting_started_with_Amazon_S3.mdwn

diff --git a/doc/forum/Getting_started_with_Amazon_S3.mdwn b/doc/forum/Getting_started_with_Amazon_S3.mdwn
new file mode 100644
index 0000000000..1ee86b57fa
--- /dev/null
+++ b/doc/forum/Getting_started_with_Amazon_S3.mdwn
@@ -0,0 +1,28 @@
+I'm just getting started with git-annex and trying to wrap my head around using it with Amazon S3. I am familiar with using git, but things are a bit different as we can't init a repo at S3 directly.
+
+I've followed http://git-annex.branchable.com/tips/using_Amazon_S3/, and performed:
+
+`git init`

+Initialized empty Git repository in /home/

+`git annex init`

+init  ok

+`git annex initremote s3 type=S3 encryption=FOOBAR bucket=foo`

+initremote s3 (encryption setup with gpg key YGTVT51715TFR) (checking bucket...) (gpg) ok

+`git annex describe s3 "Amazon S3"`

+describe s3 ok

+`git annexx add foo/`

+add foo/bar.txt

+add foo/bar.png

+...etc

+`git annex sync`

+51 files changed, 51 insertions(+)

+create mode 120000 foo/bar.txt

+create mode 120000 foo/bar.png

+...etc

+
+
+Looking at http://git-annex.branchable.com/git-annex/, I thought the files added would then be pushed to S3 by git annex sync, but that doesn't seem to be the case. I've also tried variations of got annex copy, like `git annex copy . --to s3`, without any luck.
+
+Is there a way to push to s3?
+
+Any help is appreciated!

From f97009fffe18570a01ecda6af25807f33999dd1f Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 29 May 2012 15:01:36 -0400
Subject: [PATCH 184/220] correct S3 environment variable names

---
 doc/special_remotes/S3.mdwn   | 4 ++--
 doc/tips/using_Amazon_S3.mdwn | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/special_remotes/S3.mdwn b/doc/special_remotes/S3.mdwn
index d4d3d02388..f34b078ae1 100644
--- a/doc/special_remotes/S3.mdwn
+++ b/doc/special_remotes/S3.mdwn
@@ -6,8 +6,8 @@ See [[tips/using_Amazon_S3]] and
 
 ## configuration
 
-The standard environment variables `ANNEX_S3_ACCESS_KEY_ID` and 
-`ANNEX_S3_SECRET_ACCESS_KEY` are used to supply login credentials
+The standard environment variables `AWS_S3_ACCESS_KEY_ID` and 
+`AWS_S3_SECRET_ACCESS_KEY` are used to supply login credentials
 for Amazon. When encryption is enabled, they are stored in encrypted form
 by `git annex initremote`, so you do not need to keep the environment
 variables set after the initial initalization of the remote.
diff --git a/doc/tips/using_Amazon_S3.mdwn b/doc/tips/using_Amazon_S3.mdwn
index b59ca9b4f8..d941a19f01 100644
--- a/doc/tips/using_Amazon_S3.mdwn
+++ b/doc/tips/using_Amazon_S3.mdwn
@@ -4,8 +4,8 @@ Amazon S3, and use git-annex to transfer files into the cloud.
 
 First, export your S3 credentials:
 
-	# export ANNEX_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
-	# export ANNEX_S3_SECRET_ACCESS_KEY="s3kr1t"
+	# export AWS_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
+	# export AWS_S3_SECRET_ACCESS_KEY="s3kr1t"
 
 Now, create a gpg key, if you don't already have one. This will be used
 to encrypt everything stored in S3, for your privacy. Once you have

From f9b24d1a47a863ea10ada18c4db88ccdc9c5dac7 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Tue, 29 May 2012 19:09:50 +0000
Subject: [PATCH 185/220] Added a comment

---
 ...comment_1_f50883133d5d4903cc95c0dcaa52d052._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/forum/Getting_started_with_Amazon_S3/comment_1_f50883133d5d4903cc95c0dcaa52d052._comment

diff --git a/doc/forum/Getting_started_with_Amazon_S3/comment_1_f50883133d5d4903cc95c0dcaa52d052._comment b/doc/forum/Getting_started_with_Amazon_S3/comment_1_f50883133d5d4903cc95c0dcaa52d052._comment
new file mode 100644
index 0000000000..b2211fa6c1
--- /dev/null
+++ b/doc/forum/Getting_started_with_Amazon_S3/comment_1_f50883133d5d4903cc95c0dcaa52d052._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 1"
+ date="2012-05-29T19:09:50Z"
+ content="""
+`git annex sync` only syncs git metadata, not file contents, and metadata is not stored on S3, so it does notthing (much).
+
+`git annex move . --to s3` or `git annex copy . --to s3` is the right way to send the files to S3. I'm not sure why you say it's not working. I'd try it but Amazon is not letting me sign up for S3 again right now. Can you show what goes wrong with copy?
+"""]]

From 5c82bcf886eb71f773ec42d9648368499707fc14 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Tue, 29 May 2012 19:10:42 +0000
Subject: [PATCH 186/220] Added a comment

---
 .../comment_2_f5a0883be7dbb421b584c6dc0165f1ef._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/tips/using_Amazon_S3/comment_2_f5a0883be7dbb421b584c6dc0165f1ef._comment

diff --git a/doc/tips/using_Amazon_S3/comment_2_f5a0883be7dbb421b584c6dc0165f1ef._comment b/doc/tips/using_Amazon_S3/comment_2_f5a0883be7dbb421b584c6dc0165f1ef._comment
new file mode 100644
index 0000000000..dc809cb126
--- /dev/null
+++ b/doc/tips/using_Amazon_S3/comment_2_f5a0883be7dbb421b584c6dc0165f1ef._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 2"
+ date="2012-05-29T19:10:42Z"
+ content="""
+Thanks, I've fixed that. (You could have too.. this is a wiki ;)
+"""]]

From bb393ea24a8687532164569ca1aa58ac73f264da Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Tue, 29 May 2012 19:10:46 +0000
Subject: [PATCH 187/220] Added a comment

---
 .../comment_2_5b22d67de946f4d34a4a3c7449d32988._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/special_remotes/S3/comment_2_5b22d67de946f4d34a4a3c7449d32988._comment

diff --git a/doc/special_remotes/S3/comment_2_5b22d67de946f4d34a4a3c7449d32988._comment b/doc/special_remotes/S3/comment_2_5b22d67de946f4d34a4a3c7449d32988._comment
new file mode 100644
index 0000000000..f535559aeb
--- /dev/null
+++ b/doc/special_remotes/S3/comment_2_5b22d67de946f4d34a4a3c7449d32988._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 2"
+ date="2012-05-29T19:10:46Z"
+ content="""
+Thanks, I've fixed that. (You could have too.. this is a wiki ;)
+"""]]

From 0fcbf22ed6d4bbf7ac0df7a546a00e164e23b135 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 29 May 2012 19:17:38 -0400
Subject: [PATCH 188/220] updates

---
 doc/design/assistant/cloud.mdwn  | 15 ++++++++++++++-
 doc/design/assistant/webapp.mdwn |  1 +
 2 files changed, 15 insertions(+), 1 deletion(-)

diff --git a/doc/design/assistant/cloud.mdwn b/doc/design/assistant/cloud.mdwn
index ae850c7203..bec3bc36b5 100644
--- a/doc/design/assistant/cloud.mdwn
+++ b/doc/design/assistant/cloud.mdwn
@@ -4,7 +4,8 @@ LANS.
 
 ## more cloud providers
 
-Git-annex already supports several cloud providers via [[special_remotes]].
+Git-annex already supports storing large files in 
+several cloud providers via [[special_remotes]].
 More should be added, such as:
 
 * Google drive (attractive because it's free, only 5 gb tho)
@@ -28,3 +29,15 @@ git-annex will need a way to tell the difference between these, either
 heuristically, or via configuration.
 
 Also needed for USB keys and Android gadgets.
+
+## storing git repos in the cloud
+
+Of course, one option is to just use github etc to store the git repo.
+
+Two things can store git repos in Anazon S3:
+* 
+* 
+
+Another option is to not store the git repo in the cloud, but push/pull
+peer-to-peer. When peers cannot directly talk to one-another, this could be
+bounced through something like XMPP.
diff --git a/doc/design/assistant/webapp.mdwn b/doc/design/assistant/webapp.mdwn
index c0f6b893e1..5322fe8a5f 100644
--- a/doc/design/assistant/webapp.mdwn
+++ b/doc/design/assistant/webapp.mdwn
@@ -14,6 +14,7 @@ The webapp is a web server that displays a shiny interface.
 * progress bars for each file
 * drag and drop to reorder
 * cancel and pause
+* keep it usable w/o javascript, and accessible to blind, etc
 
 ## implementation
 

From 8cca7616ae3a84e64bd3d0a880f09f18b4394b8e Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Wed, 30 May 2012 00:25:22 +0000
Subject: [PATCH 189/220]

---
 doc/special_remotes/S3.mdwn | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/special_remotes/S3.mdwn b/doc/special_remotes/S3.mdwn
index f34b078ae1..195693b3b3 100644
--- a/doc/special_remotes/S3.mdwn
+++ b/doc/special_remotes/S3.mdwn
@@ -6,8 +6,8 @@ See [[tips/using_Amazon_S3]] and
 
 ## configuration
 
-The standard environment variables `AWS_S3_ACCESS_KEY_ID` and 
-`AWS_S3_SECRET_ACCESS_KEY` are used to supply login credentials
+The standard environment variables `AWS_ACCESS_KEY_ID` and 
+`AWS_SECRET_ACCESS_KEY` are used to supply login credentials
 for Amazon. When encryption is enabled, they are stored in encrypted form
 by `git annex initremote`, so you do not need to keep the environment
 variables set after the initial initalization of the remote.

From 66d885aabc5e4afe94e303a2bd723970c6f36e48 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Wed, 30 May 2012 00:26:36 +0000
Subject: [PATCH 190/220] Added a comment

---
 .../comment_3_bcab2bd0f168954243aa9bcc9671bd94._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/special_remotes/S3/comment_3_bcab2bd0f168954243aa9bcc9671bd94._comment

diff --git a/doc/special_remotes/S3/comment_3_bcab2bd0f168954243aa9bcc9671bd94._comment b/doc/special_remotes/S3/comment_3_bcab2bd0f168954243aa9bcc9671bd94._comment
new file mode 100644
index 0000000000..abb6aacc96
--- /dev/null
+++ b/doc/special_remotes/S3/comment_3_bcab2bd0f168954243aa9bcc9671bd94._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
+ nickname="Matt"
+ subject="comment 3"
+ date="2012-05-30T00:26:33Z"
+ content="""
+Thanks! Being new here, I didn't want to overstep my boundaries. I've gone ahead and made a small edit and will do so elsewhere as needed.
+"""]]

From 2150b8790645be32c8dd79b413e147daa8816c24 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Wed, 30 May 2012 00:27:23 +0000
Subject: [PATCH 191/220]

---
 doc/tips/using_Amazon_S3.mdwn | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/tips/using_Amazon_S3.mdwn b/doc/tips/using_Amazon_S3.mdwn
index d941a19f01..128819fcbb 100644
--- a/doc/tips/using_Amazon_S3.mdwn
+++ b/doc/tips/using_Amazon_S3.mdwn
@@ -4,8 +4,8 @@ Amazon S3, and use git-annex to transfer files into the cloud.
 
 First, export your S3 credentials:
 
-	# export AWS_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
-	# export AWS_S3_SECRET_ACCESS_KEY="s3kr1t"
+	# export AWS_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
+	# export AWS_SECRET_ACCESS_KEY="s3kr1t"
 
 Now, create a gpg key, if you don't already have one. This will be used
 to encrypt everything stored in S3, for your privacy. Once you have

From c5412ca7fed26dcc294c5873a07a28f174b255ce Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Tue, 29 May 2012 20:27:52 -0400
Subject: [PATCH 192/220] fix

---
 doc/tips/using_Amazon_S3.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/tips/using_Amazon_S3.mdwn b/doc/tips/using_Amazon_S3.mdwn
index d941a19f01..380326d6f7 100644
--- a/doc/tips/using_Amazon_S3.mdwn
+++ b/doc/tips/using_Amazon_S3.mdwn
@@ -5,7 +5,7 @@ Amazon S3, and use git-annex to transfer files into the cloud.
 First, export your S3 credentials:
 
 	# export AWS_S3_ACCESS_KEY_ID="08TJMT99S3511WOZEP91"
-	# export AWS_S3_SECRET_ACCESS_KEY="s3kr1t"
+	# export AWS_SECRET_ACCESS_KEY="s3kr1t"
 
 Now, create a gpg key, if you don't already have one. This will be used
 to encrypt everything stored in S3, for your privacy. Once you have

From 2a61c5f2869c562ebd4433802c132b9bfac49ec4 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
 
Date: Wed, 30 May 2012 00:40:48 +0000
Subject: [PATCH 193/220] Added a comment

---
 .../comment_2_e90aa3259d9a12cd67daa27d42d69ab5._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Getting_started_with_Amazon_S3/comment_2_e90aa3259d9a12cd67daa27d42d69ab5._comment

diff --git a/doc/forum/Getting_started_with_Amazon_S3/comment_2_e90aa3259d9a12cd67daa27d42d69ab5._comment b/doc/forum/Getting_started_with_Amazon_S3/comment_2_e90aa3259d9a12cd67daa27d42d69ab5._comment
new file mode 100644
index 0000000000..742e8d4469
--- /dev/null
+++ b/doc/forum/Getting_started_with_Amazon_S3/comment_2_e90aa3259d9a12cd67daa27d42d69ab5._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnoUOqs_lbuWyZBqyU6unHgUduJwDDgiKY"
+ nickname="Matt"
+ subject="comment 2"
+ date="2012-05-30T00:40:45Z"
+ content="""
+It's strange. I've done some testing on another machine, and this one, and the issue seems to be with adding only certain sub-directories of the git-annex directory. Would it cause an issue with git-annex if a sub-directory was a git repo?
+"""]]

From 1209972509a6014df2b3d85c1f3fb6f418c7c27d Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawlBSuP2uCSUr5ONRVoyL7jg0ZhBc5us_w0"
 
Date: Wed, 30 May 2012 00:51:29 +0000
Subject: [PATCH 194/220] smudge broken link

---
 doc/design/assistant/android.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
index d936d53cc1..a33029b9db 100644
--- a/doc/design/assistant/android.mdwn
+++ b/doc/design/assistant/android.mdwn
@@ -32,7 +32,7 @@ Not ideal.
 
 #### implement git smudge filters
 
-See [[smudge]].
+See [[todo/smudge]].
 
 Difficult. Would make git-annex generally better.
 

From bdb3c573934728c59c9dab6295cab8d65056bad5 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Wed, 30 May 2012 00:54:38 +0000
Subject: [PATCH 195/220] Added a comment

---
 .../comment_3_c3adce7c0f29e71ed9dd07103ede2c1a._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/forum/Getting_started_with_Amazon_S3/comment_3_c3adce7c0f29e71ed9dd07103ede2c1a._comment

diff --git a/doc/forum/Getting_started_with_Amazon_S3/comment_3_c3adce7c0f29e71ed9dd07103ede2c1a._comment b/doc/forum/Getting_started_with_Amazon_S3/comment_3_c3adce7c0f29e71ed9dd07103ede2c1a._comment
new file mode 100644
index 0000000000..450a1513ce
--- /dev/null
+++ b/doc/forum/Getting_started_with_Amazon_S3/comment_3_c3adce7c0f29e71ed9dd07103ede2c1a._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.81.112"
+ subject="comment 3"
+ date="2012-05-30T00:54:38Z"
+ content="""
+If the subdirectory has a .git, then it's a separate git repo, and inside the directory, all git (and git-annex) commands in it will operate on that nested repo and ignore the outside one.
+"""]]

From 65977a558446fd4208b04614191b68bae838b044 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 30 May 2012 17:01:22 -0400
Subject: [PATCH 196/220] lock: Reset unlocked file to index, rather than to
 branch head.

Resetting an unlocked file to the branch head failed if it had just been
added, not committed, and unlocked, since the branch didbn't have it.

The code was concerned about dropping any changes that might be staged in the
index, but I cannot see why.
---
 Command/Lock.hs                                             | 6 +-----
 debian/changelog                                            | 1 +
 doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn | 6 ++++--
 3 files changed, 6 insertions(+), 7 deletions(-)

diff --git a/Command/Lock.hs b/Command/Lock.hs
index b8aedb252b..ab97b14bcc 100644
--- a/Command/Lock.hs
+++ b/Command/Lock.hs
@@ -24,9 +24,5 @@ start file = do
 
 perform :: FilePath -> CommandPerform
 perform file = do
-	liftIO $ removeFile file
-	-- Checkout from HEAD to get rid of any changes that might be 
-	-- staged in the index, and get back to the previous symlink to
-	-- the content.
-	Annex.Queue.add "checkout" [Param "HEAD", Param "--"] [file]
+	Annex.Queue.add "checkout" [Param "--"] [file]
 	next $ return True -- no cleanup needed
diff --git a/debian/changelog b/debian/changelog
index 9c38d1727f..6b57a5580c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,6 +1,7 @@
 git-annex (3.20120523) UNRELEASED; urgency=low
 
   * sync: Show a nicer message if a user tries to sync to a special remote.
+  * lock: Reset unlocked file to index, rather than to branch head.
 
  -- Joey Hess   Sun, 27 May 2012 20:55:29 -0400
 
diff --git a/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn b/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn
index 69704bdd07..9c093de389 100644
--- a/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn
+++ b/doc/bugs/unlock_then_lock_of_uncommitted_file_loses_it.mdwn
@@ -1,5 +1,7 @@
-Add a file, then unlock it, and then lock it. There is an error and the
-symlink gets deleted. 
+Add a file (do not commit), then unlock it, and then lock it.
+There is an error and the symlink gets deleted.
 
 The file will still be staged in the index, and the file content is still
 in the annex. --[[Joey]] 
+
+[[done]]

From 29b43cfa395c87f722be35fb45cfea94cd3338ea Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Wed, 30 May 2012 20:50:53 -0400
Subject: [PATCH 197/220] pairing

---
 doc/design/assistant.mdwn               |  2 +-
 doc/design/assistant/configurators.mdwn |  7 +------
 doc/design/assistant/pairing.mdwn       | 13 +++++++++++++
 3 files changed, 15 insertions(+), 7 deletions(-)
 create mode 100644 doc/design/assistant/pairing.mdwn

diff --git a/doc/design/assistant.mdwn b/doc/design/assistant.mdwn
index 63f3c56d6f..7a720a5e08 100644
--- a/doc/design/assistant.mdwn
+++ b/doc/design/assistant.mdwn
@@ -9,7 +9,7 @@ Feel free to chip in with comments! --[[Joey]]
 
 * Month 1 "like dropbox": [[!traillink inotify]] [[!traillink syncing]]
 * Month 2 "shiny webapp": [[!traillink webapp]] [[!traillink progressbars]]
-* Month 3 "easy setup": [[!traillink configurators]]
+* Month 3 "easy setup": [[!traillink configurators]] [[!traillink pairing]]
 * Month 4 "polishing": [[!traillink cloud]] [[!traillink leftovers]]
 * Months 5-6 "9k bonus round": [[!traillink Android]] [[!traillink partial_content]]
 
diff --git a/doc/design/assistant/configurators.mdwn b/doc/design/assistant/configurators.mdwn
index b9e3a66328..e0e938efdd 100644
--- a/doc/design/assistant/configurators.mdwn
+++ b/doc/design/assistant/configurators.mdwn
@@ -10,12 +10,7 @@ through setting up common use cases.
 * Create a repository (run when the web app is started without a configured
   repository too).
 * Clone this repo to a USB drive.
-* Clone this repo to another host:
-  1. Prompt for the hostname (or do avahi local machine discovery). 
-  2. Enable the two hosts to ssh to one-another and run git-annex shell.
-     (A tricky problem, if ssh keys need to be added to do that.)
-  3. Push over a clone of the repository. (Using git-annex-shell?)
-  4. Start [[syncing]].
+* Clone this repo to another host. (Needs [[pairing]])
 * Set up Amazon S3.
 * Set up rsync remote.
 * Set up encryption.
diff --git a/doc/design/assistant/pairing.mdwn b/doc/design/assistant/pairing.mdwn
new file mode 100644
index 0000000000..f33c5e11de
--- /dev/null
+++ b/doc/design/assistant/pairing.mdwn
@@ -0,0 +1,13 @@
+For git-annex to be able to clone its repo to another host, it'd be good to
+have some way of pairing devices.
+
+It could work like this:
+
+1. Prompt for the hostname (or do avahi local machine discovery). 
+2. Enable the two hosts to ssh to one-another and run git-annex shell.
+   (A tricky problem, if ssh keys need to be added to do that.)
+3. Push over a clone of the repository. (Using git-annex-shell?)
+4. Start [[syncing]].
+
+Also look into the method used by
+

From 8c759893384d78bde6247ab200fab696113186dc Mon Sep 17 00:00:00 2001
From: "http://dlaxalde.myopenid.com/" 
Date: Thu, 31 May 2012 14:36:33 +0000
Subject: [PATCH 198/220] Added a comment

---
 ...omment_1_cf19b8dc304dc37c26717174c4a98aa4._comment | 11 +++++++++++
 1 file changed, 11 insertions(+)
 create mode 100644 doc/tips/what_to_do_when_you_lose_a_repository/comment_1_cf19b8dc304dc37c26717174c4a98aa4._comment

diff --git a/doc/tips/what_to_do_when_you_lose_a_repository/comment_1_cf19b8dc304dc37c26717174c4a98aa4._comment b/doc/tips/what_to_do_when_you_lose_a_repository/comment_1_cf19b8dc304dc37c26717174c4a98aa4._comment
new file mode 100644
index 0000000000..a7fce26ef8
--- /dev/null
+++ b/doc/tips/what_to_do_when_you_lose_a_repository/comment_1_cf19b8dc304dc37c26717174c4a98aa4._comment
@@ -0,0 +1,11 @@
+[[!comment format=mdwn
+ username="http://dlaxalde.myopenid.com/"
+ nickname="dl"
+ subject="comment 1"
+ date="2012-05-31T14:36:33Z"
+ content="""
+Is there a way to have git-annex completely ignore a repository? I see that
+the `dead` command adds the uuid of the repository to `trust.log` but does
+not change `uuid.log`. Is it enough to remove the corresponding line in
+`uuid.log` and `trust.log`?
+"""]]

From 4f3b83d33043f07cbf94a6260052c1e20737bc7b Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 31 May 2012 17:00:59 +0000
Subject: [PATCH 199/220] Added a comment

---
 .../comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment

diff --git a/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment b/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment
new file mode 100644
index 0000000000..0e19b806c3
--- /dev/null
+++ b/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.8.243"
+ subject="comment 2"
+ date="2012-05-31T17:00:59Z"
+ content="""
+`dead
+"""]]

From 9c72d42cf4881840b51535091c23126857260f98 Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 31 May 2012 17:01:37 +0000
Subject: [PATCH 200/220] Added a comment

---
 .../comment_3_fa9ca81668f5faebf2f61b10f82c97d2._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/tips/what_to_do_when_you_lose_a_repository/comment_3_fa9ca81668f5faebf2f61b10f82c97d2._comment

diff --git a/doc/tips/what_to_do_when_you_lose_a_repository/comment_3_fa9ca81668f5faebf2f61b10f82c97d2._comment b/doc/tips/what_to_do_when_you_lose_a_repository/comment_3_fa9ca81668f5faebf2f61b10f82c97d2._comment
new file mode 100644
index 0000000000..a8d044c287
--- /dev/null
+++ b/doc/tips/what_to_do_when_you_lose_a_repository/comment_3_fa9ca81668f5faebf2f61b10f82c97d2._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ ip="4.153.8.243"
+ subject="comment 3"
+ date="2012-05-31T17:01:37Z"
+ content="""
+`dead` is the best we can do. The automatic merging used on the git-annex branch tends to re-add lines that are deleted in one repo when merging with another that still has them.
+"""]]

From cbace05372b05d0b7a73561e497edcae9b4abd6d Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Thu, 31 May 2012 17:01:51 +0000
Subject: [PATCH 201/220] removed

---
 .../comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment   | 8 --------
 1 file changed, 8 deletions(-)
 delete mode 100644 doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment

diff --git a/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment b/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment
deleted file mode 100644
index 0e19b806c3..0000000000
--- a/doc/tips/what_to_do_when_you_lose_a_repository/comment_2_e2f5d7d3efaaa8833a2cf2e9f1bd9962._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="http://joeyh.name/"
- ip="4.153.8.243"
- subject="comment 2"
- date="2012-05-31T17:00:59Z"
- content="""
-`dead
-"""]]

From 595d13020b4004b1813cc3bc31ed1a8edf2fe511 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 15:25:26 -0400
Subject: [PATCH 202/220] updates

---
 doc/design/assistant/android.mdwn | 13 ++++++++++++-
 doc/design/assistant/inotify.mdwn |  4 +++-
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/doc/design/assistant/android.mdwn b/doc/design/assistant/android.mdwn
index a33029b9db..90dc551794 100644
--- a/doc/design/assistant/android.mdwn
+++ b/doc/design/assistant/android.mdwn
@@ -14,6 +14,14 @@ available in the App Store.
 * git (not all git commands are needed,
   but core plumbing and a few like `git-add` are.)
 
+### Android specific features
+
+The app should be aware of power status, and avoid expensive background
+jobs when low on battery or run flat out when plugged in.
+
+The app should be aware of network status, and avoid expensive data
+transfers when not on wifi. This may need to be configurable.
+
 ### FAT sucks
 
 The main media partition will use some awful FAT filesystem format from
@@ -26,10 +34,13 @@ handle all of git's filenames.) Possible approaches to this follow.
 
 Keep only a bare git repo on Android. The app would then need to include
 a file browser to access the files in there, and adding a file would move
-it into the repo. 
+it into the repo.
 
 Not ideal.
 
+Could be improved some by registering git-annex as a file handling app on
+Android, allowing you to "send to" git-annex.
+
 #### implement git smudge filters
 
 See [[todo/smudge]].
diff --git a/doc/design/assistant/inotify.mdwn b/doc/design/assistant/inotify.mdwn
index be38ef13cd..6bb810a755 100644
--- a/doc/design/assistant/inotify.mdwn
+++ b/doc/design/assistant/inotify.mdwn
@@ -8,7 +8,9 @@ useful, it needs to:
 - notice deleted files and stage the deletion
   (tricky; there's a race with add..)
 - notice renamed files, auto-fix the symlink, and stage the new file location
-- periodically auto-commit staged changes
+- periodically auto-commit staged changes (avoid autocommitting when
+  lots of changes are coming in)
+- tunable delays before adding new files, etc
 - honor .gitignore, not adding files it excludesa
 
 Also to do:

From 6ea9a88ede48fa6d37a0ac476ca775c0edaa56a4 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 15:28:04 -0400
Subject: [PATCH 203/220] update

---
 doc/design/assistant/webapp.mdwn | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/doc/design/assistant/webapp.mdwn b/doc/design/assistant/webapp.mdwn
index 5322fe8a5f..857b5972bf 100644
--- a/doc/design/assistant/webapp.mdwn
+++ b/doc/design/assistant/webapp.mdwn
@@ -16,6 +16,11 @@ The webapp is a web server that displays a shiny interface.
 * cancel and pause
 * keep it usable w/o javascript, and accessible to blind, etc
 
+## other features
+
+* there could be a UI to export a file, which would make it be served up
+  over http by the web app
+
 ## implementation
 
 Hope to use Yesod.

From 2941d30bda3adc7420dd8f800192d7154e960bc2 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 15:48:26 -0400
Subject: [PATCH 204/220] update

---
 doc/design/assistant/webapp.mdwn | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/doc/design/assistant/webapp.mdwn b/doc/design/assistant/webapp.mdwn
index 857b5972bf..abf7b38c94 100644
--- a/doc/design/assistant/webapp.mdwn
+++ b/doc/design/assistant/webapp.mdwn
@@ -7,6 +7,9 @@ The webapp is a web server that displays a shiny interface.
   token. This guards against other users on the same system.
 * I would like to avoid passwords or other authentication methods,
   it's your local system.
+* Alternative for Linux at least would be to write a small program using
+  GTK+ Webkit, that runs the webapp, and can know what user ran it, avoiding
+  needing authentication.
 
 ## interface
 

From 263ee1d067c8ef7361ecdb096ee227011f8d25dd Mon Sep 17 00:00:00 2001
From: josh 
Date: Thu, 31 May 2012 20:02:48 +0000
Subject: [PATCH 205/220] Add explanation of how git on Windows handles
 symlinks.

---
 doc/design/assistant/windows.mdwn | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/doc/design/assistant/windows.mdwn b/doc/design/assistant/windows.mdwn
index 0b176934b9..89c3082a85 100644
--- a/doc/design/assistant/windows.mdwn
+++ b/doc/design/assistant/windows.mdwn
@@ -8,7 +8,8 @@ Stackoverflow has some details.
 
 Make git use them, as it (apparently) does not yet.
 
-(What **does** git do on Windows when it clones a repo with symlinks?)
+Currently, on Windows, git checks out symlinks as files containing the symlink
+target as their contents.
 
 ## POSIX
 

From 233d4020755429785688ab91e99bd260ffbc1490 Mon Sep 17 00:00:00 2001
From: josh 
Date: Thu, 31 May 2012 20:03:02 +0000
Subject: [PATCH 206/220] Fix typo.

---
 doc/design/assistant/windows.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/design/assistant/windows.mdwn b/doc/design/assistant/windows.mdwn
index 89c3082a85..850f98194e 100644
--- a/doc/design/assistant/windows.mdwn
+++ b/doc/design/assistant/windows.mdwn
@@ -15,4 +15,4 @@ target as their contents.
 
 Lots of ifdefs and pain to deal with POSIX calls in the code base.
 
-Or I could try to use Cywin.
+Or I could try to use Cygwin.

From c47b1ecda00cea6d9922e0733c3ab365bc2f004d Mon Sep 17 00:00:00 2001
From: josh 
Date: Thu, 31 May 2012 20:11:43 +0000
Subject: [PATCH 207/220] Discussion of NTFS symlinks, junction points, and
 reparse points.

---
 doc/design/assistant/windows.mdwn | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/doc/design/assistant/windows.mdwn b/doc/design/assistant/windows.mdwn
index 850f98194e..da669ad82c 100644
--- a/doc/design/assistant/windows.mdwn
+++ b/doc/design/assistant/windows.mdwn
@@ -6,6 +6,8 @@ Apparently new versions of Windows have something very like symlinks.
 (Or really, 3 or so things not entirely unlike symlinks and all different.)
 Stackoverflow has some details.
 
+NTFS supports symbolic links two different ways: an [[!wikipedia NTFS symbolic link]] and an [[!wikipedia NTFS_junction_point]].  The former seems like the closest analogue to POSIX symlinks.
+
 Make git use them, as it (apparently) does not yet.
 
 Currently, on Windows, git checks out symlinks as files containing the symlink
@@ -16,3 +18,7 @@ target as their contents.
 Lots of ifdefs and pain to deal with POSIX calls in the code base.
 
 Or I could try to use Cygwin.
+
+## Deeper system integration
+
+[NTFS Reparse Points](http://msdn.microsoft.com/en-us/library/aa365503%28v=VS.85%29.aspx) allow a program to define how the OS will interpret a file or directory in arbitrary ways.  This requires writing a file system filter.

From 3b09281b442e794213f2e296e42e2d74fddec733 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 19:25:33 -0400
Subject: [PATCH 208/220] add dirContentsRecursive

---
 Utility/Directory.hs | 32 ++++++++++++++++++++++++++++----
 1 file changed, 28 insertions(+), 4 deletions(-)

diff --git a/Utility/Directory.hs b/Utility/Directory.hs
index 3041361dfd..5bfd49a9c1 100644
--- a/Utility/Directory.hs
+++ b/Utility/Directory.hs
@@ -17,6 +17,7 @@ import System.FilePath
 import Control.Applicative
 import Control.Exception (bracket_)
 import System.Posix.Directory
+import System.IO.Unsafe (unsafeInterleaveIO)
 
 import Utility.SafeCommand
 import Utility.TempFile
@@ -24,14 +25,37 @@ import Utility.Exception
 import Utility.Monad
 import Utility.Path
 
+dirCruft :: FilePath -> Bool
+dirCruft "." = True
+dirCruft ".." = True
+dirCruft _ = False
+
 {- Lists the contents of a directory.
  - Unlike getDirectoryContents, paths are not relative to the directory. -}
 dirContents :: FilePath -> IO [FilePath]
-dirContents d = map (d ) . filter notcruft <$> getDirectoryContents d
+dirContents d = map (d ) . filter (not . dirCruft) <$> getDirectoryContents d
+
+{- Gets contents of directory, and then its subdirectories, recursively,
+ - and lazily. -}
+dirContentsRecursive :: FilePath -> IO [FilePath]
+dirContentsRecursive topdir = dirContentsRecursive' topdir [""]
+
+dirContentsRecursive' :: FilePath -> [FilePath] -> IO [FilePath]
+dirContentsRecursive' _ [] = return []
+dirContentsRecursive' topdir (dir:dirs) = unsafeInterleaveIO $ do
+	(files, dirs') <- collect [] [] =<< dirContents (topdir  dir)
+	files' <- dirContentsRecursive' topdir (dirs' ++ dirs)
+	return (files ++ files')
 	where
-		notcruft "." = False
-		notcruft ".." = False
-		notcruft _ = True
+		collect files dirs' [] = return (reverse files, reverse dirs')
+		collect files dirs' (entry:entries)
+			| dirCruft entry = collect files dirs' entries
+			| otherwise = do
+				let dirEntry = dir  entry
+				ifM (doesDirectoryExist $ topdir  dirEntry)
+					( collect files (dirEntry:dirs') entries
+					, collect (dirEntry:files) dirs' entries
+					)			
 
 {- Moves one filename to another.
  - First tries a rename, but falls back to moving across devices if needed. -}

From 3a10095d40cf9a9e0380b6b10e1ebe304f1537c0 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 19:47:18 -0400
Subject: [PATCH 209/220] import: New subcommand, pulls files from a directory
 outside the annex and adds them

Use case for this was developed somewhere on the Transiberian Railroad.
---
 Command/Import.hs  | 39 +++++++++++++++++++++++++++++++++++++++
 GitAnnex.hs        |  2 ++
 Seek.hs            | 10 +++++++++-
 debian/changelog   |  2 ++
 doc/git-annex.mdwn |  9 +++++++++
 5 files changed, 61 insertions(+), 1 deletion(-)
 create mode 100644 Command/Import.hs

diff --git a/Command/Import.hs b/Command/Import.hs
new file mode 100644
index 0000000000..e27a421f27
--- /dev/null
+++ b/Command/Import.hs
@@ -0,0 +1,39 @@
+{- git-annex command
+ -
+ - Copyright 2012 Joey Hess 
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Command.Import where
+
+import Common.Annex
+import Command
+import qualified Annex
+import qualified Command.Add
+
+def :: [Command]
+def = [command "import" paramPaths seek "move and add files from outside git working copy"]
+
+seek :: [CommandSeek]
+seek = [withPathContents start]
+
+start :: (FilePath, FilePath) -> CommandStart
+start (srcfile, destfile) = notBareRepo $
+	ifM (liftIO $ isRegularFile <$> getSymbolicLinkStatus srcfile)
+		( do
+			showStart "import" destfile
+			next $ perform srcfile destfile
+		, stop
+		)
+
+perform :: FilePath -> FilePath -> CommandPerform
+perform srcfile destfile = do
+	whenM (liftIO $ doesFileExist destfile) $
+		unlessM (Annex.getState Annex.force) $
+			error $ "not overwriting existing " ++ destfile ++
+				" (use --force to override)"
+ 
+	liftIO $ createDirectoryIfMissing True (parentDir destfile)
+	liftIO $ moveFile srcfile destfile
+	Command.Add.perform destfile
diff --git a/GitAnnex.hs b/GitAnnex.hs
index 9910e33d21..149b37f930 100644
--- a/GitAnnex.hs
+++ b/GitAnnex.hs
@@ -54,6 +54,7 @@ import qualified Command.Semitrust
 import qualified Command.Dead
 import qualified Command.Sync
 import qualified Command.AddUrl
+import qualified Command.Import
 import qualified Command.Map
 import qualified Command.Upgrade
 import qualified Command.Version
@@ -69,6 +70,7 @@ cmds = concat
 	, Command.Lock.def
 	, Command.Sync.def
 	, Command.AddUrl.def
+	, Command.Import.def
 	, Command.Init.def
 	, Command.Describe.def
 	, Command.InitRemote.def
diff --git a/Seek.hs b/Seek.hs
index 8d4f917e72..eed4a81558 100644
--- a/Seek.hs
+++ b/Seek.hs
@@ -4,7 +4,7 @@
  - the values a user passes to a command, and prepare actions operating
  - on them.
  -
- - Copyright 2010-2011 Joey Hess 
+ - Copyright 2010-2012 Joey Hess 
  -
  - Licensed under the GNU GPL version 3 or higher.
  -}
@@ -41,6 +41,14 @@ withFilesNotInGit a params = do
 			g <- gitRepo
 			liftIO $ (\p -> LsFiles.notInRepo force p g) l
 
+withPathContents :: ((FilePath, FilePath) -> CommandStart) -> CommandSeek
+withPathContents a params = map a . concat <$> liftIO (mapM get params)
+	where
+		get p = ifM (isDirectory <$> getFileStatus p)
+			( map (\f -> (f, makeRelative p f)) <$> dirContentsRecursive p
+			, return [(p, takeFileName p)]
+			)
+
 withWords :: ([String] -> CommandStart) -> CommandSeek
 withWords a params = return [a params]
 
diff --git a/debian/changelog b/debian/changelog
index 6b57a5580c..a110e94ce9 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -2,6 +2,8 @@ git-annex (3.20120523) UNRELEASED; urgency=low
 
   * sync: Show a nicer message if a user tries to sync to a special remote.
   * lock: Reset unlocked file to index, rather than to branch head.
+  * import: New subcommand, pulls files from a directory outside the annex
+    and adds them.
 
  -- Joey Hess   Sun, 27 May 2012 20:55:29 -0400
 
diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn
index 5d41f86e9c..c7de59cd2a 100644
--- a/doc/git-annex.mdwn
+++ b/doc/git-annex.mdwn
@@ -160,6 +160,15 @@ subdirectories).
   alternate locations from which the file can be downloaded. In this mode,
   addurl can be used both to add new files, or to add urls to existing files.
 
+* import [path ...]
+
+  Moves files from somewhere outside the git working copy, and adds them to
+  the annex. Individual files to import can be specified. 
+  If a directory is specified, all files in it are imported, and any
+  subdirectory structure inside it is preserved.
+
+	git annex import /media/camera/DCIM/
+
 # REPOSITORY SETUP COMMANDS
 
 * init [description]

From 6fd83851c13232b7810a33e8bb1d83e8a46bd354 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 21:03:24 -0400
Subject: [PATCH 210/220] Fix display of warning message when encountering a
 file that uses an unsupported backend.

---
 Backend.hs       | 8 ++++----
 Locations.hs     | 4 +++-
 debian/changelog | 2 ++
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/Backend.hs b/Backend.hs
index 8071b9b835..fa32669449 100644
--- a/Backend.hs
+++ b/Backend.hs
@@ -75,16 +75,16 @@ genKey' (b:bs) file = do
  - by examining what the file symlinks to. -}
 lookupFile :: FilePath -> Annex (Maybe (Key, Backend))
 lookupFile file = do
-	tl <- liftIO $ tryIO getsymlink
+	tl <- liftIO $ tryIO $ readSymbolicLink file
 	case tl of
 		Left _ -> return Nothing
 		Right l -> makekey l
 	where
-		getsymlink = takeFileName <$> readSymbolicLink file
-		makekey l = maybe (return Nothing) (makeret l) (fileKey l)
+		makekey l = maybe (return Nothing) (makeret l) (fileKey $ takeFileName l)
 		makeret l k = let bname = keyBackendName k in
 			case maybeLookupBackendName bname of
-				Just backend -> return $ Just (k, backend)
+				Just backend -> do
+					return $ Just (k, backend)
 				Nothing -> do
 					when (isLinkToAnnex l) $ warning $
 						"skipping " ++ file ++
diff --git a/Locations.hs b/Locations.hs
index 46a85e0ee1..db456388a6 100644
--- a/Locations.hs
+++ b/Locations.hs
@@ -155,7 +155,9 @@ gitAnnexRemotesDir r = addTrailingPathSeparator $ gitAnnexDir r  "remotes"
 
 {- Checks a symlink target to see if it appears to point to annexed content. -}
 isLinkToAnnex :: FilePath -> Bool
-isLinkToAnnex s = ("/.git/" ++ objectDir) `isInfixOf` s
+isLinkToAnnex s = ("/" ++ d) `isInfixOf` s || d `isPrefixOf` s
+	where
+		d = ".git"  objectDir
 
 {- Converts a key into a filename fragment without any directory.
  -
diff --git a/debian/changelog b/debian/changelog
index a110e94ce9..61290b1aaf 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -4,6 +4,8 @@ git-annex (3.20120523) UNRELEASED; urgency=low
   * lock: Reset unlocked file to index, rather than to branch head.
   * import: New subcommand, pulls files from a directory outside the annex
     and adds them.
+  * Fix display of warning message when encountering a file that uses an
+    unsupported backend.
 
  -- Joey Hess   Sun, 27 May 2012 20:55:29 -0400
 

From 1f951de014d0bfb3d2cf2b58e607cabefdf30d84 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 21:31:25 -0400
Subject: [PATCH 211/220] add a nice one

---
 doc/testimonials.mdwn | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/doc/testimonials.mdwn b/doc/testimonials.mdwn
index 65bc5a5fe2..f053c58398 100644
--- a/doc/testimonials.mdwn
+++ b/doc/testimonials.mdwn
@@ -9,6 +9,13 @@
 
 
 
+

+What excites me about GIT ANNEX is how it fundamentally tracks the
+backup and availability of any data you own, and allows you to share
+data with a large or small audience, ensuring that the data survives.
+

+-- Jason Scott 
+
 Seen on IRC:
 
 oh my god, git-annex is amazing

From f5de183c7168627e9bd969aa48f8215f411bc507 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 21:51:42 -0400
Subject: [PATCH 212/220] sha256sum not so optional

---
 doc/install/OSX.mdwn | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/install/OSX.mdwn b/doc/install/OSX.mdwn
index ffd9424372..08904aef9f 100644
--- a/doc/install/OSX.mdwn
+++ b/doc/install/OSX.mdwn
@@ -7,7 +7,7 @@ sudo port install git-core ossp-uuid md5sha1sum coreutils pcre
 
 sudo ln -s /opt/local/include/pcre.h  /usr/include/pcre.h # This is hack that allows pcre-light to find pcre
 
-# optional: this will enable the gnu tools, (to give sha224sum etc..., it does not override the BSD userland)
+# this will enable the gnu tools, (to give sha256sum etc..., it does not override the BSD userland)
 export PATH=$PATH:/opt/local/libexec/gnubin
 
 git clone git://git-annex.branchable.com/ git-annex

From 2183fd2abd95b6deaa9baef47e2f9c5f865123e1 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 23:15:40 -0400
Subject: [PATCH 213/220] Require that the SHA256 backend can be used when
 building, since it's the default.

---
 Backend/SHA.hs     | 2 +-
 Build/Configure.hs | 9 +++++----
 debian/changelog   | 2 ++
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/Backend/SHA.hs b/Backend/SHA.hs
index 3adac65d8c..c2a6cf9761 100644
--- a/Backend/SHA.hs
+++ b/Backend/SHA.hs
@@ -45,7 +45,7 @@ genBackendE size =
 
 shaCommand :: SHASize -> Maybe String
 shaCommand 1 = SysConfig.sha1
-shaCommand 256 = SysConfig.sha256
+shaCommand 256 = Just SysConfig.sha256
 shaCommand 224 = SysConfig.sha224
 shaCommand 384 = SysConfig.sha384
 shaCommand 512 = SysConfig.sha512
diff --git a/Build/Configure.hs b/Build/Configure.hs
index 341b8840dc..86a3479244 100644
--- a/Build/Configure.hs
+++ b/Build/Configure.hs
@@ -26,15 +26,16 @@ tests =
 	, TestCase "bup" $ testCmd "bup" "bup --version >/dev/null"
 	, TestCase "gpg" $ testCmd "gpg" "gpg --version >/dev/null"
 	, TestCase "ssh connection caching" getSshConnectionCaching
-	] ++ shaTestCases [1, 256, 512, 224, 384]
+	] ++ shaTestCases False [1, 512, 224, 384] ++ shaTestCases True [256]
 
-shaTestCases :: [Int] -> [TestCase]
-shaTestCases l = map make l
+shaTestCases :: Bool -> [Int] -> [TestCase]
+shaTestCases required l = map make l
 	where make n =
 		let
 			cmds = map (\x -> "sha" ++ show n ++ x) ["", "sum"]
 			key = "sha" ++ show n
-		in TestCase key $ maybeSelectCmd key cmds "  Sun, 27 May 2012 20:55:29 -0400
 

From 665c0fbdaafab7b3ea2737f302a8951f26e4f01a Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Thu, 31 May 2012 23:33:07 -0400
Subject: [PATCH 214/220] check at configure time for sha commands in Mac OSX
 location

---
 Build/Configure.hs   | 18 ++++++++++++------
 doc/install/OSX.mdwn |  3 ---
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/Build/Configure.hs b/Build/Configure.hs
index 86a3479244..2f79297ee9 100644
--- a/Build/Configure.hs
+++ b/Build/Configure.hs
@@ -6,6 +6,7 @@ import System.Directory
 import Data.List
 import System.Cmd.Utils
 import Control.Applicative
+import System.FilePath
 
 import Build.TestConfig
 import Utility.SafeCommand
@@ -30,12 +31,17 @@ tests =
 
 shaTestCases :: Bool -> [Int] -> [TestCase]
 shaTestCases required l = map make l
-	where make n =
-		let
-			cmds = map (\x -> "sha" ++ show n ++ x) ["", "sum"]
-			key = "sha" ++ show n
-			selector = if required then selectCmd else maybeSelectCmd
-		in TestCase key $ selector key cmds " [x, osxpath  x]) $
+			map (\x -> "sha" ++ show n ++ x) ["", "sum"]
+		-- Max OSX puts GNU tools outside PATH, so look in
+		-- the location it uses, and remember where to run them
+		-- from.
+		osxpath = "/opt/local/libexec/gnubin"
 
 tmpDir :: String
 tmpDir = "tmp"
diff --git a/doc/install/OSX.mdwn b/doc/install/OSX.mdwn
index 08904aef9f..1d8a3b9437 100644
--- a/doc/install/OSX.mdwn
+++ b/doc/install/OSX.mdwn
@@ -7,9 +7,6 @@ sudo port install git-core ossp-uuid md5sha1sum coreutils pcre
 
 sudo ln -s /opt/local/include/pcre.h  /usr/include/pcre.h # This is hack that allows pcre-light to find pcre
 
-# this will enable the gnu tools, (to give sha256sum etc..., it does not override the BSD userland)
-export PATH=$PATH:/opt/local/libexec/gnubin
-
 git clone git://git-annex.branchable.com/ git-annex
 cd git-annex
 git checkout ghc7.0

From 3812e67a054d5bd479b7a4be6d28027664a89bcb Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
 
Date: Fri, 1 Jun 2012 16:05:06 +0000
Subject: [PATCH 215/220] Added a comment: sha256 ?

---
 .../comment_5_69f71b49e6a434cc3426989102c65edf._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment

diff --git a/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment b/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment
new file mode 100644
index 0000000000..b37e9732fb
--- /dev/null
+++ b/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
+ nickname="Damien"
+ subject="sha256 ?"
+ date="2012-06-01T16:05:05Z"
+ content="""
+homebrew's `md5sha1sum` does not provide a `sha256` command that seems to be required… where does yours come from ?
+"""]]

From ca755617d6fe0835c36371a90db75b3a2b4ce4a4 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
 
Date: Fri, 1 Jun 2012 16:10:17 +0000
Subject: [PATCH 216/220] removed

---
 .../comment_5_69f71b49e6a434cc3426989102c65edf._comment   | 8 --------
 1 file changed, 8 deletions(-)
 delete mode 100644 doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment

diff --git a/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment b/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment
deleted file mode 100644
index b37e9732fb..0000000000
--- a/doc/install/OSX/comment_5_69f71b49e6a434cc3426989102c65edf._comment
+++ /dev/null
@@ -1,8 +0,0 @@
-[[!comment format=mdwn
- username="https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
- nickname="Damien"
- subject="sha256 ?"
- date="2012-06-01T16:05:05Z"
- content="""
-homebrew's `md5sha1sum` does not provide a `sha256` command that seems to be required… where does yours come from ?
-"""]]

From be45ac2fa0d201ec3a7270c33c59be05ec43bab3 Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
 
Date: Fri, 1 Jun 2012 16:13:06 +0000
Subject: [PATCH 217/220] Added a comment: sha256

---
 ...comment_5_50777853f808d57b957f8ce9a0f84b3d._comment | 10 ++++++++++
 1 file changed, 10 insertions(+)
 create mode 100644 doc/install/OSX/comment_5_50777853f808d57b957f8ce9a0f84b3d._comment

diff --git a/doc/install/OSX/comment_5_50777853f808d57b957f8ce9a0f84b3d._comment b/doc/install/OSX/comment_5_50777853f808d57b957f8ce9a0f84b3d._comment
new file mode 100644
index 0000000000..eca1761786
--- /dev/null
+++ b/doc/install/OSX/comment_5_50777853f808d57b957f8ce9a0f84b3d._comment
@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawnHrjHxJAm39x8DR4bnbazQO6H0nMNuY9c"
+ nickname="Damien"
+ subject="sha256"
+ date="2012-06-01T16:13:05Z"
+ content="""
+If you're missing the `sha256sum` command with Homebrew, it's provided by `coreutils`. You have to change your `$PATH` before running `cabal install git-annex.cabal`:
+
+    PATH=\"$(brew --prefix coreutils)/libexec/gnubin:$PATH\"
+"""]]

From f74c287e7eea1900c2155ef200f90ba5f58c7caf Mon Sep 17 00:00:00 2001
From: "http://joeyh.name/" 
Date: Fri, 1 Jun 2012 17:24:29 +0000
Subject: [PATCH 218/220] Added a comment

---
 .../comment_6_18a8df794aa0ddd294dbf17d3d4c7fe2._comment    | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 doc/install/OSX/comment_6_18a8df794aa0ddd294dbf17d3d4c7fe2._comment

diff --git a/doc/install/OSX/comment_6_18a8df794aa0ddd294dbf17d3d4c7fe2._comment b/doc/install/OSX/comment_6_18a8df794aa0ddd294dbf17d3d4c7fe2._comment
new file mode 100644
index 0000000000..5cb813776b
--- /dev/null
+++ b/doc/install/OSX/comment_6_18a8df794aa0ddd294dbf17d3d4c7fe2._comment
@@ -0,0 +1,7 @@
+[[!comment format=mdwn
+ username="http://joeyh.name/"
+ subject="comment 6"
+ date="2012-06-01T17:24:29Z"
+ content="""
+Last night I made it look in /opt/local/libexec/gnubin .. if there's another directory it could look in, let me know. I am reluctant to make it run the brew command directly.
+"""]]

From 0064fc0beb98e7254dc026b614bad2e5518ea5dc Mon Sep 17 00:00:00 2001
From: 
 "https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
 
Date: Sat, 2 Jun 2012 12:06:38 +0000
Subject: [PATCH 219/220] Added a comment

---
 .../comment_1_a48fcfbf97f0a373ea375cd8f07f0fc8._comment   | 8 ++++++++
 1 file changed, 8 insertions(+)
 create mode 100644 doc/design/assistant/comment_1_a48fcfbf97f0a373ea375cd8f07f0fc8._comment

diff --git a/doc/design/assistant/comment_1_a48fcfbf97f0a373ea375cd8f07f0fc8._comment b/doc/design/assistant/comment_1_a48fcfbf97f0a373ea375cd8f07f0fc8._comment
new file mode 100644
index 0000000000..646a03398a
--- /dev/null
+++ b/doc/design/assistant/comment_1_a48fcfbf97f0a373ea375cd8f07f0fc8._comment
@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
+ nickname="Jimmy"
+ subject="comment 1"
+ date="2012-06-02T12:06:37Z"
+ content="""
+Will statically linked binaries be provided for say Linux, OSX and *BSD?  I think having some statically linked binaries will certainly help and appeal to a lot of users.
+"""]]

From 619d765646a23d7f22ac8c0dd256be10a5a278f7 Mon Sep 17 00:00:00 2001
From: Joey Hess 
Date: Sun, 3 Jun 2012 14:27:11 -0400
Subject: [PATCH 220/220] simplify OSX installation instructions

The new Haskell Platform was released today, with the new ghc git-annex's
master branch needs, so cabal should be usable again on OSX.
---
 doc/install/OSX.mdwn | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/doc/install/OSX.mdwn b/doc/install/OSX.mdwn
index 1d8a3b9437..3c24609684 100644
--- a/doc/install/OSX.mdwn
+++ b/doc/install/OSX.mdwn
@@ -1,4 +1,4 @@
-Install Haskel Platform from [[http://hackage.haskell.org/platform/mac.html]].
+Install the Haskell Platform from [[http://hackage.haskell.org/platform/mac.html]].
 The version provided by Macports is too old to work with current versions of git-annex.
 Then execute
 
@@ -7,19 +7,10 @@ sudo port install git-core ossp-uuid md5sha1sum coreutils pcre
 
 sudo ln -s /opt/local/include/pcre.h  /usr/include/pcre.h # This is hack that allows pcre-light to find pcre
 
-git clone git://git-annex.branchable.com/ git-annex
-cd git-annex
-git checkout ghc7.0
-
 sudo cabal update
-cabal install --only-dependencies
-cabal configure
-cabal build
-cabal install --bindir=$HOME/bin
+cabal install git-annex --bindir=$HOME/bin
 

 
-Originally posted by Jon at  --[[Joey]], modified by [[kristianrumberg]]
-
 See also:
 
 * [[forum/OSX__39__s_haskell-platform_statically_links_things]]