Commit graph

3193 commits

Author SHA1 Message Date
Joey Hess
6ef82665de add annex.sshcaching config setting 2012-01-20 17:15:46 -04:00
Joey Hess
47250a153a ssh connection caching
Ssh connection caching is now enabled automatically by git-annex. Only one
ssh connection is made to each host per git-annex run, which can speed some
things up a lot, as well as avoiding repeated password prompts. Concurrent
git-annex processes also share ssh connections. Cached ssh connections are
shut down when git-annex exits.

Note: The rsync special remote does not yet participate in the ssh
connection caching.
2012-01-20 17:14:56 -04:00
Joey Hess
25f998679c typo 2012-01-20 15:06:17 -04:00
Joey Hess
61dbad505d fsck --from remote --fast
Avoids expensive file transfers, at the expense of checking file size
and/or contents.

Required some reworking of the remote code.
2012-01-20 13:23:11 -04:00
Joey Hess
e96726caa3 better design
Avoids possible repeated password prompts, at the cost of a small bit of
locking complication.
2012-01-19 21:15:13 -04:00
Joey Hess
3783ccf252 design 2012-01-19 20:41:20 -04:00
https://www.google.com/accounts/o8/id?id=AItOawnBJ6Dv1glxzzi4qIzGFNa6F-mfHIvv9Ck
7fa95eff5e 2012-01-19 23:10:25 +00:00
Joey Hess
effaa298fa optimise fsck --from normal git remotes
For a local git remote, can symlink the file.
For a git remote using rsync, can preseed any local content.

There are a few reasons to use fsck --from on a normal git remote.
One is if it's using gitosis or similar, and you don't have shell access
to run git annex locally. Another reason could be if you just want to
fsck certian files of a bare remote.
2012-01-19 17:10:44 -04:00
Joey Hess
f35a84fac7 use a different tmp file when fscking remote data
Since the content might be symlinked into place, it's not appropriate to
use withTmp here.
2012-01-19 16:56:07 -04:00
Joey Hess
71cb04bb6d optimize fsck --from directory special remote
No need to copy anything, just symlink to the file.
2012-01-19 16:14:40 -04:00
Joey Hess
06b0cb6224 add tmp flag parameter to retrieveKeyFile 2012-01-19 16:07:36 -04:00
Joey Hess
94aa6b42b5 optimise fsck --from rsync special remote
When a file is present locally, the remote's version can be rsynced to
a copy of it, which will avoid wasting a lot of bandwidth.
2012-01-19 15:49:55 -04:00
Joey Hess
711c154561 update NEWS
Add news item recommending fscking directory special remotes.

Remote news item about URL backend being removed; it was later added back
to be used by git annex addurl --fast.

Link NEWS into top level.
2012-01-19 15:27:39 -04:00
Joey Hess
90319afa41 fsck --from
Fscking a remote is now supported. It's done by retrieving
the contents of the specified files from the remote, and checking them,
so can be an expensive operation.

(Several optimisations are possible, to speed it up, of course.. This is
the slow and stupid remote fsck to start with.)

Still, if the remote is a special remote, or a git repository that you
cannot run fsck in locally, it's nice to have the ability to fsck it.

If you have any directory special remotes, now would be a good time to
fsck them, in case you were hit by the data loss bug fixed in the
previous release!
2012-01-19 15:24:05 -04:00
Joey Hess
d36525e974 convert fsckKey to a Maybe
This way it's clear when a backend does not implement its own fsck checks.
2012-01-19 13:51:30 -04:00
Joey Hess
50c063df06 add 2012-01-18 17:30:21 -04:00
Joey Hess
de96ee7208 thought 2012-01-18 16:19:57 -04:00
Joey Hess
a2428320de add news item for git-annex 3.20120116 2012-01-16 16:52:46 -04:00
Joey Hess
2837e8fef1 releasing version 3.20120116 2012-01-16 16:52:26 -04:00
Joey Hess
f161b5eb59 Fix data loss bug in directory special remote
When moving a file to the remote failed, and partially transferred content
was left behind in the directory, re-running the same move would think it
succeeded and delete the local copy.

I reproduced data loss when moving files to a partition that was almost
full. Interrupting a transfer could have similar results.

Easily fixed by using a temp file which is then moved atomically into place
once the transfer completes.

I've audited other calls to copyFileExternal, and other special remote
file transfer code; everything else seems to use temp files correctly
(rsync, git), or otherwise use atomic transfers (bup, S3).
2012-01-16 16:28:15 -04:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
0c4f12e8a2 Added a comment: Thanks and sorry 2012-01-15 20:54:55 +00:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
ba85dd41a6 un-tangle unrelated posts 2012-01-15 20:53:40 +00:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
a7f1d8c3bc rename forum/__91__Installation__93___There_is_no_available_version_of_quickcheck_that_satisfies___62____61__2.1.mdwn to forum/__91__Installation__93___base-3.0.3.2_requires_syb___61____61__0.1.0.2_however_syb-0.1.0.2_was_excluded_because_json-0.5_requires_syb___62____61__0.3.3.mdwn 2012-01-15 20:53:09 +00:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
728d7e96e0 un-tangle unrelated posts 2012-01-15 20:48:26 +00:00
http://joey.kitenet.net/
847407f860 Added a comment 2012-01-15 19:53:36 +00:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
bc2337c0d9 syb dependency problem. 2012-01-15 19:22:46 +00:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
37d42ed1e7 rename bugs/__91__Installation__93___There_is_no_available_version_of_quickcheck_that_satisfies___62____61__2.1.mdwn to forum/__91__Installation__93___There_is_no_available_version_of_quickcheck_that_satisfies___62____61__2.1.mdwn 2012-01-15 19:20:29 +00:00
Joey Hess
ba6ca1bc23 no, really, I fixed it 2012-01-15 15:07:01 -04:00
Joey Hess
caae56a7d2 Merge branch 'master' of ssh://git-annex.branchable.com 2012-01-15 15:04:35 -04:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
14113f84b5 re-opened. proposed solution doesn't help. 2012-01-15 19:02:06 +00:00
Joey Hess
da6810674c copyright 2012-01-15 14:57:52 -04:00
Joey Hess
e3ea5fe938 debhelper v9
kills that ugly python message during build
2012-01-15 14:53:38 -04:00
Joey Hess
f0c84686b2 add news item for git-annex 3.20120115 2012-01-15 14:03:00 -04:00
Joey Hess
ce608303a3 releasing version 3.20120115 2012-01-15 14:02:32 -04:00
Joey Hess
37b5b1bf0d Fix QuickCheck dependency in cabal file. 2012-01-15 13:53:51 -04:00
Joey Hess
bb70a5e5c4 Merge branch 'master' of ssh://git-annex.branchable.com 2012-01-15 13:53:17 -04:00
Joey Hess
81856c3175 add a configure check for StatFS
This way, the build log will indicate whether StatFS can be relied on.
I've tested all the failing architectures now, and on all of them,
the StatFS code now returns Nothing, rather than Just nonsense.

Also, if annex.diskreserve is set on a platform where StatFS is not
working, git-annex will complain.

Also, the Makefile was missing the sources target used when building with
cabal.
2012-01-15 13:49:32 -04:00
https://openid.stackexchange.com/user/fd55c6e3-966a-4626-865f-5d0f73e1eb88
072a3f2c99 2012-01-15 17:22:49 +00:00
Joey Hess
0eed604446 Add a sanity check for bad StatFS results.
git-annex FTBFS on s390, mips, powerpc, sparc. That StatFS code is failing
on all of them. At least on s390, the failure appears as:

Just (FileSystemStats {fsStatBlockSize = 4096, fsStatBlockCount = 0,
fsStatByteCount = 0, fsStatBytesFree = 0, fsStatBytesAvailable = 0,
fsStatBytesUsed = 0})

While I don't understand why this is happening, or how to fix it,
bandaid over it by checking for obviously bad values and returning Nothing.
That disables disk free space checking, but at least git-annex will work.

Upstream bug: http://code.google.com/p/xmobar/issues/detail?id=70
2012-01-14 17:17:20 -04:00
Joey Hess
a3d97e0c85 tweak 2012-01-14 14:31:16 -04:00
Joey Hess
5e2b4e16ba avoid multiple unnecessary stats of the index file
Up to one per file processed.
2012-01-14 12:07:36 -04:00
Joey Hess
8c87293b48 avoid unnecessary stats when traversing to parent 2012-01-14 11:48:10 -04:00
Joey Hess
92a4af8b20 avoid unnecessary chdir 2012-01-14 11:42:51 -04:00
Joey Hess
1f66af2b53 optimize away 3 stats 2012-01-14 11:28:49 -04:00
Joey Hess
ff5703ce77 tweak 2012-01-13 21:06:00 -04:00
Joey Hess
5481528058 add news item for git-annex 3.20120113 2012-01-13 16:01:31 -04:00
Joey Hess
2dc10840e7 Merge branch 'master' of ssh://git-annex.branchable.com 2012-01-13 15:52:23 -04:00
Joey Hess
b88ecbdc1b Add libghc-testpack-dev to build depends on all arches. 2012-01-13 15:50:56 -04:00
Joey Hess
9ad52ef67c add hackage target 2012-01-13 15:08:28 -04:00
http://peter-simons.myopenid.com/
cff772031d Document how to install git-annex in Nix 2012-01-13 18:53:41 +00:00