Commit graph

876 commits

Author SHA1 Message Date
Joey Hess
fa38fe0cf6 qualify imports to avoid conflict with regex-tdfa-rc 2015-05-30 02:03:09 -04:00
Joey Hess
8990d4cc68 fsck: When checksumming a file fails due to a hardware fault, the file is now moved to the bad directory, and the fsck proceeds. Before, the fsck immediately failed. 2015-05-27 16:40:03 -04:00
Joey Hess
ff629a1ec0 implment catchHardwareFault 2015-05-27 16:36:54 -04:00
Joey Hess
7cb16b9efb improve error message when checksum command exits nonzero 2015-05-27 15:43:50 -04:00
Joey Hess
ccf307307d merge haddock documentation from propellor 2015-05-27 12:39:04 -04:00
Joey Hess
6c3cea7699 need more polymorphism 2015-05-22 13:50:37 -04:00
Joey Hess
696ed9b3f9 fix windows build 2015-05-22 13:44:24 -04:00
Joey Hess
9de5cd2966 fix crash in stale transfer lockfile cleanup code
Need to differentiate between the lockfile not being locked, and it not
existing.
2015-05-19 23:35:24 -04:00
Joey Hess
846e7db245 unused imports 2015-05-19 15:05:30 -04:00
Joey Hess
b47c9fd587 honor core.sharedRepository settings in lockContent
The content file may not be owned by the user running git-annex, in which
case, setting the owner write bit was not enough to let lockContent
act on the file. However, with some core.sharedRepository configs, the file
should be writable by the user's group. So, the thing to do is to call
thawContent on it.
2015-05-19 14:53:19 -04:00
Joey Hess
1312e721ed convert lockContent to use new LockPools
Also cleaned up the code, avoiding creating a lock file if we're going to
open it for create later anyway.

And, if there's an exception while preparing to lock the file, but not at
the point of actually taking the lock, throw an exception, instead of
silently not locking and pretending to succeed.

And, on Windows, always use lock file, even if the repo somehow got into
indirect mode (maybe with cygwin git..)
2015-05-19 14:12:23 -04:00
Joey Hess
ecb0d5c087 use lock pools throughout git-annex
The one exception is in Utility.Daemon. As long as a process only
daemonizes once, which seems reasonable, and as long as it avoids calling
checkDaemon once it's already running as a daemon, the fcntl locking
gotchas won't be a problem there.

Annex.LockFile has it's own separate lock pool layer, which has been
renamed to LockCache. This is a persistent cache of locks that persist
until closed.

This is not quite done; lockContent stil needs to be converted.
2015-05-19 14:09:52 -04:00
Joey Hess
6915b71c57 lock pools to work around non-concurrency/composition safety of POSIX fcntl 2015-05-18 15:57:17 -04:00
Joey Hess
e9172263e5 comment typos 2015-05-17 14:22:14 -04:00
Joey Hess
8c2dd7d8ee Fix an unlikely race that could result in two transfers of the same key running at once.
As discussed in bug report.
2015-05-12 19:39:28 -04:00
Joey Hess
03667a162a couple of AMP warnings I missed before 2015-05-10 16:51:03 -04:00
Joey Hess
5c7cdbae46 more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
Joey Hess
ed4fe02896 disable horrible tab warning, needed in every file that Setup.hs pulls in
This is certianly a cabal bug for not passing the build options in the
cabal file when building Setup.hs.

And, why oh why did ghc enable this warning by default? So unhappy with
this choice.
2015-05-10 16:31:50 -04:00
Joey Hess
ec267aa1ea rejigger imports for clean build with ghc 7.10's AMP changes
The explict import Prelude after import Control.Applicative is a trick
to avoid a warning.
2015-05-10 16:20:30 -04:00
Joey Hess
6466dbc950 FlexibleContexts needed by ghc 7.10 2015-05-10 15:37:55 -04:00
Joey Hess
fcb8394e5f switch to Control.Monad.ST to avoid build warning in ghc 7.10 2015-05-10 14:42:16 -04:00
Joey Hess
f395ca7a41 Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. 2015-05-08 13:39:00 -04:00
Joey Hess
d9bfc78444 avoid using relative path from temp dir to dest file
That failed on OSX. The temp dir was
/var/folders/fb/pnwjj52n7fg0r9mnvpsfll180000gr/T/downloadurl
and the relative path
../../../../../../Volumes/Visitors/joeyh/git-annex/r/.git/...
Didn't work. I have no clue why, how did OSX manage to break this?

But, the relative path is longer most of the time anyway, so let's
just use the absolute path.
2015-05-07 18:47:24 -04:00
Magnus Therning
8ab56a5cf2 Replace dataenc with sandi.
The library dataenc is unmaintained and has been superseded by sandi.

Signed-off-by: Magnus Therning <magnus@therning.org>
2015-05-07 18:07:05 -04:00
Joey Hess
cf786f42a4 Support checking ftp urls for file presence. 2015-05-05 14:05:02 -04:00
Joey Hess
0b18228516 Work around wget bug #784348 which could cause it to clobber git-annex symlinks when downloading from ftp. 2015-05-05 13:53:06 -04:00
Joey Hess
3d8cf16eec add more generic interface 2015-05-05 13:51:31 -04:00
Joey Hess
38403092ae revert reversion merged from propellor 2015-04-29 14:15:08 -04:00
Joey Hess
d8ffa498ef avoid dupicate bracket import errror
Ambiguous occurrence `bracket'
    It could refer to either `Control.Exception.bracket',
                             imported from `Control.Exception' at Utility/FileMode.hs:14:27-33
                             (and originally defined in `Control.Exception.Base')
                          or `Utility.Exception.bracket',
                             imported from `Utility.Exception' at Utility/FileMode.hs:22:1-24
                             (and originally defined in `Control.Monad.Catch')
2015-04-29 13:43:11 -04:00
Joey Hess
d40dea58e3 merge generalization of writeFileProtected from propellor 2015-04-28 14:58:29 -04:00
Joey Hess
eb8ef44133 Dropped support for older versions of yesod and warp than the ones in Debian Jessie.
466 lines of compat cruft deleted!
2015-04-22 16:19:11 -04:00
Joey Hess
5f73bc3a12 Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. 2015-04-19 11:05:32 -04:00
Joey Hess
be3895be45 refactor 2015-04-19 10:57:14 -04:00
Joey Hess
addc82dab7 removed all uses of undefined from code base
It's a code smell, can lead to hard to diagnose error messages.
2015-04-19 00:38:29 -04:00
Joey Hess
8489057e8d fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss. 2015-04-18 14:13:07 -04:00
Joey Hess
cd31b69ff6 don't test with null paths 2015-04-14 15:15:29 -04:00
Joey Hess
f84ccaa4e8 fix relPathDirToFileAbs on windows with different drive letters
Since we started using this for git repos, when a remote was on another
drive, it resulted in a bogus relative path to it being used by git-annex,
which didn't work.
2015-04-14 14:16:44 -04:00
Joey Hess
2be4834822 info: Added --bytes option. 2015-04-12 14:08:40 -04:00
Joey Hess
c65e71e6a5 cleanup 2015-04-09 12:57:30 -04:00
Joey Hess
42281f12d6 bring back --quiet filtering of stdout and stderr, with deadlock fixed
I don't quite understand the cause of the deadlock. It only occurred
when git-annex-shell transferinfo was being spawned over ssh to feed
download transfer progress back. And if I removed this line from
feedprogressback, the deadlock didn't occur:
	bytes <- readSV v

The problem was not a leaked FD, as far as I could see. So what was it?
I don't know.

Anyway, this is a nice clean implementation, that avoids the deadlock.
Just fork off the async threads to handle filtering the stdout and stderr,
and let them clean up their handles whenever they decide to exit.

I've verified that the handles do get promptly closed, although a little
later than I would expect. Presumably that "little later" is what
was making waiting on the threads deadlock.

Despite the late exit, the last line of stdout and stderr appears where
I'd want it to, so I guess this is ok..
2015-04-06 20:20:52 -04:00
Joey Hess
0a89d55269 Fixes a bug in the last release that caused rsync and possibly other commands to hang at the end of a file transfer.
Stderr reader blocks waiting for all stderr, and so blocks the process ever
exiting.

I tried several ways to get around this, but no success yet. For now,
disable the stderr reader entirely.
2015-04-06 17:12:38 -04:00
Joey Hess
30aa902174 relay external special remote stderr through progress suppression machinery (eep!)
It sounds worse than it is. ;)

Some external special remotes may run commands that display progress on
stderr. If git-annex is run with --quiet, this should filter out such
displays while letting the errors through.
2015-04-04 14:54:03 -04:00
Joey Hess
2343f99c85 well along the way to fully quiet --quiet
Came up with a generic way to filter out progress messages while keeping
errors, for commands that use stderr for both.

--json mode will disable command outputs too.
2015-04-04 14:34:03 -04:00
Joey Hess
b2ad3403c6 make downloadQuiet quiet again
This was broken in commit c64ede23cd
2015-04-03 20:38:20 -04:00
Joey Hess
45e97191c3 fix stderr handling 2015-04-03 19:48:56 -04:00
Joey Hess
20fb91a7ad WIP on making --quiet silence progress, and infra for concurrent progress bars 2015-04-03 16:48:30 -04:00
Joey Hess
1c91024978 rename bothHandles -> ioHandles 2015-04-03 15:35:18 -04:00
Joey Hess
294991dacb Significantly sped up processing of large numbers of directories passed to a single git-annex command. (try 2)
New approach is to do it the expensive way for the first 100 paths
on the command line, but then assume the user doesn't care about order too
much and fall back to the cheap way that does not preserve order.
2015-04-02 01:44:32 -04:00
Joey Hess
f79502d377 Revert "Significantly sped up processing of large numbers of directories passed to a single git-annex command."
This reverts commit 705112903e.

Whoops, git ls-files does not always output in the input ordering.
That's why all this work is needed. Urk.
2015-04-02 01:24:37 -04:00
Joey Hess
bf563d5df0 comment 2015-04-02 00:34:06 -04:00