Commit graph

272 commits

Author SHA1 Message Date
Joey Hess
46b6d75274 Youtube support! (And 53 other video hosts)
When quvi is installed, git-annex addurl automatically uses it to detect
when an page is a video, and downloads the video file.

web special remote: Also support using quvi, for getting files,
or checking if files exist in the web.

This commit was sponsored by Mark Hepburn. Thanks!
2013-08-22 18:50:43 -04:00
Joey Hess
641b14b124 Debian: Recommend ssh-askpass, which ssh will use when the assistant is run w/o a tty. Closes: #719832 2013-08-16 10:07:44 +02:00
Joey Hess
59f9781a23 Debian: Run the builtin test suite as an autopkgtest. 2013-08-15 15:49:19 +02:00
Joey Hess
10f297d989 typo 2013-07-28 17:04:46 -04:00
Joey Hess
7e66d260ea importfeed: git-annex becomes a podcatcher in 150 LOC 2013-07-28 16:55:42 -04:00
Joey Hess
e95e9e2020 Enable assistant and WebDAV support on powerpc and sparc architectures, which now have the necessary dependencies built. 2013-06-21 01:50:00 -04:00
Joey Hess
9200e9a5f9 update standards-version 2013-06-17 13:59:17 -04:00
Joey Hess
9fa063a00f debian architecture build fun 2013-05-19 21:44:33 -04:00
Joey Hess
345ee4f37c Switch to MonadCatchIO-transformers for better handling of state while catching exceptions.
As seen in this bug report, the lifted exception handling using the StateT
monad throws away state changes when an action throws an exception.
http://git-annex.branchable.com/bugs/git_annex_fork_bombs_on_gpg_file/
  .. Which can result in cached values being redundantly calculated, or other
     possibly worse bugs when the annex state gets out of sync with reality.

This switches from a StateT AnnexState to a ReaderT (MVar AnnexState).
All changes to the state go via the MVar. So when an Annex action is
running inside an exception handler, and it makes some changes, they
immediately go into affect in the MVar. If it then throws an exception
(or even crashes its thread!), the state changes are still in effect.

The MonadCatchIO-transformers change is actually only incidental.
I could have kept on using lifted-base for the exception handling.
However, I'd have needed to write a new instance of MonadBaseControl
for the new monad.. and I didn't write the old instance.. I begged Bas
and he kindly sent it to me. Happily, MonadCatchIO-transformers is
able to derive a MonadCatchIO instance for my monad.

This is a deep level change. It passes the test suite! What could it break?

Well.. The most likely breakage would be to code that runs an Annex action
in an exception handler, and *wants* state changes to be thrown away.
Perhaps the state changes leaves the state inconsistent, or wrong. Since
there are relatively few places in git-annex that catch exceptions in the
Annex monad, and the AnnexState is generally just used to cache calculated
data, this is unlikely to be a problem.

Oh yeah, this change also makes Assistant.Types.ThreadedMonad a bit
redundant. It's now entirely possible to run concurrent Annex actions in
different threads, all sharing access to the same state! The ThreadedMonad
just adds some extra work on top of that, with its own MVar, and avoids
such actions possibly stepping on one-another's toes. I have not gotten
rid of it, but might try that later. Being able to run concurrent Annex
actions would simplify parts of the Assistant code.
2013-05-19 14:16:36 -04:00
Joey Hess
86912bc498 add new dep 2013-05-17 12:10:46 -04:00
Joey Hess
924ed999c5 add unix-compat to deps 2013-05-11 17:21:12 -04:00
Joey Hess
2d51434341 Avoid depending on regex-tdfa on mips, mipsel, and s390, where it fails to build. 2013-05-06 09:44:55 -04:00
Joey Hess
f387a00484 try to fix build on hurd, which lacks haskell-network-info 2013-05-06 09:39:15 -04:00
Joey Hess
25aabf4ffe add TList, built on DList 2013-04-24 16:01:01 -04:00
Joey Hess
7d11511d54 new build dep 2013-04-16 19:00:44 -04:00
Joey Hess
8b27c366b8 add wget and curl to build deps, so configure can find them 2013-04-16 14:50:31 -04:00
Joey Hess
65c7fcc77a Build debian package without using cabal, which writes to HOME. Closes: #704205 2013-03-29 11:37:25 -04:00
Joey Hess
3816c6c82a add back cabal-install (for Makefile) 2013-03-10 20:16:57 -04:00
Joey Hess
a2d94bd627 Switch from using regex-compat to regex-tdfa, as the C regex library is rather buggy. 2013-03-08 15:29:01 -04:00
Joey Hess
843df51f06 use Setup.hs rather than depending on cabal-install
Based on a patch from Peter Simons
2013-02-28 16:08:11 -04:00
Joey Hess
0151f42cdf Stop depending on testpack. 2013-02-27 23:23:41 -04:00
Joey Hess
b63b8e1b5f need build-dep on cabal-install 2013-02-27 16:37:35 -04:00
Joey Hess
f202d997f4 Now uses the Haskell uuid library, rather than needing a uuid program.
Been meaning to do this for some time; Android port was last straw.

Note that newer versions of the uuid library have a Data.UUID.V4 that
generates random UUIDs slightly more cleanly, but Debian has an old version
of the library, so I do it slightly round-about.
2013-02-10 14:52:54 -04:00
Joey Hess
d3d791c7e7 addurl --fast: Use curl, rather than haskell HTTP library, to support https. 2013-01-27 09:30:53 +11:00
Joey Hess
9ba8abffc6 depend on both wget and curl
wget is used due to having better progress output, but curl is used
in some cases where wget is not appropriate.
2013-01-27 08:48:16 +11:00
Joey Hess
20d927d530 Adjust debian package to only build-depend on DAV on architectures where it is available. 2013-01-27 08:07:22 +11:00
Joey Hess
1713ed95f7 use async to track and manage threads 2013-01-26 14:14:32 +11:00
Joey Hess
a75946fc6f Depend on git 1.7.7.6 for --no-edit. Closes: #698399 2013-01-17 21:00:05 -04:00
Joey Hess
5460414486 webdav: Avoid trying to set props, avoiding incompatability with livedrive.com. Needs DAV version 0.3. 2012-12-01 17:12:41 -04:00
Joey Hess
9fe8bb5280 only build-depend on wait and warp on Debian arches that have yesod 2012-11-26 12:59:14 -04:00
Joey Hess
6d775b704b releasing version 3.20121126 2012-11-26 12:35:39 -04:00
Joey Hess
66c0da1fa9 build-depend on a recent mtl
Old versions have an insufficiently generic definition of reader,
which only works on ReaderT and not on newtypes made of it, or something
like that.
2012-11-25 16:24:05 -04:00
Joey Hess
a4b86c63d6 webdav is fully working in non-enctypted mode 2012-11-16 00:09:22 -04:00
Joey Hess
3c039d329c update to dav 0.1, and basic uploading is working! 2012-11-15 13:46:16 -04:00
Joey Hess
0cba0cb2dd skeltal webdav special remote
Doesn't actually store anything yet, but initremote works and tests the
server.
2012-11-14 20:25:31 -04:00
Joey Hess
cf565c0098 remove async build dep
Not needed.. At least for now..
2012-11-14 11:53:59 -04:00
Joey Hess
7389a9bd84 looses xmpp build dep slightly
My upload of -2 seems to have gotten lost, but there is a +b1 binNMU
that rebuilds against the new gnutls.
2012-11-12 14:14:57 -04:00
Joey Hess
b312e54ba7 added a runTimeout function
This adds a dep on haskell's async library, but since that's been
added to the recent haskell platform release, it should not be
much hardship to my poor long-suffering library chasing users.
2012-11-11 13:38:08 -04:00
Joey Hess
dedd2a407e version gnutls d-b to avoid segfault
debian testing still has the buggy version..
2012-11-10 11:49:02 -04:00
Joey Hess
a4c6b30e2c avoid using Blaze directly
New 0.5 changes the api, rather gratuitously, so run away. I can juse use
Hamlet here.
2012-10-31 13:27:56 -04:00
Joey Hess
4757bd3728 added a missing build-dep 2012-10-31 11:50:43 -04:00
Joey Hess
87ebdc8f90 add build-dep on blaze-markup
which blaze-html depends on, so not a significant new dep
2012-10-30 12:59:05 -04:00
Joey Hess
4db379bca3 recommend bind9-host 2012-10-27 20:54:53 -04:00
Joey Hess
fc06ccf355 Merge branch 'xmpp' 2012-10-27 00:55:13 -04:00
Joey Hess
0c0ef99341 gnutls segfault fixed
Adjust build deps to ensure that only a fixed version of the library will
be used.

Also, removed the bound thread stuff, which I now think was (probably)
a red herring.
2012-10-26 23:07:30 -04:00
Joey Hess
c57edb4381 fix syntax 2012-10-26 20:47:44 -04:00
Joey Hess
d1e5f2d596 Merge branch 'master' into xmpp
Conflicts:
	Assistant/Threads/NetWatcher.hs
2012-10-26 19:42:24 -04:00
Joey Hess
8e4620a6c7 NetWatcher: When dbus connection is lost, try to reconnect.
MountWatcher can't do this, because it uses the session dbus,
and won't have access to the new DBUS_SESSION_BUS_ADDRESS if a new session
is started.

Bumped dbus library version, FD leak in it is fixed.
2012-10-26 19:38:27 -04:00
Joey Hess
52a48d1a78 Merge branch 'master' into xmpp 2012-10-25 14:43:55 -04:00
Joey Hess
b846a69451 Re-enable dbus, using a new version of the library that fixes the memory leak.
For now, when dbus goes away, the assistant keeps running but does not fall
back or reconnect. To do so needs more changes to the DBus library; in
particular a connectSessionWith and connectSystemWith to let me specify
my own clientThreadRunner.
2012-10-25 14:32:30 -04:00
Joey Hess
2c788a28a1 added network-protocol-xmpp and -DWITH_XMPP 2012-10-23 20:24:23 -04:00
Joey Hess
33a2af36f2 Depend on and use the Haskell SafeSemaphore library, which provides exception-safe versions of SampleVar and QSemN. Thanks, Ben Gamari for an excellent patch set. 2012-10-05 17:50:17 -04:00
Joey Hess
2b0423e13f Only build-depend on libghc-clientsession-dev on arches that will have the webapp. 2012-10-04 17:08:43 -04:00
Joey Hess
5849c3f24b Avoid building the webapp on Debian architectures that do not yet have template haskell and thus yesod. (Should be available for arm soonish I hope). 2012-09-29 01:28:02 -04:00
Joey Hess
4a6315fb6f demote libnss-mdns to suggests
At recommends it causes avahi-daemon to be pulled in on upgrade, which is
just too annoying to deal with avoiding on servers. MDNS is needed for
robust peering, but probably most desktop systems have it anyway; it's in
task-desktop.
2012-09-28 00:09:38 -04:00
Joey Hess
2edf15b677 remove blaze-markup 2012-09-24 14:25:32 -04:00
Joey Hess
6b36c18f2d update deps 2012-09-11 20:26:44 -04:00
Joey Hess
24bfabe263 add build dep on network-info 2012-09-07 19:44:20 -04:00
Joey Hess
a8e05c8da1 add network-multicast to bulld depends 2012-09-07 14:54:00 -04:00
Joey Hess
3305e019db add yesod-default
another dependency cabal works without here, oddly
2012-07-27 12:09:34 -04:00
Joey Hess
df00c6166c update deps
Note that here I don't need blaze-markup for cabal to succeed, but Jimmy
reports he does. Seems like Text.Blaze.Renderer.String moved from blaze to
blaze-markup in some version.
2012-07-26 13:13:23 -04:00
Joey Hess
6cecc26206 update build deps 2012-07-26 05:13:27 -04:00
Joey Hess
e6ce54de82 build fixes 2012-07-25 22:04:10 -04:00
Joey Hess
32d3cffc4c run yesod, and launch webapp on startup 2012-07-25 21:26:13 -04:00
Joey Hess
f20a40f9d4 MountWatcher thread
Currently only prints mount points when mounts happen.
2012-07-19 13:04:33 -04:00
Joey Hess
41fcb3d852 Version build dependency on STM, and allow building without it, which disables the watch command. 2012-06-26 09:15:47 -04:00
Joey Hess
7db83a1b0f demote lsof
not available on kfreebsd, and only used by watch
2012-06-21 00:28:56 -04:00
Joey Hess
91567ab8f6 make inotify a build flag etc 2012-06-17 17:15:56 -04:00
Joey Hess
7d6329e5e4 check lsof at runtime 2012-06-16 02:49:26 -04:00
Joey Hess
0052cec2b7 add lsof build deps
Check for it in configure; and add a --force option for people without it
who want to live dangerously.
2012-06-15 23:29:39 -04:00
Joey Hess
2de50f733a smart commit thread
The commit thread now has access to a channel containing the times of
all uncommitted changes. This lets it be smart about detecting busy times
when a batch job is running (such as rm -rf, or untarring something, etc),
and avoid committing until it's done. While at the same time, instantly
committing one-off changes that the user is going to expect to see
immediately.

I had to use STM to implement the channel, because of
http://hackage.haskell.org/trac/ghc/ticket/4154
While this adds a dependency, I always wanted to use STM, so this actually
makes me happy. ;)

Also happy that shouldCommit is a pure function, so other commit smartness
strategies can easily be played with. Although the current one seems pretty
good.

There is one bug, for some reason it does double commits, every time.
2012-06-10 16:07:48 -04:00
Joey Hess
59ce18d757 add hinotify dependencies 2012-06-04 13:34:22 -04:00
Joey Hess
52a158a7c6 autocorrection
git-annex (but not git-annex-shell) supports the git help.autocorrect
configuration setting, doing fuzzy matching using the restricted
Damerau-Levenshtein edit distance, just as git does. This adds a build
dependency on the haskell edit-distance library.
2012-04-12 15:37:21 -04:00
Joey Hess
d1e136193b releasing version 3.20120315 2012-03-15 12:23:34 -04:00
Joey Hess
25809ce2e0 finish bloom filters
Add tuning, docs, etc.

Not sure if status is the right place to remote size.. perhaps unused
should report the size and also warn if it sees more keys than the bloom
filter allows?
2012-03-12 16:18:35 -04:00
Joey Hess
b889581945 version dependency on openssh-client
This is only to ensure that it's as new a version as it was built with, so
partial upgrades work.
2012-02-25 19:31:46 -04:00
Joey Hess
12b89a3eb8 configure: Check if ssh connection caching is supported by the installed version of ssh and default annex.sshcaching accordingly. 2012-02-25 19:15:29 -04:00
Joey Hess
4d8afc1713 tweak wording 2012-02-15 19:43:15 -04:00
Joey Hess
91fc975964 note 7.4 needed 2012-02-04 14:51:52 -04:00
Joey Hess
ce5637498f remove Utility.Conditional and use IfElse
This drops the >>! and >>? with the nice low fixity. IfElse does have
undocumented >>=>>! and >>=>>? operators, but I deem that too fishy.
Anyway, using whenM and unlessM is easier; I sometimes mixed the operators
up.
2012-01-24 16:22:07 -04:00
Joey Hess
e3ea5fe938 debhelper v9
kills that ugly python message during build
2012-01-15 14:53:38 -04:00
Joey Hess
b88ecbdc1b Add libghc-testpack-dev to build depends on all arches. 2012-01-13 15:50:56 -04:00
Joey Hess
7e6a54f984 Added quickcheck to build dependencies, and fail if test suite cannot be built. 2012-01-03 14:52:20 -04:00
Joey Hess
8f4fdb3f97 Merge branch 'new-monad-control'
Conflicts:
	debian/changelog
2011-12-30 20:08:01 -04:00
Joey Hess
85f1f3a63a Updated to build with monad-control 0.3. 2011-12-24 23:05:23 -04:00
Joey Hess
11b154e811 prep release 2011-10-11 23:03:19 -04:00
Joey Hess
9c04d1e523 fix git 1.7.7 breakage
* This version of git-annex only works with git 1.7.7 and newer.
  The breakage with old versions is subtle, and affects
  annex.numcopies .gitattributes settings, so be sure to upgrade git
  to 1.7.7. (Debian package now depends on that version.)
* Don't pass absolute paths to git show-attr, as it started following
  symlinks when that's done in 1.7.7. Instead, use relative paths,
  which show-attr only handles 100% correctly in 1.7.7. Closes: #645046

Unfortunatly I can find no way to work with the old and new gits, as
the old had bugs that require absolute paths, while the new doesn't like
them at all. And the behavior of git show-attr in 1.7.7. is the same as
eg, git add of an absolute path to a symlink, so seems entirely
intentional and not likely to change.
2011-10-11 22:53:32 -04:00
Joey Hess
2f4d4d1c45 basic json support
This includes a generic JSONStream library built on top of Text.JSON
(somewhat hackishly).

It would be possible to stream out a single json document describing
all actions, but it's probably better for consumers if they can expect
one json document per line, so I did it that way instead.

Output from external programs used for transferring files is not
currently hidden when outputting json, which probably makes it not very
useful there. This may be dealt with if there is demand for json
output for --get or --move to be parsable.

The version, status, and find subcommands have hand-crafted output and
don't do json. The whereis subcommand needs to be modified to produce
useful json.
2011-09-01 15:22:06 -04:00
Joey Hess
6e750764b7 The wget command will now be used in preference to curl, if available.
Got tired of curl's various ugly progress bars.
2011-08-27 12:31:50 -04:00
Joey Hess
4a023dd1aa Added curl to Debian package dependencies. 2011-08-16 22:22:00 -04:00
Joey Hess
2a108982ad add monad-control to build depends
Will use this to handle exceptions in the Annex monad, yay.
2011-07-07 20:53:57 -04:00
Joey Hess
5c69ac14eb Drop the dependency on the haskell curl bindings, use regular haskell HTTP. 2011-07-04 19:33:11 -04:00
Joey Hess
6ba866ca73 updates for web remote and removing URL backend 2011-07-01 15:39:30 -04:00
Joey Hess
82b88d0676 typo 2011-05-27 20:21:13 -04:00
Joey Hess
8ed27db18f add explict build dep on hslogger
pulled in by missingh, but now used directly by git-annex
2011-05-21 13:03:13 -04:00
Joey Hess
3ab15b9f4f releasing version 0.20110516 2011-05-16 15:01:05 -04:00
Joey Hess
27774bdd56 Revert "Use haskell Crypto library instead of haskell SHA library.a"
This reverts commit 892593c5ef.

Conflicts:

	Crypto.hs
	debian/control
2011-04-26 11:24:23 -04:00
Joey Hess
7d71f8770b releasing version 0.20110425 2011-04-25 16:02:57 -04:00
Joey Hess
8512a4a1a1 Remove testpack from build depends, as it is not available on all architectures.
The test suite will not be run if it cannot be compiled.

It may be possible later to split off the quickcheck using tests into
a separate program and keep most of the tests using just hunit.
2011-04-25 12:43:22 -04:00
Joey Hess
892593c5ef Use haskell Crypto library instead of haskell SHA library.a
Since hS3 needs Crypto anyway, this actually reduces dependencies.
2011-04-21 16:37:14 -04:00
Joey Hess
6668a061a8 typo 2011-04-21 14:53:07 -04:00
Joey Hess
6fcd3e1ef7 fix S3 upload buffering problem
Provide file size to new version of hS3.
2011-04-21 10:33:17 -04:00
Joey Hess
d8329731c6 missing build dep 2011-04-21 09:58:32 -04:00
Joey Hess
43639f69f6 ghc7
* Update Debian build dependencies for ghc 7.
* Debian package is now built with S3 support. Thanks Joachim Breitner for
  making this possible, also thanks Greg Heartsfield for working to improve
  the hS3 library for git-annex.

Also hid a conflicting new symbol from Control.Monad.State
2011-04-21 02:22:40 -04:00
Joey Hess
a91a51fc03 Add missing build dep on dataenc. 2011-04-17 14:41:24 -04:00
Joey Hess
36f048979f releasing version 0.20110417 2011-04-17 12:43:36 -04:00
Joey Hess
11da36e48f build dep update 2011-04-16 23:05:26 -04:00
Joey Hess
1247bfeaa7 gpg recommended 2011-04-16 19:13:05 -04:00
Joey Hess
44c65f40b7 bup is now supported as a special type of remote. 2011-04-08 16:44:43 -04:00
Joey Hess
c1bbe43422 Add build depend on perlmagick so docs are consistently built. Closes: #621410 2011-04-06 21:53:06 -04:00
Joey Hess
175d055d4d Add Suggests on graphviz. Closes: #618039 2011-03-13 14:25:32 -04:00
Joey Hess
f4a26f01ea more tests 2011-01-06 22:22:09 -04:00
Joey Hess
f8851aad66 use Text.Regex.PCRE.Light.Char8 rather than Text.Regexp
Text.Regexp does not think that á matches . -- seems to be a unicode
problem.
2010-12-08 14:48:10 -04:00
Joey Hess
37941184f9 Rsync will now be used to resume interrupted/failed partial file transfers from a remote. 2010-12-02 16:55:21 -04:00
Joey Hess
68183b4721 add tests for uuid and xargs -0 2010-11-18 15:15:40 -04:00
Joey Hess
f03adec793 Add build dep on libghc6-testpack-dev. 2010-11-08 12:36:55 -04:00
Joey Hess
e6048b0389 document using ssh remotes 2010-10-22 15:38:31 -04:00
Joey Hess
d53519fade probably won't use SHA module 2010-10-19 16:30:17 -04:00
Joey Hess
d6911f57b7 update 2010-10-19 16:17:29 -04:00
Joey Hess
e8267f1b9e add doc wiki 2010-10-19 14:37:19 -04:00
Joey Hess
d1a455bdb4 need SHA too 2010-10-16 16:52:20 -04:00
Joey Hess
117e97ea30 debianization 2010-10-16 16:41:24 -04:00