Commit graph

705 commits

Author SHA1 Message Date
Joey Hess
138d25518d Merge branch 'master' into remotecontrol
Conflicts:
	doc/devblog/day_152__more_ssh_connection_caching.mdwn
2014-04-14 13:38:35 -04:00
Joey Hess
e53a85743e
adjust to not use cpp in modules used by configure 2014-04-14 13:37:12 -04:00
Joey Hess
2ff9ba9f74
add missing Network.URI Ord instance for Debian stable 2014-04-14 13:25:49 -04:00
Joey Hess
15917ec1a8 sync, assistant, remotedaemon: Use ssh connection caching for git pushes and pulls.
For sync, saves 1 ssh connection per remote. For remotedaemon, the same
ssh connection that is already open to run git-annex-shell notifychanges
is reused to pull from the remote.

Only potential problem is that this also enables connection caching
when the assistant syncs with a ssh remote. Including the sync it does
when a network connection has just come up. In that case, cached ssh
connections are likely to be stale, and so using them would hang.
Until I'm sure such problems have been dealt with, this commit needs to
stay on the remotecontrol branch, and not be merged to master.

This commit was sponsored by Alexandre Dupas.
2014-04-12 15:59:34 -04:00
Joey Hess
9fff243ff2
wrote test case; found bugs in date math; fixed and simplified using Data.Time.Calendar 2014-04-12 13:29:35 -04:00
Joey Hess
b815988d16
Improve handling on monthly/yearly scheduling.
Avoid back-to-back runs.
2014-04-12 12:58:32 -04:00
Joey Hess
d955cfe78a Improve handling on monthly/yearly scheduling.
Code was still buggy, it turns out (though the recursion checker caught
it). In the case of (Schedule (Monthly Nothing) AnyTime), where the last
run was on yyyy-12-31, it looped forever.

Also, the handling of (Schedule (Yearly Nothing) AnyTime) was wacky where
the last run was yyyy-12-31. It would suggest a window starting on the 3rd
for the next run (because 31 mod 28 is 3).

I think that originally I was wanted to avoid running on 01-01 if it had
just run on 12-31. But the code didn't accomplish this, and it's not
necessary anyway. This is supposed to calculate the next window meeting the
schedule, and for (Schedule (Monthly Nothing), the window starts at 01-01
and runs through 01-31. If that causes two back-to-back runs, well the next
one will not be until 02-01 at the earliest.

Also, back-to-back runs can be avoided, if desired, by using Divisible 2.
2014-04-11 21:42:43 -04:00
Joey Hess
e26dead493 guard against any remaining infinite recursion bugs; throw error if no candidate cn be found in next hundred years
Note that the exception thrown is not visible in the webapp currently
because it crashes one of Cronner's 2 worker threads, which is never
checked.
2014-04-11 18:08:46 -04:00
Joey Hess
9e16171c7f Last release didn't quite fix the high cpu issue in all cases, this should.
This is supposed to look for a day past the last day it ran, not a month
past.

Seems to work, at least in anarcat's test case.
2014-04-11 17:14:19 -04:00
Joey Hess
e0ca99b480
rename confusingly named "day" 2014-04-11 14:38:23 -04:00
Joey Hess
c678798a5c assistant: Fix high CPU usage triggered when a monthly fsck is scheduled, and the last time the job ran was a day of the month > 12. This caused a runaway loop. Thanks to Anarcat for his assistance, and to Maximiliano Curia for identifying the cause of this bug. 2014-04-11 14:34:09 -04:00
Joey Hess
a6df4698a3 Fix rsync progress parsing in locales that use comma in number display. Closes: #744148 2014-04-10 15:36:51 -04:00
Joey Hess
38862bf65d merge from propellor 2014-04-09 01:15:24 -04:00
Joey Hess
cbcb7f50d8 remotedaemon: avoid extraneous stdout output 2014-04-08 14:02:25 -04:00
Joey Hess
5af30678c7 factored out Utility.SimpleProtocol from the external special remote implementation 2014-04-05 13:29:28 -04:00
Joey Hess
611e060385 factor out library code (also used by propellor) 2014-04-03 21:25:59 -04:00
Joey Hess
b98db5d6f1 clean up warning in windows 2014-04-02 18:08:35 -04:00
Joey Hess
9b707f0116
windows fix try 2 2014-04-02 18:02:32 -04:00
Joey Hess
43ed9c7383
fix windows build 2014-04-02 17:21:36 -04:00
Joey Hess
523a98a952 merge from propellor 2014-04-01 17:53:55 -04:00
Joey Hess
bbd2efede3 refactor 2014-03-31 20:15:16 -04:00
Joey Hess
be76a0e22c avoid importing Common
it caused a loop when reusing this in propellor
2014-03-30 18:43:05 -04:00
Joey Hess
065248f3d2 Added required content configuration.
This includes checking when dropping files that any required content
configuration is satisfied. However, it does not yet include an active
check on the required content; the location log is trusted when checking
the required content expression.
2014-03-29 16:03:33 -04:00
Joey Hess
16387edd00 avoid exception when curl exits nonzero (due to eg, bad domain name) 2014-03-27 13:01:57 -04:00
Joey Hess
e426fac273 add desktop notifications
Motivation: Hook scripts for nautilus or other file managers
need to provide the user with feedback that a file is being downloaded.

This commit was sponsored by THM Schoemaker.
2014-03-22 14:12:19 -04:00
Joey Hess
b51f0e80ed
fix build with QuickCheck 2.7.1
For some reason this was working w/o a cast before, despite POSIXTime etc
being newtypes. It stopped working with the new QuickCheck:

Utility/QuickCheck.hs:31:33:
    No instance for (Integral POSIXTime)
      arising from a use of `arbitrarySizedIntegral'
    Possible fix: add an instance declaration for (Integral POSIXTime)
    In the first argument of `nonNegative', namely
      `arbitrarySizedIntegral'
    In the expression: nonNegative arbitrarySizedIntegral
    In an equation for `arbitrary':
        arbitrary = nonNegative arbitrarySizedIntegral
2014-03-20 11:08:40 -04:00
Joey Hess
1052eeface Windows: Fix some filename encoding bugs.
http://git-annex.branchable.com/bugs/Unicode_file_names_ignored_on_Windows/

Not a complete fix yet.
2014-03-19 15:57:56 -04:00
Joey Hess
b7b9afa9ad Better workaround for problem umasks when eg, setting up ssh keys. 2014-03-14 13:37:58 -04:00
Joey Hess
66b8b9c094 webapp: Use securemem for constant time auth token comparisons.
Debian stable does not have securemem, but neither does it have warp-tls,
so just disable use of securemem when not building with https support.
2014-03-12 21:41:20 -04:00
Joey Hess
06218428c7 quick fix to build with old warp-tls
Debian stable's warp-tls is too old to support the new https feature well,
so only use http with that old version.

Note that the webapp still depends on warp-tls, because the TLSSettings
type is used.
2014-03-12 12:19:48 -04:00
Joey Hess
5a37266080 use Monoid instance for Ordering 2014-03-09 14:08:45 -04:00
Joey Hess
a3fe8270ca annex.startupscan can be set to false to disable the assistant's startup scan. 2014-03-05 17:44:14 -04:00
Joey Hess
b9d6e70019
fix build w/o TDFA 2014-03-04 23:49:46 -04:00
Joey Hess
d41367cc08 remove buggy --listen=host:port support 2014-02-28 22:53:26 -04:00
Joey Hess
3c3744c9a9 use https when .git/annex/privkey.pem and .git/annex/certificate.pem exist (untested)
I have not managed to generate a key that is accepted by the old version of
warp-tls I have here.
2014-02-28 21:32:18 -04:00
Joey Hess
7ac37a7854 Probe for quvi version at run time.
Overhead: git annex addurl runs quvi --version once.
And more bloat to Annex state..
2014-02-28 14:54:02 -04:00
Joey Hess
3f6e4b8c7c fix all remaining -Wall warnings on Windows 2014-02-25 14:48:50 -04:00
Joey Hess
003fc2b7e1
add UrlOptions sum type 2014-02-24 22:00:25 -04:00
Joey Hess
c69d6eb035 Make annex.web-options be used in several places that call curl. 2014-02-24 21:29:37 -04:00
Joey Hess
24f8136504 --metadata field=value can now use globs to match, and matches case insensatively, the same as git annex view field=value does.
Also refactored glob code into its own module.
2014-02-21 18:34:34 -04:00
Joey Hess
67fd06af76 add git annex view command
(And a vpop command, which is still a bit buggy.)

Still need to do vadd and vrm, though this also adds their documentation.

Currently not very happy with the view log data serialization. I had to
lose the TDFA regexps temporarily, so I can have Read/Show instances of
View. I expect the view log format will change in some incompatable way
later, probably adding last known refs for the parent branch to View
or something like that.

Anyway, it basically works, although it's a bit slow looking up the
metadata. The actual git branch construction is about as fast as it can be
using the current git plumbing.

This commit was sponsored by Peter Hogg.
2014-02-18 18:22:20 -04:00
Joey Hess
9633c67842 filter branches (incomplete)
Promosing work toward metadata driven filter branches. A few methods
to construct them are stubbed out; all the data types and pure code
seems good.

This commit was sponsored by Walter Somerville.
2014-02-16 17:39:54 -04:00
Joey Hess
9f883192af windows: Fix start for already running daemon 2014-02-13 17:38:56 -04:00
Joey Hess
c60f0b57d2 windows: Fix daemon pid file locking.
Well, as much as it can be fixed on windows. Not atomic; not entirely
guarded against the wrong process having the pid file locked.
2014-02-13 17:02:28 -04:00
Joey Hess
f11f7520b5 windows: Fix process termination code.
The ctrl-c hack used before didn't actually seem to work.

No haskell libraries expose TerminateProcess. I tried just calling it via
FFI, but got segfaults, probably to do with the wacky process handle not
being managed correctly. Moving it all into one C function worked.

This was hell. The EvilLinker hack was just final icing on the cake.
We all know what the cake was made of.
2014-02-13 15:53:10 -04:00
Joey Hess
1b79d18a40 data types and serialization for metadata
A very haskell commit! Just data types, instances to serialize the metadata
to a nice format, and QuickCheck tests.

This commit was sponsored by Andreas Leha.
2014-02-12 17:57:32 -04:00
Joey Hess
7b19c7d25b cleanup thanks to Utility.PID 2014-02-11 15:39:51 -04:00
Joey Hess
c390e896d1 fix windows build (and make --stop work on windows, incidentially)
The Utility.PID will clean up other code soon.
2014-02-11 15:25:59 -04:00
Joey Hess
b2fae4b78f
remove unimplemented windows stubs 2014-02-11 13:18:59 -04:00
Joey Hess
ae2bc90a1d eliminating absNormPath
git-annex has been using MissingH's `abdNormPath` forever, but that's
unmaintained and possibly buggy, and doesn't work on Windows. I've been
wanting to get rid of it for some time, and finally did today, writing a
`simplifyPath` that does the things git-annex needs and will work with all
the Windows filename craziness, and takes advantage of the more modern
System.FilePath to be quite a simple peice of code. A QuickCheck test found
no important divergences from absNormPath. A good first step to making
git-annex not depend on MissingH at all.

And it fixed some weird behaviors on Windows like
`git annex add ..\subdir\file` not working.

Note that absNormPathUnix has been left alone for now.
2014-02-07 17:15:18 -04:00