I have seen some other programs do this, and think it's pretty cool. Means
you can test wherever it's deployed, as well as at build time.
My other reason for doing it is less happy. Cabal's handling of test suites
sucks, requiring duplicated info, and even when that's done, it fails to
preprocess hsc files here. Building it in avoids that and avoids having
to explicitly tell cabal to enable test suites, which would then make it
link the test executable every time, which is unnecessarily slow.
This also has the benefit that now "make fast test" does a max speed build
and tests it.
The only thing lost is ./ghci
Speed: make fast used to take 20 seconds here, when rebuilding from
touching Command/Unused.hs. With cabal, it's 29 seconds.
Pity that the library does not provide a function to extract the status
code from the StatusCodeException, so when they had to add a new field, it
breaks every single place that does it.
Various things that don't work on Android are just ifdefed out.
* the webapp (needs template haskell for arm)
* --include and --exclude globbing (needs libpcre, which is not ported;
probably I'll make it use the pure haskell glob library instead)
* annex.diskreserve checking (missing sys/statvfs.h)
* timestamp preservation support (yawn)
* S3
* WebDAV
* XMPP
The resulting 17mb binary has been tested on Android, and it is able to,
at least, print its usage message.
I need to clean up the path to hasktags. Ideally by packaging it in Debian.
Not using ghci's own tags generation because it falls over on pre-compiled
files.
This allows it to use Build.SysConfig to always install the programs
configure detected. Amoung other fixes, this ensures the right uuid
generator and checksum programs are installed.
I also cleaned up the handling of lsof's path; configure now checks for
it in PATH, but falls back to looking for it in sbin directories.
This ensures that the standalone builds will include lsof, and it'll then
be in the runshell PATH, so will work.
This does not deal with manual builds where lsof is not in path, which
will fail at runtime due to the program being missing.
For now, when dbus goes away, the assistant keeps running but does not fall
back or reconnect. To do so needs more changes to the DBus library; in
particular a connectSessionWith and connectSystemWith to let me specify
my own clientThreadRunner.
I had been using -ignore-package monads-tf to deal with this, but
the XMPP library uses monads-tf, so that also ignores it. Instead,
use PackageImports to force use of mtl in my own code.
The standalone build does not bundle its own ssh, so should be built
to support as wide an array of ssh versions as possible, so turn off
connection caching.
Unfortunatly, as implemented this forces a full rebuild when building the
standalone binary, and of course it makes it somewhat slower.
This is not ideal, but neither is probing the ssh version every time it's
run (slow), or once when initializing a repo (fragile).
Old 1.0.1 version is still supported as well. Cabal autodetects
which version is available, but in the Makefile, WITH_OLD_YESOD
has to be configured appropriately.
I have not squashed all the $newline warnings with the new Yesod.
They should go away eventually anyway as Yesod moves past that transition.
This allows setting GIT_ANNEX_TMP_BUILD_DIR to be on a ramdisk,
and all the .o files as well as the git-annex binary are written there.
A useful optimisation for me to avoid SSD write thrashing during
development.
Added a modified System.Cmd.Utils, working around bug #681621
Unfortunatly, the test suite still hangs partway through.
Some of the hangs occur within pOpen3 still. Some of the hangs
do not seem to occur within System.Cmd.Utils at all, but in some other
code.
The `cabal install` is happy as long as the files it needs are
present, but `cabal sdist` will only package up files you tell it to.
So, generate the source tarball ourselves.
The source tarball is generated by make-sdist.sh, which uses cabal
sdist to calculate the package name. Could also generate the name
from the 'Version:' field in git-annex.cabal.
The existing `sed | find | perl` hack in the Makefile was not
including the man pages in the generated git-annex.cabal. I couldn't
figure out why it didn't work; running the `find | perl` part of the
command *did* list the man pages ...
So, I set up a new hack. It produces a cleaner .cabal file and
includes the man pages in the sdist. I changed git-annex.cabal and
its generation as follows:
- git-annex.cabal is now generated by a here document in
git-annex.cabal.template.sh. The here document has inline file list
insertion, whereas before the file lists were inserted with sed.
- The 'Extra-Source-Files:' field now only includes the non-source
files: the man pages, plain text documentation, and license.
- The source dependencies are now listed in 'Other-Modules' sections
in the 'Executable' and 'Test-Suite' sections. The list of
dependencies is generated by `gen-other-modules.sh`.
- The ./debian and ./doc are no longer included in the sdist package.
These were not installed anywhere by `cabal install`. A user that
wants them could clone the git repo.
Running the tests with cabal is not yet working, i.e.
cabal configure --enable-tests && cabal build && cabal test
and
cabal install --enable-tests
fail to find Utility.Touch. However, I did not break this: it doesn't
work for the git-annex package on Hackage either. Next step is to
figure out how to deal with HSC in cabal ... or not bother, because
`make test` works. I'm worried this is a cabal bug.
To test building from sdist, I've been running
cd ../.. ; cabal sdist ; cd dist ; tar xf git-annex-3.20120605.tar.gz && cd git-annex-3.20120605 && rm -fr /tmp/git-annex && cabal install --prefix=/tmp/git-annex && tree -A /tmp/git-annex
in the dist directory. Using `cabal-dev install` is a better test,
but is very slow.
Fixes build breakage when both 'mtl' and 'monads-tf' are present:
$ make git-annex
> ghc -O2 -Wall -ignore-package monads-fd -outputdir tmp -IUtility -DWITH_S3 --make git-annex Utility/libdiskfree.o
>
> Common.hs:6:8:
> Ambiguous module name `Control.Monad.State.Strict':
> it was found in multiple packages: monads-tf-0.1.0.0 mtl-2.1.1
> make: *** [git-annex] Error 1
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
The `cabal install git-annex` doesn't install the man pages, and the
Makefile only installed the man pages as part of a full build/install.
So, I factored out the documentation parts of the Makefile.
* git-annex now behaves as git-annex-shell if symlinked to and run by that
name. The Makefile sets this up, saving some 8 mb of installed size.
* git-union-merge is a demo program, so it is no longer built by default.
This way, the build log will indicate whether StatFS can be relied on.
I've tested all the failing architectures now, and on all of them,
the StatFS code now returns Nothing, rather than Just nonsense.
Also, if annex.diskreserve is set on a platform where StatFS is not
working, git-annex will complain.
Also, the Makefile was missing the sources target used when building with
cabal.
I hate hard-coded 40 kilobyte lone file lists, and just once would like to
see a build system that does not assume it's a good idea to have a file
list, or a hardcoded file list, or a file list that can only be generated
with a crippled form of globs. But not today, thank you cabal.
This is substantially slower than using make, does not build or install
documentation, does not run the test suite, and is not particularly
recommended, but could be useful to some.
This is a new git subcommand, that does a generic union merge operation
between two refs, storing the result in a branch. It operates efficiently
without touching the working tree. It does need to write out a temporary
index file, and may need to write out some other temp files as well.
This could be useful for anything that stores data in a branch,
and needs to merge changes into that branch without actually checking the
branch out. Since conflict handling can't be done without a working copy,
the merge type is always a union merge, which is fine for data stored in
log format (as git-annex does), or in non-conflicting files
(as pristine-tar does).
This probably belongs in git proper, but it will live in git-annex for now.
---
Plan is to move .git-annex/ to a git-annex branch, and use git-union-merge
to handle merging changes when pulling from remotes.
Some preliminary benchmarking using real .git-annex/ data indicates
that it's quite fast, except for the "git add" call, which is as slow
as "git add" tends to be with a big index.
ghc 7 produces these warnings http://hackage.haskell.org/trac/ghc/ticket/4288
The specialization is enabled by -O2, and the default limit of 3 is
there to avoid specialization blowing up binary size. Perhaps that default
is a little low? I needed 4 to avoid a warning on Unused.hs, and 5 to avoid
warnings on test.hs
The test suite will not be run if it cannot be compiled.
It may be possible later to split off the quickcheck using tests into
a separate program and keep most of the tests using just hunit.
So, it would be nicer to just use Cabal and take advantage
of its conditional compilation support. But, Cabal seems to
lack good support for a package with an internal library that is used by
multiple executables. It wants to build everything twice or more.
That's too slow for me.
Anyway, fairly soon, I expect to upgrade hS3 to a requirment, and I
can just revert this.
It compiles. It sorta works. Several subcommands are FIXME marked and
broken, because things that used to accept separate --backend and --key
params need to be changed to accept just a --key that encodes all the key
info, now that there is metadata in keys.
When adding files to the annex, the symlinks pointing at the annexed
content are made to have the same mtime as the original file. While git
does not preserve that information, this allows a tool like metastore to be
used with annexed files.
The option cause it to always build to build/Main.o, no matter what
binary it was building. This caused extra work, and in some cases,
could cause the wrong code to be put into the final binary.
