Commit graph

247 commits

Author SHA1 Message Date
Joey Hess
84ac8c58db Add annex.httpheaders and annex.httpheader-command config settings
Allow custom headers to be sent with all HTTP requests.

(Requested by the Internet Archive)
2012-04-22 01:13:09 -04:00
Joey Hess
60ab3d84e1 added ifM and nuked 11 lines of code
no behavior changes
2012-03-14 17:43:34 -04:00
Joey Hess
779ec91908 more robustness fixes 2012-02-18 12:08:02 -04:00
Joey Hess
abd50e01fb don't fail with --pathdepth when file already exists 2012-02-18 12:05:13 -04:00
Joey Hess
00340dfe49 don't error out entirely if an url cannot be downloaded 2012-02-18 11:44:21 -04:00
Joey Hess
69a0161c3a fix filename limit when using --pathdepth 2012-02-16 19:37:02 -04:00
Joey Hess
d05550e803 zero still bad 2012-02-16 14:28:54 -04:00
Joey Hess
346c934409 allow pathdepth to drop from the front or take from the end (negative) 2012-02-16 14:26:53 -04:00
Joey Hess
c2245260b1 improve usage 2012-02-16 12:37:30 -04:00
Joey Hess
39c3f56b33 addurl: Add --pathdepth option. 2012-02-16 12:25:19 -04:00
Joey Hess
a86d937b5b avoid too long filename when making up a filename for addurl too 2012-02-16 02:09:09 -04:00
Joey Hess
e7aaa55c53 create parent directories as needed for addurl --file 2012-02-16 00:05:49 -04:00
Joey Hess
cbaebf538a rework git check-attr interface
Now gitattributes are looked up, efficiently, in only the places that
really need them, using the same approach used for cat-file.

The old CheckAttr code seemed very fragile, in the way it streamed files
through git check-attr.
I actually found that cad8824852
was still deadlocking with ghc 7.4, at the end of adding a lot of files.
This should fix that problem, and avoid future ones.

The best part is that this removes withAttrFilesInGit and withNumCopies,
which were complicated Seek methods, as well as simplfying the types
for several other Seek methods that had a Backend tupled in.
2012-02-13 23:52:21 -04:00
Joey Hess
a3ebf16e62 also verify new urls when adding them to existing files 2012-02-10 19:40:54 -04:00
Joey Hess
17fed709c8 addurl --fast: Verifies that the url can be downloaded (only getting its head), and records the size in the key. 2012-02-10 19:23:46 -04:00
Joey Hess
1c0bd81ba6 addurl: Normalize badly encoded urls. 2012-02-09 14:19:58 -04:00
Joey Hess
ac97454659 improve error message 2012-02-08 15:49:42 -04:00
Joey Hess
ef013506cb addurl: Added a --file option
Can be used to specify what file the url is added to. This can be used to
override the default filename that is used when adding an url, which is
based on the url. Or, when the file already exists, the url is recorded as
another location of the file.
2012-02-08 15:35:29 -04:00
Joey Hess
aa0882691b Added remote.name.annex-web-options configuration setting, which can be used to provide parameters to whichever of wget or curl git-annex uses (depends on which is available, but most of their important options suitable for use here are the same). 2012-01-02 14:20:20 -04:00
Joey Hess
52fe8a17f3 remove leftover debug print 2011-12-15 13:12:17 -04:00
Joey Hess
3f5f28b487 factor out a stopUnless
code melt for lunch
2011-12-09 12:23:45 -04:00
Joey Hess
bf460a0a98 reorder repo parameters last
Many functions took the repo as their first parameter. Changing it
consistently to be the last parameter allows doing some useful things with
currying, that reduce boilerplate.

In particular, g <- gitRepo is almost never needed now, instead
use inRepo to run an IO action in the repo, and fromRepo to get
a value from the repo.

This also provides more opportunities to use monadic and applicative
combinators.
2011-11-08 16:27:20 -04:00
Joey Hess
3d2a9f8405 cleanup 2011-10-31 17:22:55 -04:00
Joey Hess
ef5330120c bare cleanup 2011-10-29 19:30:48 -04:00
Joey Hess
f97c783283 clean up check selection code
This new approach allows filtering out checks from the default set that are
not appropriate for a command, rather than having to list every check
that is appropriate. It also reduces some boilerplate.

Haskell does not define Eq for functions, so I had to go a long way around
with each check having a unique id. Meh.
2011-10-29 15:19:05 -04:00
Joey Hess
b955238ec7 Fail if --from or --to is passed to commands that do not support them. 2011-10-27 18:56:54 -04:00
Joey Hess
5b74b130a3 refactored and generalized pre-command sanity checking 2011-10-27 16:31:35 -04:00
Joey Hess
91366c896d clean Annex stuff out of Utility/ 2011-10-16 00:04:26 -04:00
Joey Hess
ec169f84b1 migrate: Copy url logs for keys when migrating. 2011-10-15 16:36:56 -04:00
Joey Hess
1a29b5b52e reorganize log modules
no code changes
2011-10-15 16:21:08 -04:00
Joey Hess
6a6ea06cee rename 2011-10-05 16:02:51 -04:00
Joey Hess
cfe21e85e7 rename 2011-10-04 00:59:08 -04:00
Joey Hess
8ef2095fa0 factor out common imports
no code changes
2011-10-03 23:29:48 -04:00
Joey Hess
9f6b7935dd go go gadget hlint 2011-09-20 23:24:48 -04:00
Joey Hess
5ff04bf2af tweak 2011-09-15 16:59:52 -04:00
Joey Hess
35145202d2 remove command type definitions
These were a mistake, they make the type signatures harder to read and
less flexible. The CommandSeek, CommandStart, CommandPerform, and
CommandCleanup types were a good idea, but composing them with the
parameters expected is going too far.
2011-09-15 16:50:49 -04:00
Joey Hess
5c96411314 fix synopsis 2011-09-15 12:36:27 -04:00
Joey Hess
e4ba0934c2 fix / escape 2011-09-09 00:11:32 -04:00
Joey Hess
03d6209e1c addurl: Always use whole url as destination filename, rather than only its file component.
First, this ensures that git annex addurl, when run repeatedly with the
same url, doesn't create duplicate files, which it did before when it
fell back to the longer filename.

Secondly, the file part of an url is frequently not very descriptive on its
own.

The uri scheme, auth, and port is intentionally left out, as clutter.
2011-09-07 19:04:51 -04:00
Joey Hess
203148363f split groups of related functions out of Utility 2011-08-22 16:14:12 -04:00
Joey Hess
737b5d14c9 moved files around 2011-08-20 16:11:42 -04:00
Joey Hess
4545a0e78c split out generic url stuff into a helper library from Remote.Web 2011-08-16 20:49:44 -04:00
Joey Hess
dede05171b addurl: --fast can be used to avoid immediately downloading the url.
The tricky part about this is that to generate a key, the file must be
present already. Worked around by adding (back) an URL key type, which
is used for addurl --fast.
2011-08-06 14:57:22 -04:00
Joey Hess
00153eed48 unify elipsis handling
And add a simple dots-based progress display, currently only used in v2
upgrade.
2011-07-19 14:07:23 -04:00
Joey Hess
9f1577f746 remove unused backend machinery
The only remaining vestiage of backends is different types of keys. These
are still called "backends", mostly to avoid needing to change user interface
and configuration. But everything to do with storing keys in different
backends was gone; instead different types of remotes are used.

In the refactoring, lots of code was moved out of odd corners like
Backend.File, to closer to where it's used, like Command.Drop and
Command.Fsck. Quite a lot of dead code was removed. Several data structures
became simpler, which may result in better runtime efficiency. There should
be no user-visible changes.
2011-07-05 19:57:46 -04:00
Joey Hess
ace9de37e8 download urls via tmp file, and support resuming 2011-07-01 18:59:40 -04:00
Joey Hess
6bddebdb79 add the addurl command 2011-07-01 17:15:46 -04:00