Commit graph

2030 commits

Author SHA1 Message Date
Joey Hess
5f73bc3a12 Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. 2015-04-19 11:05:32 -04:00
Joey Hess
c0497aa571 switch to using cryptohash for MAC 2015-04-19 10:54:12 -04:00
Joey Hess
ef2202fd94 required: New command, like wanted, but for required content.
Also refactored some code to reduce duplication.
2015-04-18 16:04:35 -04:00
Joey Hess
8d685768d3 fsck --from remote: Avoid downloading a key if it would go over the annex.diskreserve limit. 2015-04-18 14:23:42 -04:00
Joey Hess
8489057e8d fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss. 2015-04-18 14:13:07 -04:00
Joey Hess
0def1f0b53 Fix fsck --from a git remote in a local directory, and from a directory special remote. This was a reversion caused by the relative path changes in 5.20150113.
The directory special remote was not affected in its normal configuration,
since annex-directory is an absolute path normally. But it could fail
when a relative path was used.

The git remote was affected even when an absolute path to it was used in
.git/config, since git-annex now converts all such paths to relative.
2015-04-18 13:36:12 -04:00
Joey Hess
3a078ab357 When a key's size is unknown, still check the annex.diskreserve, and avoid getting content if the disk is too full.
We can't check if there's enough disk space to download the content,
but we *can* check if there's certainly not enough!
2015-04-17 21:29:15 -04:00
Joey Hess
f84ccaa4e8 fix relPathDirToFileAbs on windows with different drive letters
Since we started using this for git repos, when a remote was on another
drive, it resulted in a bogus relative path to it being used by git-annex,
which didn't work.
2015-04-14 14:16:44 -04:00
Joey Hess
efd6a8e3aa bittorrent: Fix handling of magnet links. 2015-04-14 12:57:01 -04:00
Joey Hess
4b6dec6a12 Windows: Renamed start menu file to avoid loop in some versions of Windows where the menu file is treated as a git-annex program. 2015-04-13 14:24:07 -04:00
Joey Hess
2be4834822 info: Added --bytes option. 2015-04-12 14:08:40 -04:00
Joey Hess
b14ec45aa8 info dir: Added information about repositories that contain files in the specified directory.
This is a nearly free feature; it piggybacks on the location log lookups
done for the numcopies stats. So, the only extra overhead is updating
the map of repository sizes.

However, I had to switch to Data.Map.Strict, which needs containers 0.5.
If backporting to wheezy, will probably need to revert this commit.
2015-04-12 12:54:41 -04:00
Joey Hess
2879adc551 fix union merge to call diff-index with -- after the ref
Otherwise, if there's a file in the repo with a name matching the ref,
git could get confused and the merge not work.
2015-04-09 21:13:28 -04:00
Joey Hess
06211738c1 Fix activity log parsing.
I had some cargo culting in there that used the wrong type, so it failed
to parse old logs, and overwrote them with the new log.
2015-04-09 21:02:38 -04:00
Joey Hess
ce0a82f493 contentlocationn: New plumbing command. 2015-04-09 15:34:47 -04:00
Joey Hess
b22aacab44 prep release 2015-04-09 15:06:46 -04:00
Joey Hess
2736c59204 fix webapp reversion caused by relative path changes
webapp: When adding another local repository, and combining it with the
current repository, the new repository's remote path was set to "." rather
than the path to the current repository. This was a reversion caused by the
relative path changes in 5.20150113.
2015-04-09 14:59:08 -04:00
Joey Hess
0f740fd198 This fixes a bug in the assistant introduced by the literal pathspec changes in version 5.20150406.
git-checkignore refuses to work if any pathspec options are set. Urgh.

I audited the rest of git, and no other commands used by git-annex have
such limitations. Indeed, AFAICS, *all* other commands support
--literal-pathspecs. So, worked around this where git-checkignore is
called.
2015-04-09 13:37:06 -04:00
Joey Hess
3695094454 importfeed: Error out when passed a non-url. 2015-04-09 13:09:06 -04:00
Joey Hess
aa31af5594 importfeed: Fix feed download when curl is used.
This bug used to only afflict file:/// urls, but became more exposed
by b8f0b7309f.
2015-04-09 13:02:57 -04:00
Joey Hess
84dc5629c6 info: Display repository mode: bare when in a bare (non-direct mode) repo. 2015-04-09 12:28:26 -04:00
Joey Hess
d5a25bd567 proxy: Made it work when run in a new repository before initial commit. 2015-04-09 12:24:42 -04:00
Joey Hess
4da371af1e add: If annex.largefiles is set and does not match a file that's being added, the file will be checked into git rather than being added to the annex. Previously, git annex add skipped over such files; this new behavior is more useful in direct mode. 2015-04-08 16:14:23 -04:00
Joey Hess
279fa44cd0 fromkey, registerurl: When reading from stdin, allow the filename and url, respectively, to contain whitespace. 2015-04-07 00:58:51 -04:00
Joey Hess
42281f12d6 bring back --quiet filtering of stdout and stderr, with deadlock fixed
I don't quite understand the cause of the deadlock. It only occurred
when git-annex-shell transferinfo was being spawned over ssh to feed
download transfer progress back. And if I removed this line from
feedprogressback, the deadlock didn't occur:
	bytes <- readSV v

The problem was not a leaked FD, as far as I could see. So what was it?
I don't know.

Anyway, this is a nice clean implementation, that avoids the deadlock.
Just fork off the async threads to handle filtering the stdout and stderr,
and let them clean up their handles whenever they decide to exit.

I've verified that the handles do get promptly closed, although a little
later than I would expect. Presumably that "little later" is what
was making waiting on the threads deadlock.

Despite the late exit, the last line of stdout and stderr appears where
I'd want it to, so I guess this is ok..
2015-04-06 20:20:52 -04:00
Joey Hess
c83940393a emergency release 2015-04-06 17:13:20 -04:00
Joey Hess
0a89d55269 Fixes a bug in the last release that caused rsync and possibly other commands to hang at the end of a file transfer.
Stderr reader blocks waiting for all stderr, and so blocks the process ever
exiting.

I tried several ways to get around this, but no success yet. For now,
disable the stderr reader entirely.
2015-04-06 17:12:38 -04:00
Joey Hess
42e46a8701 avoid using --literal-pathspecs with git older than 1.8.1 which added it
Windows is still building with an older git.
2015-04-06 13:46:11 -04:00
Joey Hess
dd96043011 prep release 2015-04-06 12:50:22 -04:00
Joey Hess
9445556c97 rethought distributed fsck; instead add activity.log and expire command
This is much more space efficient!
2015-04-05 12:50:02 -04:00
Joey Hess
1d57f142f1 Merge branch 'concurrentprogress' 2015-04-04 15:01:00 -04:00
Joey Hess
20fb91a7ad WIP on making --quiet silence progress, and infra for concurrent progress bars 2015-04-03 16:48:30 -04:00
Joey Hess
bd110516c0 init: Improve fifo test to detect NFS systems that support fifos but not well enough for sshcaching.
ssh tries to hard link a fifo, and if not, complains:

muxserver_listen: link mux listener .git/annex/ssh/SHARD1@iabak.archiveteam.org.QK8zOCbtNebI7q54 => .git/annex/ssh/SHARD1@iabak.archiveteam.org: Operation not permitted
2015-04-03 14:57:10 -04:00
Joey Hess
7a44dacb74 version: Add --raw 2015-04-02 15:37:39 -04:00
Joey Hess
294991dacb Significantly sped up processing of large numbers of directories passed to a single git-annex command. (try 2)
New approach is to do it the expensive way for the first 100 paths
on the command line, but then assume the user doesn't care about order too
much and fall back to the cheap way that does not preserve order.
2015-04-02 01:44:32 -04:00
Joey Hess
f79502d377 Revert "Significantly sped up processing of large numbers of directories passed to a single git-annex command."
This reverts commit 705112903e.

Whoops, git ls-files does not always output in the input ordering.
That's why all this work is needed. Urk.
2015-04-02 01:24:37 -04:00
Joey Hess
8aa6b5f2a6 Fix truncation of parameters that could occur when using xargs git-annex.
This will only ever result in a few more git-ls-files being run than were run
before. (Only 1 more is really needed, but around 10 more are currently run
for a max length command line.)

So, no need to worry about the extra zombie, or lost laziness due to concat.
2015-04-02 01:20:07 -04:00
Joey Hess
705112903e Significantly sped up processing of large numbers of directories passed to a single git-annex command. 2015-04-02 00:19:49 -04:00
Joey Hess
656fc1c881 fsck: Added --distributed and --expire options, for distributed fsck. 2015-04-01 17:53:16 -04:00
Joey Hess
42bbed7ce5 import: --deduplicate and --cleanduplicates now output the keys corresponding to duplicated files they process. 2015-03-31 15:36:02 -04:00
Joey Hess
6b55974723 addurl --file: When used with a special remote that claims urls and checks their contents, don't override the user's provided filename with filenames that the special remote suggests. Also, don't allow adding the url if the special remote says it contains multiple files. 2015-03-31 15:20:29 -04:00
Joey Hess
f842888de3 test: Fix --list-tests
Seems list order matters, I don't know why, but have reported it on #tasty
2015-03-31 14:47:27 -04:00
Joey Hess
0a3f2757a7 Relax debian package dependencies to git >= 1:1.7.7.6 rather than needing >= 1:2.0.
Git.Merge needs >= 1.7.7.6 if built with such a version; rest of code all
probes git version at runtime.
2015-03-31 14:22:04 -04:00
Joey Hess
7ebd656eff remove changelog entry
This is confusing because I updated the affected tarballs using the version
number of the last release.
2015-03-31 13:30:39 -04:00
Joey Hess
9e25cbde20 importfeed: Avoid downloading a redundant item from a feed whose guid has been downloaded before, even when the url has changed.
To support this, always store itemid in metadata; before this was only done
when annex.genmetadata was set.
2015-03-31 13:30:13 -04:00
Joey Hess
f0195b2a43 Fix GETURLS in external special remote protocol to strip downloader prefix from logged url info before checking for the specified prefix.
This doesn't change what GETURLS returns, but only whether it matches
any prefix that the external special remote asked for.
2015-03-27 18:49:03 -04:00
Joey Hess
6b89692008 Fix make build target. Thanks, Justin Geibel. 2015-03-27 18:00:32 -04:00
Joey Hess
f35d0bf4b2 Prevent git-ls-files from double-expanding wildcards when an unexpanded wildcard is passed to a git-annex command like add or find.
Note that previously, `git annex find *.jpg` would find eg, foo/bar.jpg.
That was never intended or documented behavior, so I'm going to change it.
But this is potentially a behavior change if someone discovered that
behavior and relied on it despite it being accidental. Oh well.. can't make
an omlette w/o breaking some eggs.
2015-03-27 16:45:50 -04:00
Joey Hess
00b07d7155 Better fix for standalone tarball git-annex sync linker shim bug, that works for "git annex sync" as well as "git-annex sync". 2015-03-27 16:06:50 -04:00
Joey Hess
e473d59b92 prep release 2015-03-27 13:11:40 -04:00