mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
24599 commits 13 branches 445 tags 110 MiB
Commit graph

2138 commits

Author SHA1 Message Date
Joey Hess
7ebf234616 Stale transfer lock and info files will be cleaned up automatically when get/unused/info commands are run. 
Deleting lock files is tricky, tricky stuff. I think I got it right!
 2015-05-12 20:11:23 -04:00
Joey Hess
8c2dd7d8ee Fix an unlikely race that could result in two transfers of the same key running at once. 
As discussed in bug report.
 2015-05-12 19:39:28 -04:00
Joey Hess
61ccf95004 Avoid accumulating transfer failure log files unless the assistant is being used. 
Only the assistant uses these, and only the assistant cleans them up, so
make only git annex transferkeys write them,

There is one behavior change from this. If glacier is being used, and a
manual git annex get --from glacier fails because the file isn't available
yet, the assistant will no longer later see that failed transfer file and
retry the get. Hope no-one depended on that old behavior.
 2015-05-12 15:53:38 -04:00
Joey Hess
a812d598ef Take space that will be used by running downloads into account when checking annex.diskreserve. 2015-05-12 15:20:22 -04:00
Joey Hess
505d540360 allow building without ascii-progress, since it is not ready yet 
No progress bars with -J unless built with ascii-progress.
 2015-05-12 13:54:16 -04:00
Joey Hess
e27b97d364 Merge branch 'master' into concurrentprogress 
Conflicts:
	Command/Fsck.hs
	Messages.hs
	Remote/Directory.hs
	Remote/Git.hs
	Remote/Helper/Special.hs
	Types/Remote.hs
	debian/changelog
	git-annex.cabal
 2015-05-12 13:23:22 -04:00
Joey Hess
96d24a8161 note about git annex drop behavior change in bare repo 2015-05-12 13:02:22 -04:00
Joey Hess
4b2e3210ea drop: Now supports --all, --unused, and --key. 2015-05-12 13:00:06 -04:00
Joey Hess
db5d831d07 import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss. 2015-05-11 12:57:47 -04:00
Joey Hess
6ebf615995 cutting a point release, will be a cabal-only update 2015-05-11 12:09:02 -04:00
Yaroslav Halchenko
cb23224ba3 ENH: make debianstandalone{,-dsc} rules 
Moved rules out of debian/rules and avoided need for patching it for
standalone builds
 2015-05-11 11:50:59 -04:00
Joey Hess
26ac0753c1 more FlexibleContexts 2015-05-10 15:54:58 -04:00
Yaroslav Halchenko
e861dd1dd5 ENH: leave only git for depends of standalone -- the rest is provided 
Otherwise, versioned depends forbid installation on elderly squeeze
 2015-05-10 14:36:50 -04:00
Joey Hess
6a1814abf5 seems that webapp ssh password ssh-agent hack is broken on windows after recent ssh changes 2015-05-08 15:07:23 -04:00
Joey Hess
f1a81cfce2 prep release 2015-05-08 13:42:45 -04:00
Joey Hess
f395ca7a41 Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. 2015-05-08 13:39:00 -04:00
Joey Hess
54bee7dad4 Revert "Android: Updated bundled ssh from 6.1p1 to 6.4p1." 
This reverts commit 27836a2af9.

Build didn't actually seem to succeed, for both versions of android, so
I'll need to revisit this later.
 2015-05-08 13:05:33 -04:00
Joey Hess
0137656c39 typo 2015-05-07 19:41:05 -04:00
Joey Hess
ee94a79690 Debian's ghc now supports TH on arm! Adjust build dependencies to build the webapp on arm, and enable DAV support on arm. \o/ 
Also, adjust some other arch specific build dependencies that are now
available on more architectures in Devian unstable.
 2015-05-07 19:09:44 -04:00
Joey Hess
4ba4155067 remove workaround for now fixed bug #763078 2015-05-07 18:54:55 -04:00
Joey Hess
e09816b0e6 reorder 2015-05-07 18:12:08 -04:00
Joey Hess
4aba1c74bd remaining dataenc to sandi conversions 
I've tested all the dataenc to sandi conversions except Assistant.XMPP,
and all have unchanged behavior, including behavior on large unicode code
points.
 2015-05-07 18:07:13 -04:00
Joey Hess
18fc05ff04 Windows: Roll back to an older version of rsync from cygwin. The newer version has some dependency on a newer ssh from cygwin. 2015-05-07 15:56:40 -04:00
Joey Hess
ce285ea6fa Windows: Remove cygwin ssh, the newer version of which has stopped honoring the setting of HOME. Instead, copy msysgit's ssh into PATH. 2015-05-07 15:53:03 -04:00
Joey Hess
469242ac4d fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes: #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only) 2015-05-06 14:45:20 -04:00
Joey Hess
eeb0359a2e contentlocation, examinekey, lookupkey: Added --batch mode option. 2015-05-06 13:44:53 -04:00
Joey Hess
cf786f42a4 Support checking ftp urls for file presence. 2015-05-05 14:05:02 -04:00
Joey Hess
0b18228516 Work around wget bug #784348 which could cause it to clobber git-annex symlinks when downloading from ftp. 2015-05-05 13:53:06 -04:00
Joey Hess
0e6d3a9041 add missing env var setting for standalone build 2015-05-05 13:50:37 -04:00
Yaroslav Halchenko
d15f2a1273 BF: Adjusted debian/patches/standalone-build for recent stylistic changes 2015-05-03 12:02:30 -04:00
Joey Hess
27836a2af9 Android: Updated bundled ssh from 6.1p1 to 6.4p1. 
Tested it builds, but have not run it yet.
 2015-05-01 15:33:29 -04:00
Joey Hess
437d9db41d assistant: Added --autostop to complement --autostart. 2015-05-01 13:53:45 -04:00
Joey Hess
efb37e7c78 Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end. 2015-04-30 15:28:17 -04:00
Joey Hess
ac6b492711 import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist. 2015-04-30 14:04:36 -04:00
Joey Hess
995643665a Improve integration with KDE's file manager to work with dolphin version 14.12.3 while still being compatable with 4.14.2. Thanks, silvio. 2015-04-29 15:53:16 -04:00
Joey Hess
d8ad1d5503 import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree. 2015-04-29 13:56:41 -04:00
Joey Hess
2e54251c18 import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.) 2015-04-29 13:46:12 -04:00
Joey Hess
986e245ac1 changelog 2015-04-27 18:10:56 -04:00
Joey Hess
ee78958798 S3: Fix incompatability with bucket names used by hS3; the aws library cannot handle upper-case bucket names. git-annex now converts them to lower case automatically. 
For example, it failed to get files from a bucket named S3.

Also fixes `git annex initremote UPPERCASE type=S3`, which failed with the
new aws library, with a signing error message.
 2015-04-27 18:00:58 -04:00
Joey Hess
cfbeb1e7b7 Fix bogus failure of fsck --fast. 2015-04-27 17:40:21 -04:00
Joey Hess
22a4e92df7 S3: git annex enableremote will not create a bucket name, which failed since the bucket already exists. 2015-04-23 14:16:53 -04:00
Joey Hess
b3eccec68c S3: git annex info will show additional information about a S3 remote (endpoint, port, storage class) 2015-04-23 14:12:25 -04:00
Joey Hess
8f010bb55b also drop old dbus compat 2015-04-22 16:23:41 -04:00
Joey Hess
eb8ef44133 Dropped support for older versions of yesod and warp than the ones in Debian Jessie. 
466 lines of compat cruft deleted!
 2015-04-22 16:19:11 -04:00
Joey Hess
a15e1158c6 use BuildVersion in debian, which fixes windows build 2015-04-21 16:42:54 -04:00
Joey Hess
3b3aaf0d56 S3: Enable debug logging when annex.debug or --debug is set. 
To debug a bug report, but generally useful.
 2015-04-21 15:55:42 -04:00
Joey Hess
e651b29e90 reuse Build.Version to generate version 2015-04-20 15:56:28 -04:00
Joey Hess
b0ebb232ec style changes 2015-04-20 14:55:30 -04:00
Joey Hess
bf0e860617 changelog for https://github.com/joeyh/git-annex/pull/41 2015-04-20 14:53:14 -04:00
Joey Hess
92833a7e99 update to generate version in same format git-annex uses 2015-04-20 14:51:34 -04:00
Yaroslav Halchenko
bfef0ba69f NF: provide debian/rules build-standalone 2015-04-20 14:46:34 -04:00
Yaroslav Halchenko
38e03d5b3a NF: debian/patches/*standalone-build for providing a standalone build of git-annex 2015-04-20 14:46:33 -04:00
Joey Hess
b7be1680a7 prep release 2015-04-20 14:44:15 -04:00
Joey Hess
5948c148fb Make repo init more robust. 
The setDifferences that got added to initialize turns out to make a git
commit, and before ensureCommit has been used. Thus, repo init can fail
when the system has a broken hostname etc.

Move the ensureCommit to the very first thing to avoid this kind of breakage.
 2015-04-20 14:01:41 -04:00
Joey Hess
fdf0d708d2 reorder 2015-04-20 13:35:09 -04:00
Joey Hess
2665be44d5 update some windows libraries for newer version of cygwin 2015-04-20 11:08:32 -04:00
Joey Hess
5f73bc3a12 Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. 2015-04-19 11:05:32 -04:00
Joey Hess
c0497aa571 switch to using cryptohash for MAC 2015-04-19 10:54:12 -04:00
Joey Hess
ef2202fd94 required: New command, like wanted, but for required content. 
Also refactored some code to reduce duplication.
 2015-04-18 16:04:35 -04:00
Joey Hess
8d685768d3 fsck --from remote: Avoid downloading a key if it would go over the annex.diskreserve limit. 2015-04-18 14:23:42 -04:00
Joey Hess
8489057e8d fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss. 2015-04-18 14:13:07 -04:00
Joey Hess
0def1f0b53 Fix fsck --from a git remote in a local directory, and from a directory special remote. This was a reversion caused by the relative path changes in 5.20150113. 
The directory special remote was not affected in its normal configuration,
since annex-directory is an absolute path normally. But it could fail
when a relative path was used.

The git remote was affected even when an absolute path to it was used in
.git/config, since git-annex now converts all such paths to relative.
 2015-04-18 13:36:12 -04:00
Joey Hess
3a078ab357 When a key's size is unknown, still check the annex.diskreserve, and avoid getting content if the disk is too full. 
We can't check if there's enough disk space to download the content,
but we *can* check if there's certainly not enough!
 2015-04-17 21:29:15 -04:00
Joey Hess
86a2f9dc4d Merge branch 'master' into concurrentprogress 
Conflicts:
	debian/changelog
 2015-04-14 15:35:15 -04:00
Joey Hess
f84ccaa4e8 fix relPathDirToFileAbs on windows with different drive letters 
Since we started using this for git repos, when a remote was on another
drive, it resulted in a bogus relative path to it being used by git-annex,
which didn't work.
 2015-04-14 14:16:44 -04:00
Joey Hess
efd6a8e3aa bittorrent: Fix handling of magnet links. 2015-04-14 12:57:01 -04:00
Joey Hess
4b6dec6a12 Windows: Renamed start menu file to avoid loop in some versions of Windows where the menu file is treated as a git-annex program. 2015-04-13 14:24:07 -04:00
Joey Hess
2be4834822 info: Added --bytes option. 2015-04-12 14:08:40 -04:00
Joey Hess
b14ec45aa8 info dir: Added information about repositories that contain files in the specified directory. 
This is a nearly free feature; it piggybacks on the location log lookups
done for the numcopies stats. So, the only extra overhead is updating
the map of repository sizes.

However, I had to switch to Data.Map.Strict, which needs containers 0.5.
If backporting to wheezy, will probably need to revert this commit.
 2015-04-12 12:54:41 -04:00
Joey Hess
8077ccbd54 get, move, copy, mirror: Concurrent downloads and uploads are now supported! 
This works, and seems fairly robust. Clean get of 20 files at -J3. At -J10,
there are some messages about ssh multiplexing, probably due to a race
spinning up the ssh connection cacher. But, it manages to get all the files
ok regardless.

The progress bars are a scrambled mess though, due to bugs in
ascii-progress, which I've already filed. Particularly this one:
https://github.com/yamadapc/haskell-ascii-progress/issues/8
 2015-04-10 17:08:07 -04:00
Joey Hess
2879adc551 fix union merge to call diff-index with -- after the ref 
Otherwise, if there's a file in the repo with a name matching the ref,
git could get confused and the merge not work.
 2015-04-09 21:13:28 -04:00
Joey Hess
06211738c1 Fix activity log parsing. 
I had some cargo culting in there that used the wrong type, so it failed
to parse old logs, and overwrote them with the new log.
 2015-04-09 21:02:38 -04:00
Joey Hess
ce0a82f493 contentlocationn: New plumbing command. 2015-04-09 15:34:47 -04:00
Joey Hess
b22aacab44 prep release 2015-04-09 15:06:46 -04:00
Joey Hess
2736c59204 fix webapp reversion caused by relative path changes 
webapp: When adding another local repository, and combining it with the
current repository, the new repository's remote path was set to "." rather
than the path to the current repository. This was a reversion caused by the
relative path changes in 5.20150113.
 2015-04-09 14:59:08 -04:00
Joey Hess
0f740fd198 This fixes a bug in the assistant introduced by the literal pathspec changes in version 5.20150406. 
git-checkignore refuses to work if any pathspec options are set. Urgh.

I audited the rest of git, and no other commands used by git-annex have
such limitations. Indeed, AFAICS, *all* other commands support
--literal-pathspecs. So, worked around this where git-checkignore is
called.
 2015-04-09 13:37:06 -04:00
Joey Hess
3695094454 importfeed: Error out when passed a non-url. 2015-04-09 13:09:06 -04:00
Joey Hess
aa31af5594 importfeed: Fix feed download when curl is used. 
This bug used to only afflict file:/// urls, but became more exposed
by b8f0b7309f.
 2015-04-09 13:02:57 -04:00
Joey Hess
84dc5629c6 info: Display repository mode: bare when in a bare (non-direct mode) repo. 2015-04-09 12:28:26 -04:00
Joey Hess
d5a25bd567 proxy: Made it work when run in a new repository before initial commit. 2015-04-09 12:24:42 -04:00
Joey Hess
4da371af1e add: If annex.largefiles is set and does not match a file that's being added, the file will be checked into git rather than being added to the annex. Previously, git annex add skipped over such files; this new behavior is more useful in direct mode. 2015-04-08 16:14:23 -04:00
Joey Hess
279fa44cd0 fromkey, registerurl: When reading from stdin, allow the filename and url, respectively, to contain whitespace. 2015-04-07 00:58:51 -04:00
Joey Hess
42281f12d6 bring back --quiet filtering of stdout and stderr, with deadlock fixed 
I don't quite understand the cause of the deadlock. It only occurred
when git-annex-shell transferinfo was being spawned over ssh to feed
download transfer progress back. And if I removed this line from
feedprogressback, the deadlock didn't occur:
	bytes <- readSV v

The problem was not a leaked FD, as far as I could see. So what was it?
I don't know.

Anyway, this is a nice clean implementation, that avoids the deadlock.
Just fork off the async threads to handle filtering the stdout and stderr,
and let them clean up their handles whenever they decide to exit.

I've verified that the handles do get promptly closed, although a little
later than I would expect. Presumably that "little later" is what
was making waiting on the threads deadlock.

Despite the late exit, the last line of stdout and stderr appears where
I'd want it to, so I guess this is ok..
 2015-04-06 20:20:52 -04:00
Joey Hess
c83940393a emergency release 2015-04-06 17:13:20 -04:00
Joey Hess
0a89d55269 Fixes a bug in the last release that caused rsync and possibly other commands to hang at the end of a file transfer. 
Stderr reader blocks waiting for all stderr, and so blocks the process ever
exiting.

I tried several ways to get around this, but no success yet. For now,
disable the stderr reader entirely.
 2015-04-06 17:12:38 -04:00
Joey Hess
42e46a8701 avoid using --literal-pathspecs with git older than 1.8.1 which added it 
Windows is still building with an older git.
 2015-04-06 13:46:11 -04:00
Joey Hess
dd96043011 prep release 2015-04-06 12:50:22 -04:00
Joey Hess
9445556c97 rethought distributed fsck; instead add activity.log and expire command 
This is much more space efficient!
 2015-04-05 12:50:02 -04:00
Joey Hess
1d57f142f1 Merge branch 'concurrentprogress' 2015-04-04 15:01:00 -04:00
Joey Hess
20fb91a7ad WIP on making --quiet silence progress, and infra for concurrent progress bars 2015-04-03 16:48:30 -04:00
Joey Hess
bd110516c0 init: Improve fifo test to detect NFS systems that support fifos but not well enough for sshcaching. 
ssh tries to hard link a fifo, and if not, complains:

muxserver_listen: link mux listener .git/annex/ssh/SHARD1@iabak.archiveteam.org.QK8zOCbtNebI7q54 => .git/annex/ssh/SHARD1@iabak.archiveteam.org: Operation not permitted
 2015-04-03 14:57:10 -04:00
Joey Hess
7a44dacb74 version: Add --raw 2015-04-02 15:37:39 -04:00
Joey Hess
294991dacb Significantly sped up processing of large numbers of directories passed to a single git-annex command. (try 2) 
New approach is to do it the expensive way for the first 100 paths
on the command line, but then assume the user doesn't care about order too
much and fall back to the cheap way that does not preserve order.
 2015-04-02 01:44:32 -04:00
Joey Hess
f79502d377 Revert "Significantly sped up processing of large numbers of directories passed to a single git-annex command." 
This reverts commit 705112903e.

Whoops, git ls-files does not always output in the input ordering.
That's why all this work is needed. Urk.
 2015-04-02 01:24:37 -04:00
Joey Hess
8aa6b5f2a6 Fix truncation of parameters that could occur when using xargs git-annex. 
This will only ever result in a few more git-ls-files being run than were run
before. (Only 1 more is really needed, but around 10 more are currently run
for a max length command line.)

So, no need to worry about the extra zombie, or lost laziness due to concat.
 2015-04-02 01:20:07 -04:00
Joey Hess
705112903e Significantly sped up processing of large numbers of directories passed to a single git-annex command. 2015-04-02 00:19:49 -04:00
Joey Hess
656fc1c881 fsck: Added --distributed and --expire options, for distributed fsck. 2015-04-01 17:53:16 -04:00
Joey Hess
42bbed7ce5 import: --deduplicate and --cleanduplicates now output the keys corresponding to duplicated files they process. 2015-03-31 15:36:02 -04:00
Joey Hess
6b55974723 addurl --file: When used with a special remote that claims urls and checks their contents, don't override the user's provided filename with filenames that the special remote suggests. Also, don't allow adding the url if the special remote says it contains multiple files. 2015-03-31 15:20:29 -04:00
Joey Hess
f842888de3 test: Fix --list-tests 
Seems list order matters, I don't know why, but have reported it on #tasty
 2015-03-31 14:47:27 -04:00