Commit graph

2185 commits

Author SHA1 Message Date
Joey Hess
833cf5fff9 log: Fix reversion introduced in version 5.20150528 that broke this command. 2015-07-02 12:53:38 -04:00
Joey Hess
e417c17fc6 releasing package git-annex version 5.20150617 2015-06-17 14:17:20 -04:00
Joey Hess
f20a8f88de prep release 2015-06-16 20:29:09 -04:00
Joey Hess
c217dbf2a5 reorder 2015-06-16 20:28:32 -04:00
Joey Hess
8b74aec3ea Increased the default annex.bloomaccuracy from 1000 to 10000000
This makes git annex unused use around 48 mb more memory than it did before,
but the massive increase in accuracy makes this worthwhile for all but the
smallest systems.

Also, I want to use the bloom filter for sync --all --content, to avoid
dropping files that the preferred content doesn't want, and 1/1000
false positives would be far too many in that use case, even if it were
acceptable for unused.

Actual memory use numbers:

1000: 21.06user 3.42system 0:26.40elapsed 92%CPU (0avgtext+0avgdata 501552maxresident)k
1000000: 21.41user 3.55system 0:26.84elapsed 93%CPU (0avgtext+0avgdata 549496maxresident)k
10000000: 21.84user 3.52system 0:27.89elapsed 90%CPU (0avgtext+0avgdata 549920maxresident)k

Based on these numbers, 10 million seemed a better pick than 1 million.
2015-06-16 18:12:00 -04:00
Joey Hess
f7350b7c33 wording 2015-06-16 17:32:41 -04:00
Joey Hess
8268f7951e adjust standard preferred content to work better with git annex sync --all --content
backup: Use new "anything" terminal. This means that content that
is not unused, but has no associated file will be wanted by backup repos.

unwanted: "not anything" will result in any and all content moving
off of these repos.

incremental backup: Remove the "(include=* or unused)",
so it matches content that has no associated files
but is not unused.

client: Add a include=* to the expression. This limits it to matching
only files in the work tree. Without this change, sync --all --content
would match a key against the expression, and since it matches
exclude=archive/*, the client repo would have wanted the file content.
The "and not unused" would have kept unused objects out, but not
objects that were not known to be unused, or objects that another branch
referred to. In practice, everything would have flooded into client repos
without this change.
2015-06-16 17:18:53 -04:00
Joey Hess
a4955542a3 Fix incremental backup standard preferred content expression to match its documentation, which says it does not want files that have reached a backup repository.
Checked history and these have been out of sync from the very beginning!
2015-06-16 17:10:10 -04:00
Joey Hess
8c46ea22c2 Added new "anything" preferred content expression, which matches all versions of all files. 2015-06-16 17:03:34 -04:00
Joey Hess
29c03145e6 sync: Add support for --all and --unused. 2015-06-16 16:50:03 -04:00
Joey Hess
67f7f1b1cb info: Added json output for "backend usage", "numcopies stats", "repositories containing these files", and "transfers in progress". 2015-06-16 13:50:28 -04:00
Joey Hess
c96b333869 clarify 2015-06-15 15:30:00 -04:00
Joey Hess
9b38c14165 debian/cabal-wrapper: Removed this hack which should not be needed anymore. 2015-06-14 14:43:55 -04:00
Joey Hess
829007d629 Improve url parsing to handle some urls containing illegal [] characters in their paths.
Ie, "https://archive.org/download/zoom-2/Zoom - Release 2 (1996)(Active Software)[!].iso"
2015-06-14 13:39:44 -04:00
Joey Hess
0a998032ed Fix bug that prevented enumerating locally present objects in repos tuned with annex.tune.objecthash1=true
Need to walk 1 level of subdirs less in this case.

The git-annex branch traversal code didn't have a similar bug.
2015-06-11 15:15:05 -04:00
Joey Hess
608172fef5 add git-annex-shell link to standalone deb patch 2015-06-10 13:58:30 -04:00
Joey Hess
e0c2bcdc74 Build documentation with deterministic=1 for reproducible builds. (A new ikiwiki feature.) Closes: #785736 2015-06-09 17:53:55 -04:00
Joey Hess
800e9f2658 tahoe: Use ~/.tahoe-git-annex/ rather than ~/.tahoe/git-annex/ to avoid old versions of tahoe create-client choking. 2015-06-09 15:29:16 -04:00
Joey Hess
e4f22fa721 close debian bug too 2015-06-09 15:15:12 -04:00
Joey Hess
f9bda65245 clarify 2015-06-09 15:08:13 -04:00
Joey Hess
f8ab3bc449 dead --key: Can be used to mark a key as dead. 2015-06-09 14:52:05 -04:00
Joey Hess
6eefc5db65 fsck: Ignore keys that are known to be dead when running in --all mode or a in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead. 2015-06-09 14:08:57 -04:00
Joey Hess
2989cdfe3e re-remove hamlet dep 2015-06-06 14:13:46 -04:00
Joey Hess
5f0f063a7a S3: Publically accessible buckets can be used without creds. 2015-06-05 16:23:35 -04:00
Joey Hess
4acd28bf21 public=yes config to send AclPublicRead
In my tests, this has to be set when uploading a file to the bucket
and then the file can be accessed using the bucketname.s3.amazonaws.com
url.

Setting it when creating the bucket didn't seem to make the whole bucket
public, or allow accessing files stored in it. But I have gone ahead and
also sent it when creating the bucket just in case that is needed in some
case.
2015-06-05 14:38:01 -04:00
Joey Hess
de3bd11a2c import --clean-duplicates: Fix bug that didn't count local or trusted repo's copy of a file as one of the necessary copies to allow removing it from the import location. 2015-06-03 13:15:38 -04:00
Joey Hess
d28e8fbfd5 get --incomplete: New option to resume any interrupted downloads. 2015-06-02 14:20:38 -04:00
Joey Hess
00b094442e assistant: Consume systemd-networkd dbus events to learn about changes to network connections, as was already done with network-manager and wicd. Thanks to Sebastian Reuße for the patches. 2015-06-02 12:54:20 -04:00
Joey Hess
a6d54e49a0 sync, remotedaemon: Pass configured ssh-options even when annex.sshcaching is disabled. 2015-05-30 22:01:52 -04:00
Joey Hess
36b9c9ca5f fromkey, registerurl: Improve handling of urls that happen to also be parsable as strange keys. 2015-05-30 02:08:49 -04:00
Joey Hess
0fd4bef75e prep release 2015-05-28 10:49:04 -04:00
Joey Hess
8990d4cc68 fsck: When checksumming a file fails due to a hardware fault, the file is now moved to the bad directory, and the fsck proceeds. Before, the fsck immediately failed. 2015-05-27 16:40:03 -04:00
Joey Hess
67469ce01f Revert removal dependency on obsolete hamlet package, since the autobuilders are not ready for this change yet and it prevented them from building the webapp. Reopens: #786659 2015-05-27 14:49:58 -04:00
Joey Hess
e810564338 Revert "remove deprecated hamlet package"
This reverts commit cf650eaa99.

It's too early to do this; the linux and android autobuilder will need to
be updated to use the new version of shakespeare, and that will require a
complete refresh of them. In the meantime, this has knocked the webapp out
of the autobuilders.
2015-05-27 14:47:41 -04:00
Joey Hess
5e1f1bd252 Linux standalone, OSX app: Improve runshell script to always quote shell vars, so that it will work when eg, untarred into a directory path with spaces in its name. 2015-05-26 15:38:54 -04:00
Joey Hess
6e05764532 changelog for bug closed in last release 2015-05-24 15:45:36 -04:00
Joey Hess
77c43a388e fromkey, registerurl: Allow urls to be specified instead of keys, and generate URL keys.
This is especially useful because the caller doesn't need to generate valid
url keys, which involves some escaping of characters, and may involve
taking a md5sum of the url if it's too long.
2015-05-22 22:41:36 -04:00
Joey Hess
cab2f05b50 lintian fix 2015-05-22 16:06:16 -04:00
Joey Hess
cf650eaa99 remove deprecated hamlet package
shakespeare provides this now
2015-05-22 15:46:47 -04:00
Joey Hess
979064b1cb prep release 2015-05-22 14:20:21 -04:00
Joey Hess
1b01322de0 OSX: Corrected the location of trustedkeys.gpg, so the built-in upgrade code will find it. Fixes OSX upgrade going forward, but older versions won't upgrade themselves due to this problem. 2015-05-22 13:41:47 -04:00
Joey Hess
8e40cb6f56 Build documentation with TZ=UTC for reproducible builds. See #785736. 2015-05-19 15:45:21 -04:00
Joey Hess
ecb0d5c087 use lock pools throughout git-annex
The one exception is in Utility.Daemon. As long as a process only
daemonizes once, which seems reasonable, and as long as it avoids calling
checkDaemon once it's already running as a daemon, the fcntl locking
gotchas won't be a problem there.

Annex.LockFile has it's own separate lock pool layer, which has been
renamed to LockCache. This is a persistent cache of locks that persist
until closed.

This is not quite done; lockContent stil needs to be converted.
2015-05-19 14:09:52 -04:00
Joey Hess
6ec7c4fef4 webapp: Fix zombie xdg-open process left when opening file browser. Closes: #785498 2015-05-17 17:26:19 -04:00
Joey Hess
b284cefccc fix type in the name of --used-refspec in changelog 2015-05-14 15:46:01 -04:00
Joey Hess
823bb8031b add annex.used-refspec 2015-05-14 15:44:08 -04:00
Joey Hess
86699ff861 unused: Add --used option, which can specify a set of refs to consider used, rather than the default of considering all refs used. 2015-05-14 15:31:38 -04:00
Joey Hess
7ebf234616 Stale transfer lock and info files will be cleaned up automatically when get/unused/info commands are run.
Deleting lock files is tricky, tricky stuff. I think I got it right!
2015-05-12 20:11:23 -04:00
Joey Hess
8c2dd7d8ee Fix an unlikely race that could result in two transfers of the same key running at once.
As discussed in bug report.
2015-05-12 19:39:28 -04:00
Joey Hess
61ccf95004 Avoid accumulating transfer failure log files unless the assistant is being used.
Only the assistant uses these, and only the assistant cleans them up, so
make only git annex transferkeys write them,

There is one behavior change from this. If glacier is being used, and a
manual git annex get --from glacier fails because the file isn't available
yet, the assistant will no longer later see that failed transfer file and
retry the get. Hope no-one depended on that old behavior.
2015-05-12 15:53:38 -04:00
Joey Hess
a812d598ef Take space that will be used by running downloads into account when checking annex.diskreserve. 2015-05-12 15:20:22 -04:00
Joey Hess
505d540360 allow building without ascii-progress, since it is not ready yet
No progress bars with -J unless built with ascii-progress.
2015-05-12 13:54:16 -04:00
Joey Hess
e27b97d364 Merge branch 'master' into concurrentprogress
Conflicts:
	Command/Fsck.hs
	Messages.hs
	Remote/Directory.hs
	Remote/Git.hs
	Remote/Helper/Special.hs
	Types/Remote.hs
	debian/changelog
	git-annex.cabal
2015-05-12 13:23:22 -04:00
Joey Hess
96d24a8161 note about git annex drop behavior change in bare repo 2015-05-12 13:02:22 -04:00
Joey Hess
4b2e3210ea drop: Now supports --all, --unused, and --key. 2015-05-12 13:00:06 -04:00
Joey Hess
db5d831d07 import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss. 2015-05-11 12:57:47 -04:00
Joey Hess
6ebf615995 cutting a point release, will be a cabal-only update 2015-05-11 12:09:02 -04:00
Yaroslav Halchenko
cb23224ba3 ENH: make debianstandalone{,-dsc} rules
Moved rules out of debian/rules and avoided need for patching it for
standalone builds
2015-05-11 11:50:59 -04:00
Joey Hess
26ac0753c1 more FlexibleContexts 2015-05-10 15:54:58 -04:00
Yaroslav Halchenko
e861dd1dd5 ENH: leave only git for depends of standalone -- the rest is provided
Otherwise, versioned depends forbid installation on elderly squeeze
2015-05-10 14:36:50 -04:00
Joey Hess
6a1814abf5 seems that webapp ssh password ssh-agent hack is broken on windows after recent ssh changes 2015-05-08 15:07:23 -04:00
Joey Hess
f1a81cfce2 prep release 2015-05-08 13:42:45 -04:00
Joey Hess
f395ca7a41 Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. 2015-05-08 13:39:00 -04:00
Joey Hess
54bee7dad4 Revert "Android: Updated bundled ssh from 6.1p1 to 6.4p1."
This reverts commit 27836a2af9.

Build didn't actually seem to succeed, for both versions of android, so
I'll need to revisit this later.
2015-05-08 13:05:33 -04:00
Joey Hess
0137656c39 typo 2015-05-07 19:41:05 -04:00
Joey Hess
ee94a79690 Debian's ghc now supports TH on arm! Adjust build dependencies to build the webapp on arm, and enable DAV support on arm. \o/
Also, adjust some other arch specific build dependencies that are now
available on more architectures in Devian unstable.
2015-05-07 19:09:44 -04:00
Joey Hess
4ba4155067 remove workaround for now fixed bug #763078 2015-05-07 18:54:55 -04:00
Joey Hess
e09816b0e6 reorder 2015-05-07 18:12:08 -04:00
Joey Hess
4aba1c74bd remaining dataenc to sandi conversions
I've tested all the dataenc to sandi conversions except Assistant.XMPP,
and all have unchanged behavior, including behavior on large unicode code
points.
2015-05-07 18:07:13 -04:00
Joey Hess
18fc05ff04 Windows: Roll back to an older version of rsync from cygwin. The newer version has some dependency on a newer ssh from cygwin. 2015-05-07 15:56:40 -04:00
Joey Hess
ce285ea6fa Windows: Remove cygwin ssh, the newer version of which has stopped honoring the setting of HOME. Instead, copy msysgit's ssh into PATH. 2015-05-07 15:53:03 -04:00
Joey Hess
469242ac4d fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes: #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only) 2015-05-06 14:45:20 -04:00
Joey Hess
eeb0359a2e contentlocation, examinekey, lookupkey: Added --batch mode option. 2015-05-06 13:44:53 -04:00
Joey Hess
cf786f42a4 Support checking ftp urls for file presence. 2015-05-05 14:05:02 -04:00
Joey Hess
0b18228516 Work around wget bug #784348 which could cause it to clobber git-annex symlinks when downloading from ftp. 2015-05-05 13:53:06 -04:00
Joey Hess
0e6d3a9041 add missing env var setting for standalone build 2015-05-05 13:50:37 -04:00
Yaroslav Halchenko
d15f2a1273 BF: Adjusted debian/patches/standalone-build for recent stylistic changes 2015-05-03 12:02:30 -04:00
Joey Hess
27836a2af9 Android: Updated bundled ssh from 6.1p1 to 6.4p1.
Tested it builds, but have not run it yet.
2015-05-01 15:33:29 -04:00
Joey Hess
437d9db41d assistant: Added --autostop to complement --autostart. 2015-05-01 13:53:45 -04:00
Joey Hess
efb37e7c78 Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end. 2015-04-30 15:28:17 -04:00
Joey Hess
ac6b492711 import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist. 2015-04-30 14:04:36 -04:00
Joey Hess
995643665a Improve integration with KDE's file manager to work with dolphin version 14.12.3 while still being compatable with 4.14.2. Thanks, silvio. 2015-04-29 15:53:16 -04:00
Joey Hess
d8ad1d5503 import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree. 2015-04-29 13:56:41 -04:00
Joey Hess
2e54251c18 import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.) 2015-04-29 13:46:12 -04:00
Joey Hess
986e245ac1 changelog 2015-04-27 18:10:56 -04:00
Joey Hess
ee78958798 S3: Fix incompatability with bucket names used by hS3; the aws library cannot handle upper-case bucket names. git-annex now converts them to lower case automatically.
For example, it failed to get files from a bucket named S3.

Also fixes `git annex initremote UPPERCASE type=S3`, which failed with the
new aws library, with a signing error message.
2015-04-27 18:00:58 -04:00
Joey Hess
cfbeb1e7b7 Fix bogus failure of fsck --fast. 2015-04-27 17:40:21 -04:00
Joey Hess
22a4e92df7 S3: git annex enableremote will not create a bucket name, which failed since the bucket already exists. 2015-04-23 14:16:53 -04:00
Joey Hess
b3eccec68c S3: git annex info will show additional information about a S3 remote (endpoint, port, storage class) 2015-04-23 14:12:25 -04:00
Joey Hess
8f010bb55b also drop old dbus compat 2015-04-22 16:23:41 -04:00
Joey Hess
eb8ef44133 Dropped support for older versions of yesod and warp than the ones in Debian Jessie.
466 lines of compat cruft deleted!
2015-04-22 16:19:11 -04:00
Joey Hess
a15e1158c6 use BuildVersion in debian, which fixes windows build 2015-04-21 16:42:54 -04:00
Joey Hess
3b3aaf0d56 S3: Enable debug logging when annex.debug or --debug is set.
To debug a bug report, but generally useful.
2015-04-21 15:55:42 -04:00
Joey Hess
e651b29e90 reuse Build.Version to generate version 2015-04-20 15:56:28 -04:00
Joey Hess
b0ebb232ec style changes 2015-04-20 14:55:30 -04:00
Joey Hess
bf0e860617 changelog for https://github.com/joeyh/git-annex/pull/41 2015-04-20 14:53:14 -04:00
Joey Hess
92833a7e99 update to generate version in same format git-annex uses 2015-04-20 14:51:34 -04:00
Yaroslav Halchenko
bfef0ba69f NF: provide debian/rules build-standalone 2015-04-20 14:46:34 -04:00
Yaroslav Halchenko
38e03d5b3a NF: debian/patches/*standalone-build for providing a standalone build of git-annex 2015-04-20 14:46:33 -04:00
Joey Hess
b7be1680a7 prep release 2015-04-20 14:44:15 -04:00