Commit graph

2589 commits

Author SHA1 Message Date
Joey Hess
ee8113959a
point to bug report 2015-12-28 13:40:00 -04:00
Joey Hess
cdad377ef1
Debian: Adjust build dependencies for webapp, DAV.
Now available on mips, mipsel, but temporarily removed armel since build is
failing there.

If armel would just get caught up, I could remove the per-arch specs
entirely.

Maybe time to turn maint of this over to richih?
2015-12-28 12:57:15 -04:00
Joey Hess
121f5d5b0c
annex.thin
Decided it's too scary to make v6 unlocked files have 1 copy by default,
but that should be available to those who need it. This is consistent with
git-annex not dropping unused content without --force, etc.

* Added annex.thin setting, which makes unlocked files in v6 repositories
  be hard linked to their content, instead of a copy. This saves disk
  space but means any modification of an unlocked file will lose the local
  (and possibly only) copy of the old version.
* Enable annex.thin by default on upgrade from direct mode to v6, since
  direct mode made the same tradeoff.
* fix: Adjusts unlocked files as configured by annex.thin.
2015-12-27 15:59:59 -04:00
Joey Hess
bb6719678e
update walkthrough and add tip about using v6 unlocked files
The walkthrough should make sense now both for v5 and v6 repo users.
2015-12-26 16:47:23 -04:00
Joey Hess
41b2caa5f6
reword 2015-12-26 13:06:53 -04:00
Joey Hess
87f0708f88
persistent-sqlite is now a hard build dependency, since v6 repository mode needs it. 2015-12-26 13:00:52 -04:00
Joey Hess
39048e4568
Merge branch 'master' into smudge 2015-12-22 18:10:40 -04:00
Joey Hess
a82b7d0044
update 2015-12-22 18:04:43 -04:00
Joey Hess
2dce8081a6
addurl: Added --with-files option. 2015-12-22 12:20:39 -04:00
Joey Hess
ca2c977704
wip v6 support for assistant
Files are not yet added to v6 repos in unlocked mode.
2015-12-21 18:41:15 -04:00
Joey Hess
d82b110da8
Merge branch 'master' into smudge 2015-12-21 17:12:46 -04:00
Joey Hess
a8b398c1fa
addurl: Added --batch option. 2015-12-21 12:57:13 -04:00
Joey Hess
35827e2705
status: On crippled filesystems, was displaying M for all annexed files that were present. Probably caused by a change to what git status displays in this situation. Fixed by treating files git thinks are modified the same as typechanged files. 2015-12-19 13:36:40 -04:00
Joey Hess
f1a73de560
prep release 2015-12-18 12:09:55 -04:00
Joey Hess
029111b89a
Merge branch 'master' into smudge 2015-12-16 13:07:46 -04:00
Joey Hess
96dd0f4ebe
improve temp dir security
http://bugs.debian.org/807341

* Fix insecure temporary permissions when git-annex repair is used in
  in a corrupted git repository.

  Other calls to withTmpDir didn't leak any potentially private data,
  but repair clones the git repository to a temp directory which is made
  using the user's umask. Thus, it might expose a git repo that is
  otherwise locked down.

* Fix potential denial of service attack when creating temp dirs.

  Since withTmpDir used easily predictable temporary directory names,
  an attacker could create foo.0, foo.1, etc and as long as it managed to
  keep ahead of it, could prevent it from ever returning.

  I'd rate this as a low utility DOS attack. Most attackers in a position
  to do this could just fill up the disk /tmp is on to prevent anything
  from writing temp files. And few parts of git-annex use withTmpDir
  anyway, so DOS potential is quite low.

Examined all callers of withTmpDir and satisfied myself that
switching to mkdtmp and so getting a mode 700 temp dir wouldn't break any
of them.

Note that withTmpDirIn continues to not force temp dir to 700.
But it's only used for temp directories inside .git/annex/wherever/
so that is not a problem.

Also re-audited all other uses of temp files and dirs in git-annex.
2015-12-15 20:21:48 -04:00
Joey Hess
7d0e79b9e1
Use git-annex init --version=6 to get v6 for now
Not ready to make it default because of the direct mode upgrade needing to
all happen at once.
2015-12-15 17:17:13 -04:00
Joey Hess
3ba6d84559
clarify 2015-12-15 14:18:44 -04:00
Joey Hess
8a660a7b14
add: In v6 mode, acts on modified files.
Same as was done in direct mode, except in v6 mode add always adds files
locked, so
2015-12-15 14:17:00 -04:00
Joey Hess
f5d4ee9b63
Merge branch 'master' into smudge 2015-12-15 12:01:21 -04:00
Joey Hess
04e00146ed
Debian: Build depend on concurrent-output.
In unstable now.
2015-12-12 12:19:31 -04:00
Joey Hess
c608a752a5
Merge branch 'master' into smudge 2015-12-11 13:50:31 -04:00
Joey Hess
abd66c7089
fsck: Failed to honor annex.diskreserve when checking a remote. 2015-12-11 13:50:27 -04:00
Joey Hess
0f126440ca
webdav: When testing the WebDAV server, send a file with content. The empty file it was sending tickled bugs in some php WebDAV server. 2015-12-11 12:13:20 -04:00
Joey Hess
e2c8dc6778
v6 git-annex unlock
Note that the implementation uses replaceFile, so that the actual
replacement of the work tree file is atomic. This seems a good property to
have!

It would be possible for unlock in v6 mode to be run on files that do not
have their content present. However, that would be a behavior change from
before, and I don't see any immediate need to support it, so I didn't
implement it.
2015-12-10 16:12:48 -04:00
Joey Hess
3d936fdb59
Merge branch 'master' into smudge 2015-12-10 14:07:11 -04:00
Joey Hess
a0528db092
Add S3 features to git-annex version output. 2015-12-10 11:42:49 -04:00
Joey Hess
3719d1b390
make clear when code is using deprecated direct mode files 2015-12-09 19:43:15 -04:00
Joey Hess
bf98d2bd77
Merge branch 'master' into smudge 2015-12-09 17:58:59 -04:00
Joey Hess
e74e612945
prep release 2015-12-08 11:14:28 -04:00
Joey Hess
63c466449f
wording 2015-12-07 12:30:10 -04:00
Joey Hess
62a2fba1cd
Merge branch 'master' into smudge 2015-12-07 12:29:34 -04:00
Joey Hess
2936153fc4
fix temp filename
Was not putting it inside the temp dir, but next to it!

This was just wrong, and it led to a longer filename that desired being
used, leading to some bug reports.
2015-12-06 16:54:01 -04:00
Joey Hess
2fe21d47c5
init: Configure .git/info/attributes to use git-annex as a smudge filter.
Note that this changes the default behavior of git add in a newly
initialized repository; it will add files to the annex.

Don't like that this could break workflows, but it's necessary in order for
any pointer files in the repo to be handled by git-annex.
2015-12-04 17:57:15 -04:00
Joey Hess
ccc49861ca
add v6; keep v5 working for now and manual upgrade
Since all places where a repo is used in direct mode need to have git-annex
upgraded before the repo can safely be converted to v6, the upgrade needs
to be manual for now.

I suppose that at some point I'll want to drop all the direct mode support
code. At that point, will stop supporting v5, and will need to auto-upgrade
any remaining v5 repos. If possible, I'd like to carry the direct mode
support for say, a year or so, to give people plenty of time to upgrade and
avoid disruption.
2015-12-04 16:14:48 -04:00
Joey Hess
37a5e2d419
dropunused: Make more robust when trying to drop an object that has already been dropped.
Before it crashed trying to lock the not-present content and prevented
dropping anything else. Instead, succeed.
2015-12-03 15:58:00 -04:00
Joey Hess
e9f43c07f9
webapp: Fix bugs that could result in a relative path such as "." being written to ~/.config/git-annex/autostart
and ignore any such  relative paths in the file

This was a reversion caused by the relative path changes in 5.20150113.
2015-12-02 15:57:30 -04:00
Joey Hess
f16e235983
addurl, importfeed: Changed to honor annex.largefiles settings, when the content of the url is downloaded. (Not when using --fast or --relaxed.)
importfeed just calls addurl functions, so inherits this from it.

Note that addurl still generates a temp file, and uses that key to download
the file. It just adds it to the work tree at the end when the file is small.
2015-12-02 15:12:33 -04:00
Joey Hess
dc8099872a
import: Changed to honor annex.largefiles settings. 2015-12-02 14:49:03 -04:00
Joey Hess
f7d63a0117
tahoe: Include tahoe capabilities in whereis display. 2015-11-30 15:35:53 -04:00
Joey Hess
0d432dd1a4
annex object file mode for core.sharedRepository
When core.sharedRepository is set, annex object files are not made mode
444, since that prevents a user other than the file owner from locking
them. Instead, a mode such as 664 is used in this case.
2015-11-18 15:45:32 -04:00
Joey Hess
c2674308c0
map: Improve display of git remotes with non-ssh urls, including http and gcrypt. 2015-11-18 15:08:55 -04:00
Joey Hess
e97fce35a6
Display progress meter in -J mode when downloading from the web.
Including in addurl, and get --from web, but also in S3 and External
special remotes when a web url is known for content in those remotes.
2015-11-16 21:00:54 -04:00
Joey Hess
7943442dff
Display progress meter in -J mode when copying from a local git repo, to a local git repo, and from a remote git repo.
Had everything available, just didn't combine the progress meter with the
other places progress is sent to update it. (And to a remote repo already
did show progress.)

Most special remotes should already display progress meters with -J,
same as without it. One exception to this is the web, since it relies on
wget/curl progress display without -J. Still todo..
2015-11-16 19:32:30 -04:00
Joey Hess
09d04fa483
wording 2015-11-16 17:06:34 -04:00
Yaroslav Halchenko
c60afa0891
BF: standalone debian package -- dh_makeshlibs --noscripts
To avoid creation of unnecessary trigger calling out to ldconfig
via activate-noawait which is not present on older releases (e.g. squeeze)
2015-11-16 17:05:30 -04:00
Joey Hess
9f7e4ed811
Build with -j1 again to get reproducible build.
This was in the cabal file earlier, and was removed because it broke the
android cross build. Moving to the git-annex target of the Makefile
will make it be used for Debian packages etc but not android cross builds
or make fast or when users build with cabal.
2015-11-16 16:51:24 -04:00
Joey Hess
2864dc6d15
typo 2015-11-16 15:53:30 -04:00
Joey Hess
b5df6c7b93
prep release 2015-11-16 14:18:01 -04:00
Joey Hess
6fc01da184
wording 2015-11-16 11:39:45 -04:00
Joey Hess
bb86eebfbd
init: Automatically enable annex.pidlock when necessary. 2015-11-13 13:35:29 -04:00
Joey Hess
aa4192aea6
pid locking configuration and abstraction layer for git-annex
(not actually used anywhere yet)
2015-11-12 17:50:34 -04:00
Joey Hess
7c741302cc
assistant: Pass ssh-options through 3 more git pull/push calls that were missed before.
It was used for regular pull, but not for regular push, tagged push, or the
fallback fetching.
2015-11-10 16:52:30 -04:00
Joey Hess
4b02af57b6
display a message in the unlikely scenario of fsking a dead repository 2015-11-10 14:44:58 -04:00
Joey Hess
cd7929034a
fsck: When fscking a dead repo, avoid incorrect "fixing location log" message.
keyLocations doesn't return locations in dead repos, but if we're fscking a
dead repo, we want to look at what locations are actually logged for it.
2015-11-10 13:59:04 -04:00
Joey Hess
361d31fde7
Make git-annex-standalone.deb include the git-annex html documentation, desktop file, and base completion file, same as the regular git-annex.deb.
It already had a doc-base file relating to the html documentation, and
there's no reason not to include the other stuff.
2015-11-10 13:21:05 -04:00
Joey Hess
d9d87fdbc9
Make the git-annex-standalone.deb prevent runshell from installing wrappers into $HOME/.ssh 2015-11-10 12:53:46 -04:00
Joey Hess
3e49f9fcab
runshell: Avoid failing when $HOME/.ssh does not exist and cannot be created. 2015-11-10 11:42:06 -04:00
Joey Hess
08bb3b1b1d
quvi may output utf-8 encoded data when the conifigured locale doesn't support that; avoid crashing on such invalid encoding. 2015-11-09 12:19:23 -04:00
Joey Hess
7938b87864
add: Fix error recovery rollback to not move the injested file content out of the annex back to the file, because other files may point to that same content. Instead, copy the injected file content out to recover.
That was not a data loss, but it came close!
2015-11-06 15:28:20 -04:00
Joey Hess
51e60259e1
fix replaceFile makeAnnexLink race
replaceFile created a temp file, which was guaranteed to not overlap with
another temp file. However, makeAnnexLink then deleted that file, in
preparation for making the symlink in its place. This caused a race, since
some other replaceFile could create a temp file, using the same name!

I was able to reproduce the race easily running git-annex add -J10 in a
directory with 100 files (all with different contents). Some files would
get ingested into the annex, but their annex links would fail to be added.

There could be other situations where this same problem could occur.
Perhaps when the assistant is adding a file, if the user manually also ran
git-annex add. Perhaps in cases not involving adding a file.

The new replaceFile makes a temprary directory, which is guaranteed to be
unique, and doesn't make a temp file in there. makeAnnexLink can thus
create the symlink without problem and the race is avoided.

Audited all calls to replaceFile to make sure that the old behavior of
providing an empty temp file was not relied on.

The general problem of asking for a temp file and deleting it as part of
the process of using it could reach beyond replaceFile. Did some quick
audits and didn't find other cases of it. Probably only symlink creation
stuff would tend to make that mistake, mostly.
2015-11-06 15:08:19 -04:00
Joey Hess
416c7656a1
Concurrent progress bars are now displayed when using -J with a command that moves file contents around. 2015-11-06 13:44:57 -04:00
Joey Hess
7dc90f2225
import: Avoid very ugly error messages when the directory files are imported to is not a directort, but perhaps an annexed file. 2015-11-05 18:46:05 -04:00
Joey Hess
5db7d435e7
-J for add/addurl/import 2015-11-05 18:24:15 -04:00
Joey Hess
c4d45ef83d
drop -Jn 2015-11-04 17:13:20 -04:00
Joey Hess
3d0f41518d
parallel fsck (yes, these changes are all it takes now!) 2015-11-04 16:28:14 -04:00
Joey Hess
4fd03ccd7b
concurrent-output, first pass
Output without -Jn should be unchanged from before. With -Jn,
concurrent-output is used for messages, but regions are not used yet, so
it's a mess.
2015-11-04 13:45:34 -04:00
Joey Hess
61987c6983
point release for cabal file fixes
Seems that some changes to the cabal file a few months ago resulted in a
git-annex that broke stackage infrastructure.
2015-11-03 12:10:01 -04:00
Joey Hess
628429f1b0
When cabal install is run with the desktop file location not writable, display a warning, but continue successfully.
Installing the desktop file etc is a niceity of git-annex's cabal install,
but not a requirement.

closes https://github.com/fpco/stackage/issues/726
2015-11-03 10:47:33 -04:00
Joey Hess
0717ad4440
Avoid installing desktop file and program file if cabal install git-annex is run as root, since that is not a systemwide install, but to /root, and so generating a systemwide desktop file is not right. 2015-11-03 10:38:08 -04:00
Joey Hess
29d250923b
prep release 2015-11-02 12:41:33 -04:00
Joey Hess
4153507864
Fix failure to build with aws-0.13.0 and finish nearline support.
* Fix failure to build with aws-0.13.0.
* When built with aws-0.13.0, the S3 special remote can be used to create
  google nearline buckets, by setting storageclass=NEARLINE.
2015-11-02 11:14:03 -04:00
Joey Hess
ef5496b8de
Catch up with current git behavior when both repo and repo.git exist; it seems it now prefers repo in this case, although historically it may have preferred repo.git. 2015-10-26 15:35:55 -04:00
Joey Hess
640dba43b6
enableremote: List uuids and descriptions of remotes that can be enabled, and accept either the uuid or the description in leu if the name. 2015-10-26 14:55:40 -04:00
Henrik Ahlgren
87a00e3608
Changelog typo: remove, not remote 2015-10-21 10:10:55 -04:00
Joey Hess
268800d590
Symlink timestamp preservation code uses functions from unix-2.7.0 when available, which should be more portable. 2015-10-21 02:22:18 -04:00
Joey Hess
b9c6a56b0e
Use statvfs on OSX.
Fixes a recent-ish build warning on about 64 bit vs non.

This is the method used by the disk-free-space library, and I tested it to
yield the same results on even 10 tb drives on OSX -- so it's getting 64
bit values.
2015-10-19 17:09:06 -04:00
Joey Hess
b1c0b7175f
Debian: Remote menu file, since a desktop file is provided and lintian says there can be only one. 2015-10-19 14:23:48 -04:00
Joey Hess
4cc892f938
prep release 2015-10-19 14:08:14 -04:00
Joey Hess
806819be57
Avoid displaying network transport warning when a ssh remote does not yet have an annex.uuid set.
Instead, only display transport error if the configlist output doesn't
include an annex.uuid line, even an empty one.

A recent change made git-annex init try to get all the remote uuids, and so
the transport error would be displayed by it. It was also displayed when
eg, copying files to a remote that had no uuid yet.
2015-10-15 15:36:54 -04:00
Joey Hess
27eaa6f410
avoid making post-merge-conflict-resolution commit when no conflicts were resolved
sync, merge, assistant: When git merge failed for a reason other than a
conflicted merge, such as a crippled filesystem not allowing particular
characters in filenames, git-annex would make a merge commit that could
omit such files or otherwise be bad. Fixed by aborting the whole merge
process when git merge fails for any reason other than a merge conflict.
2015-10-15 14:22:46 -04:00
Joey Hess
c32a2429ed
S3: Fix support for using https.
Was using the http-only Manager before, not the tls-capable one.
2015-10-15 10:37:06 -04:00
Joey Hess
9e90c033d3
Changed drop ordering when using git annex sync --content or the assistant, to drop from remotes first and from the local repo last. This works better with the behavior changes to drop in many cases. 2015-10-14 12:33:02 -04:00
Joey Hess
f9adb905fc
Avoid unncessary write to the location log when a file is unlocked and then added back with unchanged content.
Implemented with no additional overhead of compares etc.

This is safe to do for presence logs because of their locality of change;
a given repo's presence logs are only ever changed in that repo, or in a
repo that has just been actively changing the content of that repo.

So, we don't need to worry about a split-brain situation where there'd
be disagreement about the location of a key in a repo. And so, it's ok to
not update the timestamp when that's the only change that would be made
due to logging presence info.
2015-10-12 14:46:47 -04:00
Joey Hess
6145f905e0
improve display when lockcontent fails
/dev/null stderr; ssh is still able to display a password prompt
despite this

Show some messages so the user knows it's locking a remote, and
knows if that locking failed.
2015-10-09 17:31:02 -04:00
Joey Hess
6d4f741d5f
cleanup changelog 2015-10-09 16:18:16 -04:00
Joey Hess
cf79dffa4c
improve drop proof code 2015-10-09 11:09:46 -04:00
Joey Hess
5240a9f315
git-annex-shell: Added lockcontent command, to prevent dropping of key's content. 2015-10-08 14:47:46 -04:00
Joey Hess
976110b992 Debian: Remove dependency on transformers library, as it is now included in ghc.
Backporters will need to revert this.
2015-10-07 12:26:48 -04:00
Joey Hess
61c45072e6 Debian: Add torrent library to build-depends as it's packaged now, and stop recommending bittornado | bittorrent. 2015-10-07 09:08:37 -04:00
Joey Hess
1ac79e6fe5 copy --auto was checking the wrong repo's preferred content. (--from was checking what --to should, and vice-versa.) Fixed this bug, which was introduced in version 5.20150727. 2015-10-06 17:29:44 -04:00
Joey Hess
933fef6ae0 Merge branch 'winprocfix' 2015-10-04 15:46:25 -04:00
Joey Hess
06f1f03e7a Ported disk free space checking code to work on Solaris.
On Solaris, using f_bsize provided a value that is apparently much larger
than the real block size. The solaris docs for statvfs say
f_bsize is the "preferred" file system block size, and I guess the
filesystem prefers larger blocks, but uses smaller ones or something.
The docs also say that f_frsize is the "fundamental" block size.

Switched to using f_frsize on Linux and kFreeBSD too, since I guess
f_bsize could in theory vary the same way there too. Assuming that Solaris
is not violating the posix spec, I guess the linux man page for statvfs
is not as well written and I misunderstood it.
2015-10-02 16:31:15 -04:00
Joey Hess
16d7035ef7 Allow building with S3 disabled again. 2015-10-02 12:12:02 -04:00
Joey Hess
2fb3722ce9 Do verification of checksums of annex objects downloaded from remotes.
* When annex objects are received into git repositories, their checksums are
  verified then too.
* To get the old, faster, behavior of not verifying checksums, set
  annex.verify=false, or remote.<name>.annex-verify=false.
* setkey, rekey: These commands also now verify that the provided file
  matches the key, unless annex.verify=false.
* reinject: Already verified content; this can now be disabled by
  setting annex.verify=false.

recvkey and reinject already did verification, so removed now duplicate
code from them. fsck still does its own verification, which is ok since it
does not use getViaTmp, so verification doesn't happen twice when using fsck
--from.
2015-10-01 15:56:39 -04:00
Joey Hess
d4d880c1a4 prep release 2015-09-30 14:32:45 -04:00
Joey Hess
828bafebea mention ancient build 2015-09-29 11:41:13 -04:00
Joey Hess
028cdccc65 Added stack.yaml to support easy builds from source with stack.
Update install instructions.

Note that xmpp is disabled in the stack.yaml, because that needs C
libraries that would take extra work for the user to install.
xmpp is moving toward deprecation anyway.
2015-09-26 06:20:09 -04:00
Joey Hess
f2b6ebd502 status: Show added but not yet committed files.
Seems easy, but git ls-files can't list the right subset of files.
So, I wrote a whole new parser for git status output, and converted the
status command to use that.

There are a few other small behavior changes. The order changed. Unlocked
files show as T. In indirect mode, deleted files were not shown before, and
that's fixed. Regular files checked directly into git and modified
were not shown before, and are now.
2015-09-22 17:32:28 -04:00
Joey Hess
dc2f1f09b7 Improve robustness of direct mode merge, avoiding a crash if the index file is missing.
I couldn't find a good way to make an *empty* index file (zero byte file
won't do), so I punted and just don't make index.lock when there's no index
yet. This means some other git process could race and write an index file
at the same time as the merge is ongoing, in theory. Only happens in new
repos though.
2015-09-22 13:00:18 -04:00
Joey Hess
1dcb86498e Improve ~/.ssh/config modification code to not add trailing spaces to lines it cannot parse.
"Host\n" is a valid line, and actually gets parsed ok, but this also holds
for other lines that it fails to parse for some reason.
2015-09-22 12:06:10 -04:00
Joey Hess
f33e760aab Merge branch 's3-classes' 2015-09-22 11:05:53 -04:00
Joey Hess
20205b6073 avoid hard dependency on new version of aws 2015-09-22 11:04:26 -04:00
Joey Hess
3058e11520 webapp: Remove the "disable remote" feature from the UI.
It's confusing, easy to select by accident and get into a situation the
webapp offers no easy recovery from, and pausing syncing works just as well
in most situations.
2015-09-21 13:05:44 -04:00
Joey Hess
0ebde659bf assistant: When updating ~/.ssh/config, preserve any symlinks. 2015-09-21 12:39:13 -04:00
Joey Hess
26d6566307 S3 storage classes expansion
Added support for storageclass=STANDARD_IA to use Amazon's
new Infrequently Accessed storage.

Also allows using storageclass=NEARLINE to use Google's NearLine storage.

The necessary changes to aws to support this are in
https://github.com/aristidb/aws/pull/176
2015-09-17 17:20:01 -04:00
Joey Hess
9e48c04d15 info: Don't allow use in a non-git-annex repository, since it uses the git-annex branch and would create it if it were missing.
I made the change to allow in 2014 without any rationalle or associated
request that I can find.
2015-09-16 12:25:43 -04:00
Joey Hess
23f3bd6453 prep release 2015-09-16 10:31:41 -04:00
Joey Hess
16947ef654 Fix bug in combination of preferred and required content settings. When one was set to the empty string and the other set to some expression, this bug caused all files to be wanted, instead of only files matching the expression.
Avoid: MAny `MOr` otherexpression
Which matches anything.
2015-09-15 12:50:14 -04:00
Joey Hess
9cfb96c53d Special remotes configured with autoenable=true will be automatically enabled when git-annex init is run. 2015-09-14 14:49:48 -04:00
Joey Hess
3f47d1b351 Improve bash completion, so it completes names of remotes and backends in appropriate places.
Not necessarily everywhere, but a lot of the most often used places.

Re the use of .Internal, see
https://github.com/pcapriotti/optparse-applicative/issues/155
2015-09-14 13:19:04 -04:00
Joey Hess
ffa8221517 annex.hardlink extended to also try to use hard links when copying from the repository to a remote.
Also, it used to only check that one of the repos was not in direct mode;
now when either repo is direct mode, annex.hardlink won't have an effect.
2015-09-14 12:13:38 -04:00
Joey Hess
1cc1f9f4e5 sync: Add --no-commit, --no-pull, --no-push options to turn off parts of the sync process, as well as supporting --commit, --pull, --push, and --no-content options to specify the (current) default behavior. 2015-09-13 13:15:35 -04:00
Joey Hess
1355a241ea now fully working even when git is not in path
The vbs launchers now are passed the path where git-annex was installed to,
so they will work when it's not in path.
2015-09-11 13:18:21 -04:00
Joey Hess
c7a6296ef3 get git-annex working even if user doesn't add git to path
This is pretty complicated, but I have both "git-annex" and "git annex"
working both in the git bash shell even with git not added to path.
And, when git's added to path, both work from MS-DOS prompt window too.

I think that the webapp startup does still need git in path, so
instructions will keep saying to do that. But, users often disregard them,
and hopefully this will reduce support traffic.

Also, switched the wget from the cygwin one to the msys2 one, avoiding the
complication of needing to bundle any cygwin dlls.
2015-09-11 12:44:27 -04:00
Joey Hess
6158036e23 Switched to using git for Windows, rather than msysgit.
Using msysgit with git-annex is no longer supported.

At the same time, I'm updating the rsync.exe in my downloads repository
with the one from msys2.

Note that rsync is currently still being ldded and installed in Git/cmd/
like the other cygwin programs. The ldd fails and this failure is ignored.
It would be better to special case it to go in Git/usr/bin/, so that the
user can't run rsync in a dos prompt window, which doesn't work, as it needs
additional libs. However, as far as git-annex running rsync running ssh,
it works ok in this location.

Removed the ssh.cmd and ssh-keygen.cmd; these are not needed with git for
windows. Keeping them would let ssh be run manually from a dos prompt
window, but that's not really a goal.
2015-09-10 19:16:30 -04:00
Joey Hess
0390efae8c support gpg.program
When gpg.program is configured, it's used to get the command to run for
gpg. Useful on systems that have only a gpg2 command or want to use it
instead of the gpg command.
2015-09-09 18:06:49 -04:00
Joey Hess
4ed82e5328 fsck: Work around bug in persistent that broke display of problematically encoded filenames on stderr when using --incremental. 2015-09-09 17:02:00 -04:00
Joey Hess
f6c03414cf Make full option parsing be done when not in a git repo, so --help can be displayed for commands that require a git repo, etc. 2015-09-09 15:55:13 -04:00
Joey Hess
c60181bd82 unused: Fix reversion in 5.20150727 that broke parsing of the --unused-refspec option. Thanks, Øyvind A. Holm. 2015-09-09 14:33:35 -04:00
Joey Hess
2b326594a9 info: Support querying info of individual files in direct mode. 2015-09-09 14:15:56 -04:00
Joey Hess
97962591d6 init: Fix reversion in detection of repo made with git clone --shared 2015-09-09 13:56:37 -04:00
Joey Hess
19dbe2a611 webapp: Fix support for entering password when setting up a ssh remote. 2015-09-03 11:03:08 -07:00
Joey Hess
86e638567a Fix Windows build to work with ghc 7.10
It was failing at link time, some problem with terminatePID.
Re-implemented that to not use a C wrapper function, which cleared up the
problem. Removed old EvilLinker hack with must have been related to the
same problem.

Note that I have not tested this with older ghc's. In
f11f7520b5 I mention having tried this
approach before, and getting segfaults.. So, who knows. It seems to work
fine with ghc 7.10 at least.
2015-09-01 14:51:14 -07:00
Joey Hess
33978f8448 prep release 2015-08-24 14:12:38 -07:00
Joey Hess
0b7a8b72bb Fix building without database.
Ben Boeckel had a patch, but..
Actually, that was not the only place that used ScheduleIncremental when
built w/o database. Since the data type doesn't need database stuff,
I've instead fixed this build problem by exposing the
ScheduleIncremental constructor to database-less builds.
2015-08-23 15:39:29 -07:00
Joey Hess
d20cd4659a Reorder declaration to fix build with yesod-core > 1.4.13. Thanks, Michael Alan Dorman. 2015-08-23 10:44:09 -07:00
Joey Hess
485465107a Makefile: Pass LDFLAGS, CFLAGS, and CPPFLAGS through ghc and on to ld, cc, and cpp.
As a result of the Makefile changes, the Debian package is built
with various hardening options. Although their benefit to a largely
haskell program is unknown.
2015-08-19 13:53:57 -04:00
Joey Hess
fec1568054 typo 2015-08-19 13:17:07 -04:00
Joey Hess
c242e248e8 Fix reversion in init when ran as root, introduced in version 5.20150731. 2015-08-19 12:36:17 -04:00
Joey Hess
0f5d6c09ac importfeed --relaxed: Avoid hitting the urls of items in the feed. 2015-08-19 12:24:55 -04:00
Joey Hess
ba56d10c60 Avoid building the assistant on the hurd, since an inotify equivilant is not yet implemented in git-annex for the hurd.
Note that https://github.com/haskell/hackage-server/issues/269 is fixed, so
hopefully I can upload this to hackage this time.
2015-08-17 11:53:16 -04:00
Joey Hess
ab85a48bc0 merge lintian overrides from debian 2015-08-15 11:45:37 -04:00
Joey Hess
0050215ed1 enable parallel build for fast mode and explicitly disable for production
Currently, ghc has issues getting reproducible builds with parallel
building. https://ghc.haskell.org/trac/ghc/ticket/4012
2015-08-14 17:39:56 -04:00
Joey Hess
a4fe6e037b oops, didn't mean to commit these changes yet 2015-08-14 14:22:53 -04:00
Joey Hess
290ff35b53 comment 2015-08-14 14:13:49 -04:00
Joey Hess
87b4229b23 sync: Support --jobs
* sync: Support --jobs
* sync --content: Avoid unnecessary second pull from remotes when
  no file transfers are made.
2015-08-14 13:49:55 -04:00
Joey Hess
6bc46e384e Added WHEREIS to external special remote protocol. 2015-08-13 17:27:50 -04:00
Joey Hess
43aa881b47 --debug is passed along to git-annex-shell when git-annex is in debug mode. 2015-08-13 15:05:39 -04:00
Joey Hess
983a95f021 Sped up downloads of files from ssh remotes, reducing the non-data-transfer overhead 6x. 2015-08-13 14:20:28 -04:00
Joey Hess
7584e47ba3 --debug log messages are now timestamped with fractional seconds. 2015-08-12 14:42:49 -04:00
Joey Hess
730573e0ba prep release 2015-08-12 11:16:38 -04:00
Joey Hess
ff97803279 clarify 2015-08-11 18:49:06 -04:00
Joey Hess
23e9d3bb77 Fix setting/setting/viewing metadata that contains unicode or other special characters, when in a non-unicode locale.
Oh boy, not again. So, another place that the filesystem encoding needs to
be applied. Yay.

In passing, I changed decodeBS so if a NUL is embedded in the input, the
resulting FilePath doesn't get truncated at that NUL. This was needed to
make prop_b64_roundtrips pass, and on reviewing the callers of decodeBS, I
didn't see any where this wouldn't make sense. When a FilePath is used to
operate on the filesystem, it'll get truncated at a NUL anyway, whereas if
a String is being used for something else, it might conceivably have a NUL
in it, and we wouldn't want it to get truncated when going through
decodeBS.
(NB: There may be a speed impact from this change.)
2015-08-11 18:40:59 -04:00
Joey Hess
0f66f766b0 metadata: Fix reversion introduced in 5.20150727 that caused display of metadata to not work. 2015-08-11 13:19:01 -04:00
Joey Hess
f6f3ad7be5 tyo 2015-08-09 11:51:51 -04:00
Joey Hess
8167f6fb40 Improve Setup.hs file so that cabal copy --destdir works. Thanks, Magnus_Therning. 2015-08-09 10:30:35 -04:00
Joey Hess
0ec9bc2200 Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library.
While cryptohash has SHA3 support, it has not been updated for the final
version of the spec. Note that cryptonite has not been ported to all arches
that cryptohash builds on yet.
2015-08-06 15:02:25 -04:00
Joey Hess
367d1352da git-annex-shell: Don't let configlist auto-init repository when in readonly mode.
This was potentially a hole in the readonly mode armor even before my last
commit. If the user could push a git-annex branch to a repo, they could get
git-annex-shell to initialize the repo. After my last commit, the user
didn't even need to be allowed to push a branch to init the repo, so
this hole certianly needs to be closed now.
2015-08-05 14:09:25 -04:00
Joey Hess
c5b8484c2e Simplify setup process for a ssh remote.
Now it suffices to run git remote add, followed by git-annex sync. Now the
remote is automatically initialized for use by git-annex, where before the
git-annex branch had to manually be pushed before using git-annex sync.
Note that this involved changes to git-annex-shell, so if the remote is
using an old version, the manual push is still needed.

Implementation required git-annex-shell be changed, so configlist can
autoinit a repository even when no git-annex branch has been pushed yet.
Unfortunate because we'll have to wait for it to get deployed to servers
before being able to rely on this change in the documentation.

Did consider making git-annex sync push the git-annex branch to repos that
didn't have a uuid, but this seemed difficult to do without complicating it
in messy ways.

It would be cleaner to split a command out from configlist to handle
the initialization. But this is difficult without sacrificing backwards
compatability, for users of old git-annex versions which would not use the
new command.
2015-08-05 13:49:58 -04:00
Joey Hess
7133b68afe Linux standalone: Work around problem that prevented it from working properly if unpacked into a directory that contains ":" or ";" in its name. 2015-08-04 16:08:19 -04:00
Joey Hess
f041a65c33 Windows: Fix bug that caused git-annex sync to fail due to missing environment variable.
I think that the problem was caused by windows not having a concept of an
env var that is set, but to the empty string. So, GIT_ANNEX_SSHOPTION
got set to "" and was not seen as set at all.

Easy fix, which also makes git-annex sync a little faster is to not set
GIT_SSH, when GIT_ANNEX_SSHOPTION has no options. Might as well let git use
ssh per usual in this case, no need to run git-annex as the proxy ssh
command..
2015-08-04 15:27:48 -04:00
Joey Hess
3cff287b26 proxy: Fix behavior when run in subdirectory of git repo.
This fixes a reversion introduced by relative path changes back last winter.

The root cause is simplifyPath "../foo" was incorrectly yielding "foo".

absPathFrom seems quite horrible. Probably most things that use it should
use </> instead.
2015-08-04 14:58:21 -04:00
Joey Hess
c812018cc5 proxy: Fix removal of files deleted by the proxied command.
Git.Ref.headSha doesn't really work in direct mode as there's not a head,
so it was actually diffing against the empty tree and so not removing any
deleted files. Get the sha of the current branch instead, which is the same
thing Command.Sync does.
2015-08-04 14:43:42 -04:00
Joey Hess
6c15cdfcb8 proxy: Fix proxy git commit of non-annexed files in direct mode.
* proxy: Fix proxy git commit of non-annexed files in direct mode.
* proxy: If a non-proxied git command, such as git revert
  would normally fail because of unstaged files in the work tree,
  make the proxied command fail the same way.
2015-08-04 14:01:59 -04:00
Joey Hess
dfd6981785 remove workaround for old bug #763078 2015-08-04 13:00:22 -04:00
Joey Hess
098b33690e Added back debian/cabal-wrapper, since it still seems needed after all. 2015-08-04 12:25:06 -04:00
Joey Hess
7cf0cb48ee Tighten dependency on optparse-applicative to 0.11.0.
Type of str changed in 0.11.
2015-08-02 19:05:24 -04:00
Joey Hess
bc4129cc77 fsck: Commit incremental fsck database after every 1000 files fscked, or every 5 minutes, whichever comes first.
Previously, commits were made every 1000 files fscked.

Also, improve docs
2015-07-31 16:42:15 -04:00
Joey Hess
9dfe03dbcd Improve shutdown due to --time-limit, especially for fsck
* Perform a clean shutdown when --time-limit is reached.
  This includes running queued git commands, and cleanup actions normally
  run when a command is finished.
* fsck: Commit incremental fsck database when --time-limit is reached.
  Previously, some of the last files fscked did not make it into the
  database when using --time-limit.

Note that this changes Annex.addCleanup hooks, to run after --time-limit
expires. Fsck was using such a hook to clean up after a
--incremental-schedule, and that shouldn't run when --time-limit exipires
it. So, instead, moved that cleanup code to be run by cleanupIncremental.
Resulted in some data type juggling.
2015-07-31 16:01:54 -04:00
Joey Hess
e954b165a1 prep release 2015-07-31 11:34:53 -04:00
Joey Hess
14d15dcf0e response 2015-07-31 11:34:09 -04:00
Joey Hess
b30324fec7 init: Detect when the filesystem is crippled such that it ignores attempts to remove the write bit from a file, and enable direct mode. Seen with eg, NTFS fuse on linux. 2015-07-30 14:06:17 -04:00
Joey Hess
b7a5d9c3e1 The last release accidentially removed a number of options from the copy command. (-J, file matching options, etc). These have been added back. 2015-07-30 13:33:35 -04:00
Joey Hess
506452012c Fix rsync special remote to work when -Jn is used for concurrent uploads. 2015-07-30 13:29:45 -04:00
Joey Hess
29f3049def webapp: Support enabling known gitlab.com remotes. 2015-07-27 16:03:22 -04:00
Joey Hess
b62f8fa6ad prep release 2015-07-27 12:25:03 -04:00
Joey Hess
98aa61e766 Merge branch 'gitlab' 2015-07-27 12:22:35 -04:00
Joey Hess
ee25170de5 changelog for gitlab 2015-07-27 12:17:35 -04:00
Joey Hess
1fb9ab342b Support building without persistent database on for systems that lack TH. This removes support for incremental fsck. 2015-07-25 17:37:09 -04:00
Joey Hess
49d102f98b addurl now accepts --prefix and --suffix options to adjust the filenames used 2015-07-21 12:50:05 -04:00
Joey Hess
676115220c wording 2015-07-20 20:41:55 -04:00
Joey Hess
4043837050 reword 2015-07-20 20:30:27 -04:00
Joey Hess
bc9c33e846 looks like richih will maintain it in debian 2015-07-20 20:23:56 -04:00
Joey Hess
e99affb71b Adjust debian build deps: The webapp can now build on arm64, s390x and hurd-i386. WebDAV support is also available on those architectures. 2015-07-20 20:20:34 -04:00
Joey Hess
4cc8433d5d correct old changelog entry
It looked at permalinks, not guids
2015-07-20 14:59:20 -04:00
Joey Hess
f95a8c8672 importfeed: Look at not only permalinks, but now also guids to identify previously downloaded files.
I've seen rss feeds that have no permalinks, only guids (which are
sometimes in the form of permalinks, argh/sigh).

I had previously avoided trusting guids to be globally unique, because my
survey of rss feeds that I subscribe to shows a lot of pretty bad
"guids" like "2 at http://serialpodcast.org" or even worse "oth20150401-hq".
Worry was that two podcasts that are generating guids so badly, that
there's no guarantee they're actually globally unique.

But, I'm seeing too many url changes that result in redundant files, so
let's try this. If feeds are so broken that guids overlap, they could just
as well incorrectly call them permalinks too.
2015-07-20 14:56:57 -04:00
Joey Hess
3c134ee21a sync --content: Fix bug that caused files to be uploaded to eg, more archive remotes than wanted copies, only to later be dropped to satisfy the preferred content settings. 2015-07-20 14:24:13 -04:00
Joey Hess
113ad85a15 reword 2015-07-20 13:32:34 -04:00
Joey Hess
36cf98a24f update 2015-07-20 10:50:59 -04:00
Joey Hess
afe6a53bca Fix bug that prevented uploads to remotes using new-style chunking from resuming after the last successfully uploaded chunk.
"checkPresent baser" was wrong; the baser has a dummy checkPresent action
not the real one. So, to fix this, we need to call preparecheckpresent to
get a checkpresent action that can be used to check if chunks are present.

Note that, for remotes like S3, this means that the preparer is run,
which opens a S3 handle, that will be used for each checkpresent of a
chunk. That's a good thing; if we're resuming an upload that's already many
chunks in, it'll reuse that same http connection for each chunk it checks.
Still, it's not a perfectly ideal thing, since this is a different http
connection that the one that will be used to upload chunks. It would be
nice to improve the API so that both use the same http connection.
2015-07-16 15:01:27 -04:00
Joey Hess
5de3b4d07a update for completion changes 2015-07-16 14:57:23 -04:00
Joey Hess
02d522a12e Debian package (and any other packages built using make install) now includes bash completion. 2015-07-13 12:26:06 -04:00
Joey Hess
b95a48fe45 converted Assistant and Watch; avoid duplicate arg parsing for no repo mode 2015-07-13 11:42:42 -04:00
Joey Hess
3d8a8e9792 convert Dead, and allow multiple --key options 2015-07-13 10:26:54 -04:00
Joey Hess
adec382bc2 converted MetaData, eliminating a global value from Annex state .. beautiful 2015-07-12 09:25:43 -04:00
Joey Hess
dd54d9b617 Merge branch 'master' into optparse-applicative
Conflicts:
	debian/changelog
2015-07-10 17:07:54 -04:00
Joey Hess
3074633955 typo 2015-07-10 16:52:58 -04:00
Joey Hess
bda6a020e6 prep release 2015-07-10 16:39:05 -04:00
Joey Hess
b4d22e6d49 doc updates 2015-07-10 13:49:37 -04:00
Joey Hess
aea694ed3f Removed support for optparse-applicative versions older than 0.10.
Debian stable has 0.10.0.
2015-07-07 23:54:46 -04:00
Joey Hess
b11d2f5a8a unused: --used-refspec can now be configured to look at refs in the reflog. This provides a way to not consider old versions of files to be unused after they have reached a specified age, when the old refs in the reflog expire.
May be slow.
2015-07-07 17:13:50 -04:00
Joey Hess
a51b98cdd5 sync: When annex.autocommit=false, avoid making any commit of local changes, while still merging with remote to the extent possible. 2015-07-07 16:36:11 -04:00
Joey Hess
81ad277a85 add: Stage symlinks the same as git add would, even if they are not a link to annexed content. 2015-07-07 16:15:30 -04:00
Joey Hess
60fc25e9d3 merge: Avoid creating the synced/master branch.
The branch needs to be created when merging from the remote in sync,
since we diff between it and the remote's sync branch. But git annex merge
should not be creating sync branches.
2015-07-07 15:53:27 -04:00
Joey Hess
625303226d import: Fix failure of cross-device import on Windows.
As well as import, 2 other places ran "mv" manually, so changed them to use
moveFile as well.
2015-07-07 14:48:23 -04:00
Joey Hess
f8bc2d159e clarify 2015-07-07 13:20:57 -04:00
Joey Hess
751f9c40ce On linux, pass --as-needed to linker to avoid linking with unused shared libraries including libyaml. 2015-07-07 13:19:13 -04:00
Joey Hess
dbd41159e0 Support git's undocumented core.sharedRepository=2 value, which is equivilant to "world". 2015-07-06 15:33:44 -04:00
Joey Hess
174da80ddc bugfix: Pass --full-tree when using git ls-files to get a list of files on the git-annex branch, so it works when run in a subdirectory. This bug affected git-annex unused, and potentially also transitions running code and other things. 2015-07-06 14:09:54 -04:00
Joey Hess
1529add61a Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency. 2015-07-02 17:44:25 -04:00
Joey Hess
4af3dc72c1 assistant: Fix ANNEX_SHELL_DIR written to ~/.ssh/authorized_keys in local pairing to be the absolute path to the repository, not "."
This was a reversion caused by the relative path changes in 5.20150113.

Other uses of addAuthorizedKeys seem to be ok. If the user enters a
directory like ~/annex, it writes GIT_ANNEX_SHELL_DIRECTORY=annex, and
git-annex-shell assumes that's relative to HOME.
2015-07-02 15:10:50 -04:00
Joey Hess
20edbf850c assistant: Fix local pairing to not include newline in ssh pubkey, which is rejected on the other end for security reasons. 2015-07-02 14:50:20 -04:00
Joey Hess
1f35141aff assistant --autostart: First any daemons that are already running, which might be left over from a previous login session and so unable to use the ssh agent of a new login session. 2015-07-02 14:16:53 -04:00
Joey Hess
833cf5fff9 log: Fix reversion introduced in version 5.20150528 that broke this command. 2015-07-02 12:53:38 -04:00
Joey Hess
e417c17fc6 releasing package git-annex version 5.20150617 2015-06-17 14:17:20 -04:00
Joey Hess
f20a8f88de prep release 2015-06-16 20:29:09 -04:00
Joey Hess
c217dbf2a5 reorder 2015-06-16 20:28:32 -04:00
Joey Hess
8b74aec3ea Increased the default annex.bloomaccuracy from 1000 to 10000000
This makes git annex unused use around 48 mb more memory than it did before,
but the massive increase in accuracy makes this worthwhile for all but the
smallest systems.

Also, I want to use the bloom filter for sync --all --content, to avoid
dropping files that the preferred content doesn't want, and 1/1000
false positives would be far too many in that use case, even if it were
acceptable for unused.

Actual memory use numbers:

1000: 21.06user 3.42system 0:26.40elapsed 92%CPU (0avgtext+0avgdata 501552maxresident)k
1000000: 21.41user 3.55system 0:26.84elapsed 93%CPU (0avgtext+0avgdata 549496maxresident)k
10000000: 21.84user 3.52system 0:27.89elapsed 90%CPU (0avgtext+0avgdata 549920maxresident)k

Based on these numbers, 10 million seemed a better pick than 1 million.
2015-06-16 18:12:00 -04:00
Joey Hess
f7350b7c33 wording 2015-06-16 17:32:41 -04:00
Joey Hess
8268f7951e adjust standard preferred content to work better with git annex sync --all --content
backup: Use new "anything" terminal. This means that content that
is not unused, but has no associated file will be wanted by backup repos.

unwanted: "not anything" will result in any and all content moving
off of these repos.

incremental backup: Remove the "(include=* or unused)",
so it matches content that has no associated files
but is not unused.

client: Add a include=* to the expression. This limits it to matching
only files in the work tree. Without this change, sync --all --content
would match a key against the expression, and since it matches
exclude=archive/*, the client repo would have wanted the file content.
The "and not unused" would have kept unused objects out, but not
objects that were not known to be unused, or objects that another branch
referred to. In practice, everything would have flooded into client repos
without this change.
2015-06-16 17:18:53 -04:00
Joey Hess
a4955542a3 Fix incremental backup standard preferred content expression to match its documentation, which says it does not want files that have reached a backup repository.
Checked history and these have been out of sync from the very beginning!
2015-06-16 17:10:10 -04:00
Joey Hess
8c46ea22c2 Added new "anything" preferred content expression, which matches all versions of all files. 2015-06-16 17:03:34 -04:00
Joey Hess
29c03145e6 sync: Add support for --all and --unused. 2015-06-16 16:50:03 -04:00
Joey Hess
67f7f1b1cb info: Added json output for "backend usage", "numcopies stats", "repositories containing these files", and "transfers in progress". 2015-06-16 13:50:28 -04:00
Joey Hess
c96b333869 clarify 2015-06-15 15:30:00 -04:00
Joey Hess
9b38c14165 debian/cabal-wrapper: Removed this hack which should not be needed anymore. 2015-06-14 14:43:55 -04:00
Joey Hess
829007d629 Improve url parsing to handle some urls containing illegal [] characters in their paths.
Ie, "https://archive.org/download/zoom-2/Zoom - Release 2 (1996)(Active Software)[!].iso"
2015-06-14 13:39:44 -04:00
Joey Hess
0a998032ed Fix bug that prevented enumerating locally present objects in repos tuned with annex.tune.objecthash1=true
Need to walk 1 level of subdirs less in this case.

The git-annex branch traversal code didn't have a similar bug.
2015-06-11 15:15:05 -04:00
Joey Hess
608172fef5 add git-annex-shell link to standalone deb patch 2015-06-10 13:58:30 -04:00
Joey Hess
e0c2bcdc74 Build documentation with deterministic=1 for reproducible builds. (A new ikiwiki feature.) Closes: #785736 2015-06-09 17:53:55 -04:00
Joey Hess
800e9f2658 tahoe: Use ~/.tahoe-git-annex/ rather than ~/.tahoe/git-annex/ to avoid old versions of tahoe create-client choking. 2015-06-09 15:29:16 -04:00
Joey Hess
e4f22fa721 close debian bug too 2015-06-09 15:15:12 -04:00
Joey Hess
f9bda65245 clarify 2015-06-09 15:08:13 -04:00
Joey Hess
f8ab3bc449 dead --key: Can be used to mark a key as dead. 2015-06-09 14:52:05 -04:00
Joey Hess
6eefc5db65 fsck: Ignore keys that are known to be dead when running in --all mode or a in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead. 2015-06-09 14:08:57 -04:00
Joey Hess
2989cdfe3e re-remove hamlet dep 2015-06-06 14:13:46 -04:00
Joey Hess
5f0f063a7a S3: Publically accessible buckets can be used without creds. 2015-06-05 16:23:35 -04:00
Joey Hess
4acd28bf21 public=yes config to send AclPublicRead
In my tests, this has to be set when uploading a file to the bucket
and then the file can be accessed using the bucketname.s3.amazonaws.com
url.

Setting it when creating the bucket didn't seem to make the whole bucket
public, or allow accessing files stored in it. But I have gone ahead and
also sent it when creating the bucket just in case that is needed in some
case.
2015-06-05 14:38:01 -04:00
Joey Hess
de3bd11a2c import --clean-duplicates: Fix bug that didn't count local or trusted repo's copy of a file as one of the necessary copies to allow removing it from the import location. 2015-06-03 13:15:38 -04:00
Joey Hess
d28e8fbfd5 get --incomplete: New option to resume any interrupted downloads. 2015-06-02 14:20:38 -04:00
Joey Hess
00b094442e assistant: Consume systemd-networkd dbus events to learn about changes to network connections, as was already done with network-manager and wicd. Thanks to Sebastian Reuße for the patches. 2015-06-02 12:54:20 -04:00
Joey Hess
a6d54e49a0 sync, remotedaemon: Pass configured ssh-options even when annex.sshcaching is disabled. 2015-05-30 22:01:52 -04:00
Joey Hess
36b9c9ca5f fromkey, registerurl: Improve handling of urls that happen to also be parsable as strange keys. 2015-05-30 02:08:49 -04:00
Joey Hess
0fd4bef75e prep release 2015-05-28 10:49:04 -04:00
Joey Hess
8990d4cc68 fsck: When checksumming a file fails due to a hardware fault, the file is now moved to the bad directory, and the fsck proceeds. Before, the fsck immediately failed. 2015-05-27 16:40:03 -04:00
Joey Hess
67469ce01f Revert removal dependency on obsolete hamlet package, since the autobuilders are not ready for this change yet and it prevented them from building the webapp. Reopens: #786659 2015-05-27 14:49:58 -04:00
Joey Hess
e810564338 Revert "remove deprecated hamlet package"
This reverts commit cf650eaa99.

It's too early to do this; the linux and android autobuilder will need to
be updated to use the new version of shakespeare, and that will require a
complete refresh of them. In the meantime, this has knocked the webapp out
of the autobuilders.
2015-05-27 14:47:41 -04:00
Joey Hess
5e1f1bd252 Linux standalone, OSX app: Improve runshell script to always quote shell vars, so that it will work when eg, untarred into a directory path with spaces in its name. 2015-05-26 15:38:54 -04:00
Joey Hess
6e05764532 changelog for bug closed in last release 2015-05-24 15:45:36 -04:00
Joey Hess
77c43a388e fromkey, registerurl: Allow urls to be specified instead of keys, and generate URL keys.
This is especially useful because the caller doesn't need to generate valid
url keys, which involves some escaping of characters, and may involve
taking a md5sum of the url if it's too long.
2015-05-22 22:41:36 -04:00
Joey Hess
cab2f05b50 lintian fix 2015-05-22 16:06:16 -04:00
Joey Hess
cf650eaa99 remove deprecated hamlet package
shakespeare provides this now
2015-05-22 15:46:47 -04:00
Joey Hess
979064b1cb prep release 2015-05-22 14:20:21 -04:00
Joey Hess
1b01322de0 OSX: Corrected the location of trustedkeys.gpg, so the built-in upgrade code will find it. Fixes OSX upgrade going forward, but older versions won't upgrade themselves due to this problem. 2015-05-22 13:41:47 -04:00
Joey Hess
8e40cb6f56 Build documentation with TZ=UTC for reproducible builds. See #785736. 2015-05-19 15:45:21 -04:00
Joey Hess
ecb0d5c087 use lock pools throughout git-annex
The one exception is in Utility.Daemon. As long as a process only
daemonizes once, which seems reasonable, and as long as it avoids calling
checkDaemon once it's already running as a daemon, the fcntl locking
gotchas won't be a problem there.

Annex.LockFile has it's own separate lock pool layer, which has been
renamed to LockCache. This is a persistent cache of locks that persist
until closed.

This is not quite done; lockContent stil needs to be converted.
2015-05-19 14:09:52 -04:00
Joey Hess
6ec7c4fef4 webapp: Fix zombie xdg-open process left when opening file browser. Closes: #785498 2015-05-17 17:26:19 -04:00
Joey Hess
b284cefccc fix type in the name of --used-refspec in changelog 2015-05-14 15:46:01 -04:00
Joey Hess
823bb8031b add annex.used-refspec 2015-05-14 15:44:08 -04:00
Joey Hess
86699ff861 unused: Add --used option, which can specify a set of refs to consider used, rather than the default of considering all refs used. 2015-05-14 15:31:38 -04:00
Joey Hess
7ebf234616 Stale transfer lock and info files will be cleaned up automatically when get/unused/info commands are run.
Deleting lock files is tricky, tricky stuff. I think I got it right!
2015-05-12 20:11:23 -04:00
Joey Hess
8c2dd7d8ee Fix an unlikely race that could result in two transfers of the same key running at once.
As discussed in bug report.
2015-05-12 19:39:28 -04:00
Joey Hess
61ccf95004 Avoid accumulating transfer failure log files unless the assistant is being used.
Only the assistant uses these, and only the assistant cleans them up, so
make only git annex transferkeys write them,

There is one behavior change from this. If glacier is being used, and a
manual git annex get --from glacier fails because the file isn't available
yet, the assistant will no longer later see that failed transfer file and
retry the get. Hope no-one depended on that old behavior.
2015-05-12 15:53:38 -04:00
Joey Hess
a812d598ef Take space that will be used by running downloads into account when checking annex.diskreserve. 2015-05-12 15:20:22 -04:00
Joey Hess
505d540360 allow building without ascii-progress, since it is not ready yet
No progress bars with -J unless built with ascii-progress.
2015-05-12 13:54:16 -04:00
Joey Hess
e27b97d364 Merge branch 'master' into concurrentprogress
Conflicts:
	Command/Fsck.hs
	Messages.hs
	Remote/Directory.hs
	Remote/Git.hs
	Remote/Helper/Special.hs
	Types/Remote.hs
	debian/changelog
	git-annex.cabal
2015-05-12 13:23:22 -04:00
Joey Hess
96d24a8161 note about git annex drop behavior change in bare repo 2015-05-12 13:02:22 -04:00
Joey Hess
4b2e3210ea drop: Now supports --all, --unused, and --key. 2015-05-12 13:00:06 -04:00
Joey Hess
db5d831d07 import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss. 2015-05-11 12:57:47 -04:00
Joey Hess
6ebf615995 cutting a point release, will be a cabal-only update 2015-05-11 12:09:02 -04:00
Yaroslav Halchenko
cb23224ba3 ENH: make debianstandalone{,-dsc} rules
Moved rules out of debian/rules and avoided need for patching it for
standalone builds
2015-05-11 11:50:59 -04:00
Joey Hess
26ac0753c1 more FlexibleContexts 2015-05-10 15:54:58 -04:00
Yaroslav Halchenko
e861dd1dd5 ENH: leave only git for depends of standalone -- the rest is provided
Otherwise, versioned depends forbid installation on elderly squeeze
2015-05-10 14:36:50 -04:00
Joey Hess
6a1814abf5 seems that webapp ssh password ssh-agent hack is broken on windows after recent ssh changes 2015-05-08 15:07:23 -04:00
Joey Hess
f1a81cfce2 prep release 2015-05-08 13:42:45 -04:00
Joey Hess
f395ca7a41 Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. 2015-05-08 13:39:00 -04:00
Joey Hess
54bee7dad4 Revert "Android: Updated bundled ssh from 6.1p1 to 6.4p1."
This reverts commit 27836a2af9.

Build didn't actually seem to succeed, for both versions of android, so
I'll need to revisit this later.
2015-05-08 13:05:33 -04:00
Joey Hess
0137656c39 typo 2015-05-07 19:41:05 -04:00
Joey Hess
ee94a79690 Debian's ghc now supports TH on arm! Adjust build dependencies to build the webapp on arm, and enable DAV support on arm. \o/
Also, adjust some other arch specific build dependencies that are now
available on more architectures in Devian unstable.
2015-05-07 19:09:44 -04:00
Joey Hess
4ba4155067 remove workaround for now fixed bug #763078 2015-05-07 18:54:55 -04:00
Joey Hess
e09816b0e6 reorder 2015-05-07 18:12:08 -04:00
Joey Hess
4aba1c74bd remaining dataenc to sandi conversions
I've tested all the dataenc to sandi conversions except Assistant.XMPP,
and all have unchanged behavior, including behavior on large unicode code
points.
2015-05-07 18:07:13 -04:00
Joey Hess
18fc05ff04 Windows: Roll back to an older version of rsync from cygwin. The newer version has some dependency on a newer ssh from cygwin. 2015-05-07 15:56:40 -04:00
Joey Hess
ce285ea6fa Windows: Remove cygwin ssh, the newer version of which has stopped honoring the setting of HOME. Instead, copy msysgit's ssh into PATH. 2015-05-07 15:53:03 -04:00
Joey Hess
469242ac4d fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes: #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only) 2015-05-06 14:45:20 -04:00
Joey Hess
eeb0359a2e contentlocation, examinekey, lookupkey: Added --batch mode option. 2015-05-06 13:44:53 -04:00
Joey Hess
cf786f42a4 Support checking ftp urls for file presence. 2015-05-05 14:05:02 -04:00
Joey Hess
0b18228516 Work around wget bug #784348 which could cause it to clobber git-annex symlinks when downloading from ftp. 2015-05-05 13:53:06 -04:00
Joey Hess
0e6d3a9041 add missing env var setting for standalone build 2015-05-05 13:50:37 -04:00
Yaroslav Halchenko
d15f2a1273 BF: Adjusted debian/patches/standalone-build for recent stylistic changes 2015-05-03 12:02:30 -04:00
Joey Hess
27836a2af9 Android: Updated bundled ssh from 6.1p1 to 6.4p1.
Tested it builds, but have not run it yet.
2015-05-01 15:33:29 -04:00
Joey Hess
437d9db41d assistant: Added --autostop to complement --autostart. 2015-05-01 13:53:45 -04:00
Joey Hess
efb37e7c78 Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end. 2015-04-30 15:28:17 -04:00
Joey Hess
ac6b492711 import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist. 2015-04-30 14:04:36 -04:00
Joey Hess
995643665a Improve integration with KDE's file manager to work with dolphin version 14.12.3 while still being compatable with 4.14.2. Thanks, silvio. 2015-04-29 15:53:16 -04:00
Joey Hess
d8ad1d5503 import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree. 2015-04-29 13:56:41 -04:00
Joey Hess
2e54251c18 import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.) 2015-04-29 13:46:12 -04:00
Joey Hess
986e245ac1 changelog 2015-04-27 18:10:56 -04:00
Joey Hess
ee78958798 S3: Fix incompatability with bucket names used by hS3; the aws library cannot handle upper-case bucket names. git-annex now converts them to lower case automatically.
For example, it failed to get files from a bucket named S3.

Also fixes `git annex initremote UPPERCASE type=S3`, which failed with the
new aws library, with a signing error message.
2015-04-27 18:00:58 -04:00
Joey Hess
cfbeb1e7b7 Fix bogus failure of fsck --fast. 2015-04-27 17:40:21 -04:00
Joey Hess
22a4e92df7 S3: git annex enableremote will not create a bucket name, which failed since the bucket already exists. 2015-04-23 14:16:53 -04:00
Joey Hess
b3eccec68c S3: git annex info will show additional information about a S3 remote (endpoint, port, storage class) 2015-04-23 14:12:25 -04:00
Joey Hess
8f010bb55b also drop old dbus compat 2015-04-22 16:23:41 -04:00
Joey Hess
eb8ef44133 Dropped support for older versions of yesod and warp than the ones in Debian Jessie.
466 lines of compat cruft deleted!
2015-04-22 16:19:11 -04:00
Joey Hess
a15e1158c6 use BuildVersion in debian, which fixes windows build 2015-04-21 16:42:54 -04:00
Joey Hess
3b3aaf0d56 S3: Enable debug logging when annex.debug or --debug is set.
To debug a bug report, but generally useful.
2015-04-21 15:55:42 -04:00
Joey Hess
e651b29e90 reuse Build.Version to generate version 2015-04-20 15:56:28 -04:00
Joey Hess
b0ebb232ec style changes 2015-04-20 14:55:30 -04:00
Joey Hess
bf0e860617 changelog for https://github.com/joeyh/git-annex/pull/41 2015-04-20 14:53:14 -04:00