Commit graph

2100 commits

Author SHA1 Message Date
Joey Hess
e97fce35a6
Display progress meter in -J mode when downloading from the web.
Including in addurl, and get --from web, but also in S3 and External
special remotes when a web url is known for content in those remotes.
2015-11-16 21:00:54 -04:00
Joey Hess
7943442dff
Display progress meter in -J mode when copying from a local git repo, to a local git repo, and from a remote git repo.
Had everything available, just didn't combine the progress meter with the
other places progress is sent to update it. (And to a remote repo already
did show progress.)

Most special remotes should already display progress meters with -J,
same as without it. One exception to this is the web, since it relies on
wget/curl progress display without -J. Still todo..
2015-11-16 19:32:30 -04:00
Joey Hess
9f7e4ed811
Build with -j1 again to get reproducible build.
This was in the cabal file earlier, and was removed because it broke the
android cross build. Moving to the git-annex target of the Makefile
will make it be used for Debian packages etc but not android cross builds
or make fast or when users build with cabal.
2015-11-16 16:51:24 -04:00
Joey Hess
2864dc6d15
typo 2015-11-16 15:53:30 -04:00
Joey Hess
b5df6c7b93
prep release 2015-11-16 14:18:01 -04:00
Joey Hess
6fc01da184
wording 2015-11-16 11:39:45 -04:00
Joey Hess
bb86eebfbd
init: Automatically enable annex.pidlock when necessary. 2015-11-13 13:35:29 -04:00
Joey Hess
aa4192aea6
pid locking configuration and abstraction layer for git-annex
(not actually used anywhere yet)
2015-11-12 17:50:34 -04:00
Joey Hess
7c741302cc
assistant: Pass ssh-options through 3 more git pull/push calls that were missed before.
It was used for regular pull, but not for regular push, tagged push, or the
fallback fetching.
2015-11-10 16:52:30 -04:00
Joey Hess
4b02af57b6
display a message in the unlikely scenario of fsking a dead repository 2015-11-10 14:44:58 -04:00
Joey Hess
cd7929034a
fsck: When fscking a dead repo, avoid incorrect "fixing location log" message.
keyLocations doesn't return locations in dead repos, but if we're fscking a
dead repo, we want to look at what locations are actually logged for it.
2015-11-10 13:59:04 -04:00
Joey Hess
361d31fde7
Make git-annex-standalone.deb include the git-annex html documentation, desktop file, and base completion file, same as the regular git-annex.deb.
It already had a doc-base file relating to the html documentation, and
there's no reason not to include the other stuff.
2015-11-10 13:21:05 -04:00
Joey Hess
d9d87fdbc9
Make the git-annex-standalone.deb prevent runshell from installing wrappers into $HOME/.ssh 2015-11-10 12:53:46 -04:00
Joey Hess
3e49f9fcab
runshell: Avoid failing when $HOME/.ssh does not exist and cannot be created. 2015-11-10 11:42:06 -04:00
Joey Hess
08bb3b1b1d
quvi may output utf-8 encoded data when the conifigured locale doesn't support that; avoid crashing on such invalid encoding. 2015-11-09 12:19:23 -04:00
Joey Hess
7938b87864
add: Fix error recovery rollback to not move the injested file content out of the annex back to the file, because other files may point to that same content. Instead, copy the injected file content out to recover.
That was not a data loss, but it came close!
2015-11-06 15:28:20 -04:00
Joey Hess
51e60259e1
fix replaceFile makeAnnexLink race
replaceFile created a temp file, which was guaranteed to not overlap with
another temp file. However, makeAnnexLink then deleted that file, in
preparation for making the symlink in its place. This caused a race, since
some other replaceFile could create a temp file, using the same name!

I was able to reproduce the race easily running git-annex add -J10 in a
directory with 100 files (all with different contents). Some files would
get ingested into the annex, but their annex links would fail to be added.

There could be other situations where this same problem could occur.
Perhaps when the assistant is adding a file, if the user manually also ran
git-annex add. Perhaps in cases not involving adding a file.

The new replaceFile makes a temprary directory, which is guaranteed to be
unique, and doesn't make a temp file in there. makeAnnexLink can thus
create the symlink without problem and the race is avoided.

Audited all calls to replaceFile to make sure that the old behavior of
providing an empty temp file was not relied on.

The general problem of asking for a temp file and deleting it as part of
the process of using it could reach beyond replaceFile. Did some quick
audits and didn't find other cases of it. Probably only symlink creation
stuff would tend to make that mistake, mostly.
2015-11-06 15:08:19 -04:00
Joey Hess
416c7656a1
Concurrent progress bars are now displayed when using -J with a command that moves file contents around. 2015-11-06 13:44:57 -04:00
Joey Hess
7dc90f2225
import: Avoid very ugly error messages when the directory files are imported to is not a directort, but perhaps an annexed file. 2015-11-05 18:46:05 -04:00
Joey Hess
5db7d435e7
-J for add/addurl/import 2015-11-05 18:24:15 -04:00
Joey Hess
c4d45ef83d
drop -Jn 2015-11-04 17:13:20 -04:00
Joey Hess
3d0f41518d
parallel fsck (yes, these changes are all it takes now!) 2015-11-04 16:28:14 -04:00
Joey Hess
4fd03ccd7b
concurrent-output, first pass
Output without -Jn should be unchanged from before. With -Jn,
concurrent-output is used for messages, but regions are not used yet, so
it's a mess.
2015-11-04 13:45:34 -04:00
Joey Hess
61987c6983
point release for cabal file fixes
Seems that some changes to the cabal file a few months ago resulted in a
git-annex that broke stackage infrastructure.
2015-11-03 12:10:01 -04:00
Joey Hess
628429f1b0
When cabal install is run with the desktop file location not writable, display a warning, but continue successfully.
Installing the desktop file etc is a niceity of git-annex's cabal install,
but not a requirement.

closes https://github.com/fpco/stackage/issues/726
2015-11-03 10:47:33 -04:00
Joey Hess
0717ad4440
Avoid installing desktop file and program file if cabal install git-annex is run as root, since that is not a systemwide install, but to /root, and so generating a systemwide desktop file is not right. 2015-11-03 10:38:08 -04:00
Joey Hess
29d250923b
prep release 2015-11-02 12:41:33 -04:00
Joey Hess
4153507864
Fix failure to build with aws-0.13.0 and finish nearline support.
* Fix failure to build with aws-0.13.0.
* When built with aws-0.13.0, the S3 special remote can be used to create
  google nearline buckets, by setting storageclass=NEARLINE.
2015-11-02 11:14:03 -04:00
Joey Hess
ef5496b8de
Catch up with current git behavior when both repo and repo.git exist; it seems it now prefers repo in this case, although historically it may have preferred repo.git. 2015-10-26 15:35:55 -04:00
Joey Hess
640dba43b6
enableremote: List uuids and descriptions of remotes that can be enabled, and accept either the uuid or the description in leu if the name. 2015-10-26 14:55:40 -04:00
Henrik Ahlgren
87a00e3608
Changelog typo: remove, not remote 2015-10-21 10:10:55 -04:00
Joey Hess
268800d590
Symlink timestamp preservation code uses functions from unix-2.7.0 when available, which should be more portable. 2015-10-21 02:22:18 -04:00
Joey Hess
b9c6a56b0e
Use statvfs on OSX.
Fixes a recent-ish build warning on about 64 bit vs non.

This is the method used by the disk-free-space library, and I tested it to
yield the same results on even 10 tb drives on OSX -- so it's getting 64
bit values.
2015-10-19 17:09:06 -04:00
Joey Hess
b1c0b7175f
Debian: Remote menu file, since a desktop file is provided and lintian says there can be only one. 2015-10-19 14:23:48 -04:00
Joey Hess
4cc892f938
prep release 2015-10-19 14:08:14 -04:00
Joey Hess
806819be57
Avoid displaying network transport warning when a ssh remote does not yet have an annex.uuid set.
Instead, only display transport error if the configlist output doesn't
include an annex.uuid line, even an empty one.

A recent change made git-annex init try to get all the remote uuids, and so
the transport error would be displayed by it. It was also displayed when
eg, copying files to a remote that had no uuid yet.
2015-10-15 15:36:54 -04:00
Joey Hess
27eaa6f410
avoid making post-merge-conflict-resolution commit when no conflicts were resolved
sync, merge, assistant: When git merge failed for a reason other than a
conflicted merge, such as a crippled filesystem not allowing particular
characters in filenames, git-annex would make a merge commit that could
omit such files or otherwise be bad. Fixed by aborting the whole merge
process when git merge fails for any reason other than a merge conflict.
2015-10-15 14:22:46 -04:00
Joey Hess
c32a2429ed
S3: Fix support for using https.
Was using the http-only Manager before, not the tls-capable one.
2015-10-15 10:37:06 -04:00
Joey Hess
9e90c033d3
Changed drop ordering when using git annex sync --content or the assistant, to drop from remotes first and from the local repo last. This works better with the behavior changes to drop in many cases. 2015-10-14 12:33:02 -04:00
Joey Hess
f9adb905fc
Avoid unncessary write to the location log when a file is unlocked and then added back with unchanged content.
Implemented with no additional overhead of compares etc.

This is safe to do for presence logs because of their locality of change;
a given repo's presence logs are only ever changed in that repo, or in a
repo that has just been actively changing the content of that repo.

So, we don't need to worry about a split-brain situation where there'd
be disagreement about the location of a key in a repo. And so, it's ok to
not update the timestamp when that's the only change that would be made
due to logging presence info.
2015-10-12 14:46:47 -04:00
Joey Hess
6145f905e0
improve display when lockcontent fails
/dev/null stderr; ssh is still able to display a password prompt
despite this

Show some messages so the user knows it's locking a remote, and
knows if that locking failed.
2015-10-09 17:31:02 -04:00
Joey Hess
6d4f741d5f
cleanup changelog 2015-10-09 16:18:16 -04:00
Joey Hess
cf79dffa4c
improve drop proof code 2015-10-09 11:09:46 -04:00
Joey Hess
5240a9f315
git-annex-shell: Added lockcontent command, to prevent dropping of key's content. 2015-10-08 14:47:46 -04:00
Joey Hess
976110b992 Debian: Remove dependency on transformers library, as it is now included in ghc.
Backporters will need to revert this.
2015-10-07 12:26:48 -04:00
Joey Hess
61c45072e6 Debian: Add torrent library to build-depends as it's packaged now, and stop recommending bittornado | bittorrent. 2015-10-07 09:08:37 -04:00
Joey Hess
1ac79e6fe5 copy --auto was checking the wrong repo's preferred content. (--from was checking what --to should, and vice-versa.) Fixed this bug, which was introduced in version 5.20150727. 2015-10-06 17:29:44 -04:00
Joey Hess
933fef6ae0 Merge branch 'winprocfix' 2015-10-04 15:46:25 -04:00
Joey Hess
06f1f03e7a Ported disk free space checking code to work on Solaris.
On Solaris, using f_bsize provided a value that is apparently much larger
than the real block size. The solaris docs for statvfs say
f_bsize is the "preferred" file system block size, and I guess the
filesystem prefers larger blocks, but uses smaller ones or something.
The docs also say that f_frsize is the "fundamental" block size.

Switched to using f_frsize on Linux and kFreeBSD too, since I guess
f_bsize could in theory vary the same way there too. Assuming that Solaris
is not violating the posix spec, I guess the linux man page for statvfs
is not as well written and I misunderstood it.
2015-10-02 16:31:15 -04:00
Joey Hess
16d7035ef7 Allow building with S3 disabled again. 2015-10-02 12:12:02 -04:00