Commit graph

699 commits

Author SHA1 Message Date
Joey Hess
a22eaaae27 comment 2015-02-09 14:16:42 -04:00
Joey Hess
69a9c98e71 glacier: Detect when the glacier command in PATH is the wrong one, from boto, rather than from glacier-cli, and refuse to use it, since the boto program fails to fail when passed parameters it does not understand. 2015-02-06 14:39:27 -04:00
Joey Hess
1af8107fec windows build fix 2015-01-29 13:46:57 -04:00
Joey Hess
e0187d5d12 test suite found a problem with today's work
". def" did not do what I thought it would, at all.
2015-01-28 18:05:08 -04:00
Joey Hess
009bd050c1 implement annex.tune.objecthashlower
Split out Annex.DirHashes which never really belonged in Locations.
2015-01-28 16:52:08 -04:00
Joey Hess
e8c376e0ad import Data.Default in Common 2015-01-28 16:11:28 -04:00
Joey Hess
0fd5f257d0 groundwork for parameterizing hash depth 2015-01-28 15:55:17 -04:00
Joey Hess
32fac4b71b remove unnecessary use of MissingH 2015-01-21 13:36:48 -04:00
Joey Hess
afc5153157 update my email address and homepage url 2015-01-21 12:50:09 -04:00
Joey Hess
4f657aa14e add getFileSize, which can get the real size of a large file on Windows
Avoid using fileSize which maxes out at just 2 gb on Windows.
Instead, use hFileSize, which doesn't have a bounded size.
Fixes support for files > 2 gb on Windows.

Note that the InodeCache code only needs to compare a file size,
so it doesn't matter it the file size wraps. So it has been
left as-is. This was necessary both to avoid invalidating existing inode
caches, and because the code passed FileStatus around and would have become
more expensive if it called getFileSize.

This commit was sponsored by Christian Dietrich.
2015-01-20 17:09:24 -04:00
Joey Hess
534c29deae implemented old Richih wishlist about remote/uuid info
* info: Can now display info about a given uuid.
  * Added to remote/uuid info: Count of the number of keys present
    on the remote, and their size. This is rather expensive to calculate,
    so comes last and --fast will disable it.
  * Git remote info now includes the date of the last sync with the remote.
2015-01-13 18:13:14 -04:00
Joey Hess
3bab5dfb1d revert parentDir change
Reverts 965e106f24

Unfortunately, this caused breakage on Windows, and possibly elsewhere,
because parentDir and takeDirectory do not behave the same when there is a
trailing directory separator.
2015-01-09 13:11:56 -04:00
Joey Hess
965e106f24 made parentDir return a Maybe FilePath; removed most uses of it
parentDir is less safe than takeDirectory, especially when working
with relative FilePaths. It's really only useful in loops that
want to terminate at /

This commit was sponsored by Audric SCHILTKNECHT.
2015-01-06 18:55:56 -04:00
Joey Hess
6b3d0cb11a bittorrent: Fix locking problem when using addurl file://
Fixes:
/home/joey/tmp/xxx/.git/annex/misctmp/torrent18347: openFile: resource busy (file is locked)
2014-12-30 13:07:20 -04:00
Joey Hess
c9a3e80d32 fixed all remaining build warnings on Windows 2014-12-29 17:30:20 -04:00
Joey Hess
27fb7e514d Fix build with -f-S3. 2014-12-19 16:53:25 -04:00
Joey Hess
ef12386924 When possible, build with the haskell torrent library for parsing torrent files. 2014-12-18 14:26:10 -04:00
Joey Hess
e2214f6ac8 remove default untrusted hack for bittorrent
This is better handled by checkPresent always failing.
2014-12-17 15:38:00 -04:00
Joey Hess
e08fa65131 note about http://hackage.haskell.org/package/torrent 2014-12-17 15:34:38 -04:00
Joey Hess
6ca54c521d make checkkey always fail for torrents
See comment.
2014-12-17 14:54:54 -04:00
Joey Hess
2192c54877 more robust fallback when a file is available from multiple torrents and some torrent files cannot be downloaded 2014-12-17 14:38:04 -04:00
Joey Hess
bf9df3fc7e fix fencepost error and aria resume after partial download of multi-file torrent 2014-12-17 14:21:48 -04:00
Joey Hess
3a7d0be120 remove excess directory 2014-12-17 14:17:19 -04:00
Joey Hess
d5cbbe1b9a fix torrentUrlNum when there is no #n 2014-12-17 14:07:05 -04:00
Joey Hess
7e422269a6 move dummy uuids to Annex.UUID 2014-12-17 13:57:52 -04:00
Joey Hess
af05ac3ec2 add aria2 progress parsing 2014-12-17 13:40:04 -04:00
Joey Hess
a7690de016 Added bittorrent special remote
addurl behavior change: When downloading an url ending in .torrent,
it will download files from bittorrent, instead of the old behavior
of adding the torrent file to the repository.

Added Recommends on aria2 and bittornado | bittorrent.

This commit was sponsored by Asbjørn Sloth Tønnesen.
2014-12-16 23:22:46 -04:00
Joey Hess
65bce2c80d reformat 2014-12-16 15:26:13 -04:00
Joey Hess
67c05daf5e sanitize filepaths provided by checkUrl 2014-12-11 20:08:49 -04:00
Joey Hess
8a17bcb0be simplify external special remote implementation 2014-12-11 17:44:27 -04:00
Joey Hess
bce7e0dd96 use subdir for addurl when it creates multiple files
The --file parameter specifies the subdir in this mode.
2014-12-11 16:09:56 -04:00
Joey Hess
2cd84fcc8b Expand checkurl to support recommended filename, and multi-file-urls
This commit was sponsored by an anonymous bitcoiner.
2014-12-11 15:33:42 -04:00
Joey Hess
7ae16bb6f7 Revert "let url claims optionally include a suggested filename"
This reverts commit 85df9c30e9.

Putting filename in the claim was a bad idea.
2014-12-11 14:09:57 -04:00
Joey Hess
85df9c30e9 let url claims optionally include a suggested filename 2014-12-11 12:47:57 -04:00
Joey Hess
aafb121068 unmangled mangled urls from the log before passing to external special remote 2014-12-08 19:27:40 -04:00
Joey Hess
30bf112185 Urls can now be claimed by remotes. This will allow creating, for example, a external special remote that handles magnet: and *.torrent urls. 2014-12-08 19:15:07 -04:00
Joey Hess
ee27298b91 implement CLAIMURL for external special remote 2014-12-08 13:57:13 -04:00
Joey Hess
cb6e16947d add stub claimUrl 2014-12-08 13:40:15 -04:00
Joey Hess
8093008ef4 External special remote protocol now includes commands for setting and getting the urls associated with a key. 2014-12-08 13:32:46 -04:00
Joey Hess
911ba8d972 Merge branch 's3-aws' 2014-12-03 14:10:52 -04:00
Joey Hess
55fa1789dd Don't show "(gpg)" when decrypting the remote encryption cipher, since this could be taken to read that's the only time git-annex runs gpg, which is not the case. 2014-12-02 13:50:45 -04:00
Joey Hess
0a891fcfc5 support S3 front-end used by globalways.net
This threw an unusual exception w/o an error message when probing to see if
the bucket exists yet. So rather than relying on tryS3, catch all
exceptions.

This does mean that it might get an exception for some transient network
error, think this means the bucket DNE yet, and try to create it, and then
fail when it already exists.
2014-11-05 12:42:12 -04:00
Joey Hess
93feefae05 Revert "work around minimum part size problem"
This reverts commit a42022d8ff.

I misunderstood the cause of the problem.
2014-11-04 16:21:55 -04:00
Joey Hess
a42022d8ff work around minimum part size problem
When uploading the last part of a file, which was 640229 bytes, S3 rejected
that part: "Your proposed upload is smaller than the minimum allowed size"

I don't know what the minimum is, but the fix is just to include the last
part into the previous part. Since this can result in a part that's
double-sized, use half-sized parts normally.
2014-11-04 16:06:13 -04:00
Joey Hess
ad2125e24a fix a couple type errors and the progress bar 2014-11-04 15:39:48 -04:00
Joey Hess
fccdd61eec fix memory leak
Unfortunately, I don't fully understand why it was leaking using the old
method of a lazy bytestring. I just know that it was leaking, despite
neither hGetUntilMetered nor byteStringPopper seeming to leak by
themselves.

The new method avoids the lazy bytestring, and simply reads chunks from the
handle and streams them out to the http socket.
2014-11-04 15:22:08 -04:00
Joey Hess
29871e320c combine 2 checks 2014-11-04 14:47:18 -04:00
Joey Hess
0f78f197eb casts; now fully working.. but still leaking
Still seems to buffer the whole partsize in memory, but I'm pretty sure my
code is not what's doing it. See https://github.com/aristidb/aws/issues/142
2014-11-03 21:12:15 -04:00
Joey Hess
f0551578d6 this should avoid leaking memory 2014-11-03 20:49:30 -04:00
Joey Hess
4230b56b79 logic error 2014-11-03 20:15:33 -04:00