Commit graph

37891 commits

Author SHA1 Message Date
Joey Hess
45daa931ed
update 2020-09-05 11:36:47 -04:00
Joey Hess
e36bae74da
Exposed annex.forward-retry git config
One reason is, 5 is an arbitrary number so ought to be configurable.

The real reason though, is I wanted to make the man page explain when
forward retry can override annex.retry, and having a config made the
man page easier to write.
2020-09-04 15:16:40 -04:00
Joey Hess
2bb933eb60
import: Retry downloads that fail
Also, using the transfer machinery for this makes eg, git-annex info show
in-progress imports, and makes --notify-start/finish work.
2020-09-04 13:54:05 -04:00
Joey Hess
46eb48d7c0
Retry transfers to exporttree=yes remotes same as for other remotes
The comment about noRetry is not well-justified, because transfers to many
remotes cannot be resumed, but retries are still allowed for those.
2020-09-04 13:24:08 -04:00
Joey Hess
1a42b2c5a3
combine retry deciders in better way
This fixes the problem that, if forwardRetry was checked for the first 5
and decided to retry, the 6th would go to configuredRetry which would
see the counter was 6 and so wait retry-delay*2^5 seconds (default 32).

Now, it waits for retry-delay before each retry, even when forwardRetry
initiated the retry.
2020-09-04 12:48:30 -04:00
Joey Hess
1d244bafbd
Limit retrying of failed transfers when forward progress is being made to 5
To avoid some unusual edge cases where too much retrying could result in
far more data transfer than makes sense.
2020-09-04 12:46:37 -04:00
Joey Hess
820d4368b3
remove unused isSymLink
That made Utility.FileMode depend on unix, but Utility.Tmp now depends
on it, and is used by Setup, which does not. So it was easiest to remove
this, especially since it's not used.
2020-09-02 14:59:35 -04:00
Joey Hess
376de6fcce
fix build 2020-09-02 14:59:22 -04:00
Joey Hess
22a7f14e7b
better dummy value
no need for empty string when it's going to be thrown away..
2020-09-02 14:55:43 -04:00
Joey Hess
6e9a4f50f3
make viaTmp honor umask
Fixed several cases where files were created without file mode bits that
the umask would usually set. This included exports to the directory special
remote, torrent files used by the bittorrent special remote, hooks written
by git-annex init, and some log files in .git/annex/

Audited all calls, looking for ones that didn't want the umask bits to be
set. All such turned out to already set the specific restrictive file mode
they wanted.
2020-09-02 14:54:07 -04:00
Joey Hess
6361f7c310
make removeAuthorizedKeys robust if the file DNE
Noticed this could potentially crash, although the only thing using it
would normally create the file first, if something then deleted it..
2020-09-02 14:37:42 -04:00
Joey Hess
eed20fe3b7
fix some file modes in calls to withTmpFileIn to honor umask
Also audited for other calls to openTempFile, and all are ok,
except for viaTmp which will need further work.

Remote.Directory fixed to set umask mode when writing to an export,
although it has another one using viaTmp that's not fixed.
Will make exports that are published via a http server running as
another user work, for example.

Remote.BitTorrent fixed to set umask mode when downloading the torrent
file. Normally this does not matter as that file does not hang around
after the download, but if a bittorrent download were started by one user,
got interrupted and then another user ran it, this will let them access
the torrent file created by the first user.
2020-09-02 14:36:08 -04:00
Joey Hess
5a9f518a42
Merge branch 'master' of ssh://git-annex.branchable.com into master 2020-09-02 12:25:52 -04:00
Joey Hess
6781be2141
bug report 2020-09-02 12:25:27 -04:00
Joey Hess
26724fb331
display actual download errors
Eg, when config prohibits accessing localhost, need to show that
message, not a generic "download failed".
2020-09-02 12:21:10 -04:00
Joey Hess
31e5785bf7
avoid multiple download failed messages when learning
Also only display one progress meter for all download attempts, to avoid
a bunch of blank lines.
2020-09-02 12:01:50 -04:00
kyle
94c4da582d fix example: meant to prune --force and include rest of output 2020-09-02 15:51:49 +00:00
Ilya_Shlyakhter
1a6a73eeac Added a comment: verifying source contents 2020-09-02 15:35:22 +00:00
Joey Hess
00937c4813
when downloading same content from multiple urls, only display error if all fail 2020-09-02 11:35:07 -04:00
Joey Hess
854cd2ad47
httpalso: support exporttree=yes
Also tested what happens if the other special remote has importtree=yes
and exporttree=yes, and in that case, download via httpalso works too,
without needing to implement any importtree methods here.

It might be possible to make it automatically set exporttree=yes if the
--sameas does. Didn't try, will probably be layering issues.

Or perhaps it should be inherited by sameas like some
other configs? But then, wouldn't it also make sense to inherit
importree=yes? But as shown here, it's not needed by this kind of
remote.
2020-09-02 11:26:00 -04:00
Joey Hess
8656afd3e1
rename http special remote to httpalso
"http" was too generic and easy to confuse with web. The new name makes
clear it's used in addition to some other remote. And other protocols
can use the same naming scheme.
2020-09-02 10:41:53 -04:00
kyle
2225c69fa2 content verification before transfer to special remote 2020-09-02 14:02:37 +00:00
hans.ekbrand@aabd6f2f1f93eab77ef85618d24892ebafc6e20f
6c2db1415a 2020-09-02 13:11:03 +00:00
hans.ekbrand@aabd6f2f1f93eab77ef85618d24892ebafc6e20f
5384b70bde 2020-09-02 13:10:31 +00:00
hans.ekbrand@aabd6f2f1f93eab77ef85618d24892ebafc6e20f
b16725a377 2020-09-02 13:09:04 +00:00
yarikoptic
03cea49042 Added a comment 2020-09-02 02:53:23 +00:00
Joey Hess
a47787934a
remove Show Cipher
committed by accident, and could have been a security hole, but since
this compiles, it was not
2020-09-01 18:11:22 -04:00
yarikoptic
871257ee23 Added a comment 2020-09-01 20:50:02 +00:00
Joey Hess
20b06266d1
thought 2020-09-01 16:04:40 -04:00
Joey Hess
48dde6d5b0
link 2020-09-01 16:02:03 -04:00
Joey Hess
4bcfd56902
todo 2020-09-01 16:00:49 -04:00
Joey Hess
955f309cd5
comment 2020-09-01 15:51:41 -04:00
Joey Hess
d80876920f
Merge branch 'master' of ssh://git-annex.branchable.com into master 2020-09-01 15:36:56 -04:00
Joey Hess
5b177317b4
comment 2020-09-01 15:36:28 -04:00
yarikoptic
a70260160f noise about 500s 2020-09-01 19:29:29 +00:00
Joey Hess
dc45fc39e8
Merge branch 'master' of ssh://git-annex.branchable.com into master 2020-09-01 15:18:42 -04:00
Joey Hess
571ec900ac
Added http special remote, which is useful for accessing other remotes that publish content stored in them via http/https.
With automatic layout learning!
2020-09-01 15:16:35 -04:00
Joey Hess
fccc9ab442
note common need of these two todos 2020-09-01 12:13:41 -04:00
Joey Hess
d91b2b9fe2
close as dup 2020-09-01 11:58:08 -04:00
gb@4a49bb1afcf3d183bba8f07297b0395808768c6c
778e103632 2020-08-31 14:46:21 +00:00
gb@4a49bb1afcf3d183bba8f07297b0395808768c6c
358a20c5a0 2020-08-30 16:50:45 +00:00
falsifian
55c156a533 Update name of attribute in haskellPackages. 2020-08-28 21:52:18 +00:00
Joey Hess
05dbc449f1
move stack file used by i386ancient autobuilder 2020-08-28 15:16:10 -04:00
Joey Hess
41ebed3941
Support git remotes where .git is a file, not a directory
Eg when --separate-git-dir was used, and core.symlinks=false.

This commit was sponsored by Brock Spratlen on Patreon.
2020-08-28 15:08:14 -04:00
Joey Hess
cde3e5eb0c
test: Stop gpg-agent daemons that are started for the test framework's gpg key
They normally shutdown when the GNUPGHOME directory is deleted, but on
NFS they keep the directory from being deleted. And also, this avoids
a number of them piling up while the test suite is running.
2020-08-28 14:28:42 -04:00
Joey Hess
6fb7ecde35
need lts-13.11 after all
the newer one fails at the end of build with an unavailable libc symbol.
2020-08-28 14:07:31 -04:00
Joey Hess
7cefa377d6
further changes to get back to last working build state on OSX
Seems 13.11 was not being used for a while, but 14.27 works.

For some reason, stack was getting confused when I set the magicmime
flag on the command line, but setting it in stack.yaml didn't confuse
it. Ugh.
2020-08-28 13:59:00 -04:00
Joey Hess
be42e04aa8
close 2020-08-28 13:55:54 -04:00
Joey Hess
6bdfeea3b1
close 2020-08-28 13:54:42 -04:00
Joey Hess
5eceef899a
close 2020-08-28 13:53:15 -04:00