Commit graph

38647 commits

Author SHA1 Message Date
Joey Hess
74c1e0660b
propagate git-annex -c on to transferrer child process
git -c was already propagated via environment, but need this for
consistency.

Also, notice it does not use gitAnnexChildProcess to run the
transferrer. So nothing is done about avoid it taking the
pid lock. It's possible that the caller is already doing something that
took the pid lock, and if so, the transferrer will certianly fail,
since it needs to take the pid lock too. This may prevent combining
annex.stalldetection with annex.pidlock, but I have not verified it's
really a problem. If it was, it seems git-annex would have to take
the pid lock when starting a transferrer, and hold it until shutdown,
or would need to take pid lock when starting to use a transferrer,
and hold it until done with a transfer and then drop it. The latter
would require starting the transferrer with pid locking disabled for the
child process, so assumes that the transferrer does not do anyting that
needs locking when not running a transfer.
2020-12-15 11:36:25 -04:00
Joey Hess
00526a6739
pass along -c options to child git-annex processes 2020-12-15 10:49:29 -04:00
Joey Hess
87de360e98
populate new field 2020-12-15 10:37:07 -04:00
Joey Hess
ed68a2166d
importfeed: Avoid using youtube-dl when a feed does not contain an enclosure, but only a link to an url which youtube-dl does not support
This is common in some feeds, which might mix some items with enclosures,
with others that link to posts or whatever. Before this, it would try to
use youtube-dl and fail, or if youtube-dl was not allowed, it would
incorrectly complain that an url was supported by youtube-dl.
2020-12-15 01:13:21 -04:00
Joey Hess
16315b7812
typo 2020-12-14 21:35:31 -04:00
Joey Hess
5e094d02d6
avoid using MatchingKey where MatchingFile can be used now
This is actually matching worktree files, and now that a Key can be
provided along with the file when doing that, using MatchingFile
reflects that.
2020-12-14 17:54:25 -04:00
Joey Hess
9de5506f19
improve readability and fix a warning 2020-12-14 17:48:30 -04:00
Joey Hess
8b248d84fb
comment 2020-12-14 17:43:48 -04:00
Joey Hess
9244874304
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-14 17:42:25 -04:00
Joey Hess
01527b21d8
add key to FileInfo
MatchingKey is not the thing to use when matching on actual worktreee
files.

Fix reversion in 8.20201116 that made include= and exclude= in
preferred/required content expressions match a path relative to the current
directory, rather than the path from the top of the repository.
2020-12-14 17:42:02 -04:00
yarikoptic
65ab8a2163 Added a comment 2020-12-14 20:59:06 +00:00
Joey Hess
205a837e8a
clarify comment 2020-12-14 16:52:53 -04:00
Joey Hess
fac5ccb2e2
add USE_SYSTEM_LIBS=1 for osxapp build 2020-12-14 15:34:27 -04:00
Joey Hess
9da06e1f75
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-14 14:59:03 -04:00
Joey Hess
75acf5f440
improve some edge cases around partial initialization
* Guard against running in a repo where annex.uuid is set but
  annex.version is set, or vice-versa.
* Avoid autoinit when a repo does not have annex.version or annex.uuid
  set, but has a git-annex objects directory, suggesting it was used
  by git-annex before.
2020-12-14 13:17:43 -04:00
Joey Hess
230e1c88a9
improve display 2020-12-14 13:13:53 -04:00
Joey Hess
19e26f091d
rename and refactor 2020-12-14 12:32:21 -04:00
yarikoptic
ef55021fe1 Added a comment 2020-12-14 14:07:47 +00:00
jkniiv
ec1202871d 2020-12-14 05:07:20 +00:00
Joey Hess
9aaab02e44
add 2020-12-13 18:50:35 -04:00
adina.wagner@2a4cac6443aada2bd2a329b8a33f4a7b87cc8eff
5a4aa4ae0c Added a comment 2020-12-12 20:32:05 +00:00
muks@6bded6d879d80c43cdf4893fabb785876eea0a7b
0e8cf6a1e9 Added a comment 2020-12-12 10:35:07 +00:00
Joey Hess
5c7392b942
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-11 18:31:09 -04:00
Joey Hess
0d0f6d9c23
fix stall detection to actually work when fully stalled
When fully stalled, the progress bar doesn't update, so waiting on a
MVar would block forever. There's no need to wait anyway, just wake up
after sleeping the configured period and check the current value.

Luckily Viasat makes it really easy for me to notice this kind of
mistake, by stalling long TCP connections frequently.
2020-12-11 18:28:46 -04:00
Joey Hess
3e608dfc13
mention annex.stalldetection and don't recommend too infrequent progress updates 2020-12-11 18:26:24 -04:00
kyle
c3e0856b45 Added a comment 2020-12-11 20:56:04 +00:00
Joey Hess
3c76a31b15
response and related todo 2020-12-11 16:21:16 -04:00
Joey Hess
4a8723246d
avoid transferrer committing the git-annex branch on shutdown
The parent is will do it when it shuts down, and having both of them
trying to do it at the same time seems like something good to avoid.
2020-12-11 16:16:07 -04:00
Joey Hess
e6692b66f1
remove
I need to think about this some more, not clear if it's a todo
item specific to stalldetection at all. Remotes with this behavior
also show no progress when run with -J. And some other remotes don't
update any progress meters at all, eg adb is that way and so are hook
remotes and of course external remotes don't have to send progress info.
2020-12-11 15:49:39 -04:00
Joey Hess
fadf47557f
note 2020-12-11 15:48:03 -04:00
Joey Hess
90eadbce49
close 2020-12-11 15:45:55 -04:00
Joey Hess
d3f78da0ed
propagate signals to the transferrer process group
Done on unix, could not implement it on windows quite.

The signal library gets part of the way needed for windows.
But I had to open https://github.com/pmlodawski/signal/issues/1 because
it lacks raiseSignal.

Also, I don't know what the equivilant of getProcessGroupIDOf is on
windows. And System.Process does not provide a way to send any signal to
a process group except for SIGINT.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-12-11 15:32:00 -04:00
Joey Hess
79c765b727
meant to close this earlier 2020-12-11 14:25:54 -04:00
Joey Hess
e1cf8bd35c
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-11 12:53:14 -04:00
Joey Hess
0c46ee5ce0
simplify transferr protocol 2020-12-11 12:52:22 -04:00
Joey Hess
095cdc7e83
extend transferrer protocol to send progress bar total size updates
New protocol is not back-compat with old one, but it's never been
released so that's ok.
2020-12-11 12:42:28 -04:00
Joey Hess
94b323a8e8
use TotalSize more extensively 2020-12-11 12:10:43 -04:00
Joey Hess
263fd1d459
ugh 2020-12-11 11:51:46 -04:00
Joey Hess
a422a056f2
make getViaTmpFrom no longer update location log
All callers adjusted to update it themselves.

In Command.ReKey, and Command.SetKey, the cleanup action already did,
so it was updating the log twice before.

This fixes a bug when annex.stalldetection is set, as now
Command.Transferrer can skip updating the location log, and let it be
updated by the calling process.
2020-12-11 11:50:13 -04:00
kyle
e0780877d7 Added a comment 2020-12-11 15:29:26 +00:00
Joey Hess
a6ed23b82f
todo 2020-12-11 11:09:19 -04:00
muks@6bded6d879d80c43cdf4893fabb785876eea0a7b
16f2fcd778 Added a comment 2020-12-11 02:18:30 +00:00
muks@6bded6d879d80c43cdf4893fabb785876eea0a7b
cf4a669d86 2020-12-11 01:56:38 +00:00
yarikoptic
e0965b831c Added a comment 2020-12-10 21:55:20 +00:00
Joey Hess
108c7d96dd
comment 2020-12-10 17:52:26 -04:00
Joey Hess
8c0ff6b8cd
Merge branch 'master' of ssh://git-annex.branchable.com 2020-12-10 17:47:22 -04:00
Joey Hess
d3c11eac5d
2 bugs involving new stalldetection feature 2020-12-10 17:46:58 -04:00
yarikoptic
c2071567ca Initial report on export -J6 to S3 failing due to "transfer already in progress" 2020-12-10 21:16:56 +00:00
Joey Hess
cedad7b37d
refactor 2020-12-10 16:33:52 -04:00
Joey Hess
6a11b6fab8
Support special remotes that are configured with importtree=yes but without exporttree=yes
There was no particular reason not to support this, other than maybe a lack
of a use case. One use case would of course be a remote that you want to
avoid overwriting content on. A new use case is the idea of importing from
backups, eg borg, where exporting is not necessarily supported at all.

This commit was sponsored by Brock Spratlen on Patreon.
2020-12-10 13:17:40 -04:00