Joey Hess
f5e09a1dbe
incremental verification for S3
...
Sponsored-by: Dartmouth College's DANDI project
2021-08-18 15:07:00 -04:00
Joey Hess
d154e7022e
incremental verification for web special remote
...
Except when configuration makes curl be used. It did not seem worth
trying to tail the file when curl is downloading.
But when an interrupted download is resumed, it does not read the whole
existing file to hash it. Same reason discussed in
commit 7eb3742e4b76d1d7a487c2c53bf25cda4ee5df43; that could take a long
time with no progress being displayed. And also there's an open http
request, which needs to be consumed; taking a long time to hash the file
might cause it to time out.
Also in passing implemented it for git and external special remotes when
downloading from the web. Several others like S3 are within striking
distance now as well.
Sponsored-by: Dartmouth College's DANDI project
2021-08-18 15:02:22 -04:00
Joey Hess
1dca3ba26a
status update
2021-08-18 12:58:27 -04:00
Joey Hess
d67da1f4db
idea
2021-08-18 12:39:03 -04:00
Joey Hess
4bbc6a25fa
comment
2021-08-17 10:28:18 -04:00
yarikoptic
b80dd29d6a
initial report about needing newer gpg to get tests pass
2021-08-16 21:58:07 +00:00
yarikoptic
08ca0d8961
Added a comment: note on sec= on the mount
2021-08-16 21:38:13 +00:00
Joey Hess
ffa1f6ed30
Merge branch 'master' of ssh://git-annex.branchable.com
2021-08-16 17:30:04 -04:00
Joey Hess
f8463ad52f
status update
2021-08-16 17:29:39 -04:00
Joey Hess
b1622eb932
incremental verify for directory special remote
...
Added fileRetriever', which will let the remaining special remotes
eventually also support incremental verify.
Sponsored-by: Dartmouth College's DANDI project
2021-08-16 16:51:33 -04:00
Joey Hess
ec82299730
status update
...
I was wrong about S3 supporting tailVerify.
2021-08-16 15:15:32 -04:00
yarikoptic
b66308a214
update description that it is isilon under
2021-08-16 17:44:01 +00:00
matthias.risze@9f2c8f7faed4cac1905d1bf1ee4524d708c13688
03ca13f5c5
Added a comment: type=git special remote cannot be enabled, no uuid is generated
2021-08-16 13:02:04 +00:00
https://christian.amsuess.com/chrysn
fa69431266
migrate script: Get full list of remotes that have a file; doc updates; progress output; corner case fixes
2021-08-15 19:01:37 +00:00
https://christian.amsuess.com/chrysn
905fef31b3
Added a comment: Another example
2021-08-15 17:42:54 +00:00
https://christian.amsuess.com/chrysn
29a1274f99
migrate script: Do whereis work before to speed up processing
2021-08-15 11:48:58 +00:00
https://christian.amsuess.com/chrysn
69fc2a22b3
Added a comment: annex-to-annex
2021-08-15 11:47:31 +00:00
https://christian.amsuess.com/chrysn
a5f620a1d9
migrate script: Fix accidentally commented-out fsck run
2021-08-15 11:04:37 +00:00
https://christian.amsuess.com/chrysn
088d6c4cd0
migrate script: Don't try on all remotes, look where it is and drop from there (faster)
2021-08-15 10:37:53 +00:00
spwhitton
9e801dc906
Added a comment: annex-to-annex
2021-08-13 22:08:57 +00:00
https://christian.amsuess.com/chrysn
a59637412c
Convert previously missing attachment
2021-08-13 20:58:57 +00:00
https://christian.amsuess.com/chrysn
cb36451f26
Tool to drop migrated files for good
2021-08-13 20:53:48 +00:00
Joey Hess
e46a7dff6f
fix windows build
2021-08-13 16:36:33 -04:00
Joey Hess
037bf68269
Merge branch 'master' of ssh://git-annex.branchable.com
2021-08-13 16:35:20 -04:00
Joey Hess
751242b55e
status update
2021-08-13 16:34:18 -04:00
jkniiv@b330fc3a602d36a37a67b2a2d99d4bed3bb653cb
41ef5da4e0
the fact that I needed a modification/patch to build mentioned
2021-08-13 03:42:10 +00:00
jkniiv@b330fc3a602d36a37a67b2a2d99d4bed3bb653cb
3dc6c7a9a0
prop_view_roundtrips fails (occasionally)
2021-08-13 03:31:45 +00:00
jkniiv@b330fc3a602d36a37a67b2a2d99d4bed3bb653cb
57884e5442
windows build fails as of 7550ef9a2
2021-08-13 02:17:50 +00:00
Joey Hess
7550ef9a2c
Merge branch 'master' of ssh://git-annex.branchable.com
2021-08-12 14:50:12 -04:00
Joey Hess
51d59fb260
comment
2021-08-12 14:49:48 -04:00
yarikoptic
6318c0f27f
a report on the flood of failing tests on discovery
2021-08-11 20:25:51 +00:00
Joey Hess
2e54564061
Merge branch 'master' of ssh://git-annex.branchable.com
2021-08-11 14:51:05 -04:00
jasonb@ab4484d9961a46440958fa1a528e0fc435599057
285026eb91
Added a comment: I have this behavior consistently on the 2 repos I use
2021-08-11 18:49:41 +00:00
Joey Hess
7eb3742e4b
incremental verify for chunked remotes
...
Simply feed each chunk in turn to the incremental verifier.
When resuming an interrupted retrieve, it does not do incremental
verification. That would need to read the file, up to the resume point,
and feed it to the incremental verifier. That seems easy to get wrong.
Also it would mean extra work done before the transfer can start. Which
would complicate displaying progress, and would perhaps not appear to the
user as if it was resuming from where it left off. Instead, in that
situation, return UnVerified, and let the verification be done in a
separate pass.
Granted, Annex.CopyFile does manage all that, but it's not complicated
by dealing with chunks too.
Sponsored-by: Dartmouth College's DANDI project
2021-08-11 14:42:49 -04:00
Lukey
e134f411d4
Added a comment
2021-08-11 18:25:51 +00:00
gabrielhidasy@c3d26e2c0b3e669d012f06736616088b42ad0dbe
b9a9273a87
2021-08-11 16:29:37 +00:00
Ilya_Shlyakhter
2df44abad8
Added a comment: sorry
2021-08-10 16:28:33 +00:00
Joey Hess
d424f43116
comment
2021-08-09 16:00:57 -04:00
Joey Hess
885bbed2d4
sheeeeeeeeesh
2021-08-09 15:33:59 -04:00
yarikoptic
8ede4b606d
Added a comment
2021-08-09 17:58:55 +00:00
yarikoptic
c7e4af1652
Added a comment
2021-08-09 17:34:39 +00:00
Ilya_Shlyakhter
c4b166aa17
Added a comment: standalone build version vs standard release version
2021-08-09 17:28:31 +00:00
Joey Hess
f54b9f2389
comment
2021-08-09 13:03:19 -04:00
Joey Hess
9d684e4dfa
response
2021-08-09 12:46:10 -04:00
Joey Hess
56fbf57e5f
typo
2021-08-09 12:44:20 -04:00
Joey Hess
5990942b6c
don't use changelog version in commit message
...
changelog may have a new unreleased version open already
2021-08-09 12:31:48 -04:00
Joey Hess
c9b1b7d067
close
2021-08-09 12:31:36 -04:00
alex
1801400bbb
2021-08-09 04:21:11 +00:00
jgsuess@732b8c62c50d8595d7b1d58eea11e5019c2308b1
251c24b388
Added a comment: Automatic watch for the heuristic
2021-08-07 12:25:02 +00:00
yarikoptic
4bebc46ce5
get failing to get if with --debug
2021-08-06 22:11:46 +00:00
