Commit graph

40140 commits

Author SHA1 Message Date
Joey Hess
73ccf34763
closing 2021-06-30 11:47:39 -04:00
yarikoptic
0e7906e802 Added a comment 2021-06-30 14:26:34 +00:00
jwodder
d296fb9693 2021-06-30 13:16:19 +00:00
falsifian
0a9a9c44e7 Added a comment 2021-06-30 00:09:23 +00:00
Atemu
99266d7875 Added a comment 2021-06-29 22:57:39 +00:00
Joey Hess
d9949944d1
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-29 13:30:58 -04:00
yarikoptic
fe32661cf5 Added a comment 2021-06-29 17:27:35 +00:00
Joey Hess
75d29de8d4
avoid using CopyFile
If .git/config symlinks are ever a thing, this will handle them better.

But mostly, this is to avoid git-repair needing to include CopyFile,
which would complicate it unduely.
2021-06-29 13:21:21 -04:00
Joey Hess
6b0d732746
repair: Fix reversion in version 8.20200522 that prevented fetching missing objects from remotes
In commit dfc4e641b5 git repair was changed
to use remote name, not url, when fetching. But it fetches into a temporary
git repo, which doesn't have remotes configured. Oops.

(In my defense, that commit was made just as covid lockdown started. But
testing? Urk.)

Sponsored-by: Mark Reidenbach on Patreon
2021-06-29 13:15:15 -04:00
Joey Hess
199391befe
make repair interruption safe
Fixed bug that interrupting git-annex repair (or assistant) while it was
fixing repository corruption would lose objects that were contained in pack
files.

Unpack all pack files and move objects into place *before* deleting the
pack files. The old approach moved the pack files to a temp directory
before unpacking them, which was not interruption safe.

Sponsored-By: Jochen Bartl on Patreon
2021-06-29 13:14:28 -04:00
Joey Hess
a492553eca
some analysis 2021-06-28 15:00:21 -04:00
Joey Hess
26b0895187
close 2021-06-28 13:14:19 -04:00
Joey Hess
34b958144d
close 2021-06-28 13:11:13 -04:00
Joey Hess
5594d00f8c
close 2021-06-28 13:08:57 -04:00
Joey Hess
ef5fb1a853
close 2021-06-28 13:05:26 -04:00
Joey Hess
d7d9363677
comment 2021-06-28 12:54:41 -04:00
Joey Hess
9cb5ed2446
avoid using cp --reflink=auto on windows
Sponsored-by: Dartmouth College's Datalad project
2021-06-28 12:51:40 -04:00
Joey Hess
b8e32e200e
addurl, importfeed: Added --no-raw option
Forces eg, download with youtube-dl without falling back to raw download.

Since youtube-dl failing due to an url not being supported is difficult to
distinguish from it failing due to being blocked in some way, this can be
useful to avoid the fallback of git-annex downloading the raw web page and
adding that.

Since --raw also prevents using special remotes, --no-raw also
allows special remote downloads. Although it's always possible that some
special remote may claim an url and fall back to raw download of the
content, which --no-raw cannot prevent.

Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-06-27 11:14:51 -04:00
Joey Hess
3a14648142
dropping unused marks as dead
Dropping an object with drop --unused or dropunused will mark it as
dead, preventing fsck --all from complaining about it after it's been
dropped from all repositories.

If another repository still has a copy, it won't be treated as dead
until it's also dropped from there.

The drop has to use --unused, can't be --key or something else, because
this indicates that the user has recently ran git-annex unused. If it
checked the unused log on every drop, bad things would happen when the
unused log was out of date, eg a file used to be unused but then got
re-added. Marking such a file as dead could be confusing. When the user
uses --unused/dropunused, they must consider the unused information to be
up-to-date.

The particular workflow this enables is:

	git annex add foo
	git annex unannex foo
	git annex unused
	git annex drop --unused / dropunused
	git annex fsck --all # no warnings

The docs for git-annex unannex say to use git-annex unused and dropunused,
so the user should be pointed in this direction when they want to undo an
accidental add.

Sponsored-by: Brock Spratlen on Patreon
2021-06-25 15:22:26 -04:00
Joey Hess
b08b9ed210
fix innacuracy
--all does not skip dead keys for all commands;
git-annex fsck checks if a key is dead and skips it.
2021-06-25 14:48:36 -04:00
Joey Hess
0c1df95dd7
comment 2021-06-25 14:15:14 -04:00
Joey Hess
172c7e22e9
comment 2021-06-25 13:52:59 -04:00
Joey Hess
0966bd2de4
comment 2021-06-25 13:33:27 -04:00
Joey Hess
df2001aa88
Improve display of errors when transfers fail
Transfers from or to a local git repo could fail without a reason being
given, if the content failed to verify, or if the object file's stat
changed while it was being copied. Now display messages in these cases.

Sponsored-by: Jack Hill on Patreon
2021-06-25 13:17:04 -04:00
Joey Hess
f5595ea063
comment 2021-06-25 12:13:25 -04:00
Joey Hess
e2cc9bf53d
comment 2021-06-25 12:11:05 -04:00
Joey Hess
f57804e72c
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-25 12:06:49 -04:00
Joey Hess
51c696679f
avoid using temp file size when deciding whether to retry failed transfer
When stall detection is enabled, and a transfer is in progress,
it would display a doubled message:

(transfer already in progress, or unable to take transfer lock) (transfer already in progress, or unable to take transfer lock)

That happened because the forward retry decider had a start size of 0,
and an end size of whatever amount of the object the other process had
downloaded. So it incorrectly thought that the transferrer process had
made progress, when it had in fact immediately given up with that
message.

Instead, use the reported value from the progress meter. If a remote
does not report progress, this will mean it doesn't forward retry, in a
situation where it used to. But most remotes do report progress, and any
remote that does not can be fixed to, by using watchFileSize when
downloading. Also, some remotes might preallocate the temp file (eg
bittorrent), so relying on statting its size at this level to get
progress is dubious.

The same change was made to Annex/Transfer.hs, although only
Annex/TransferrerPool.hs needed to be changed to avoid the duplicate
message.

(An alternate fix would have been to start the retry decider with the
size of the object file before downloading begins, rather than 0.)

Sponsored-by: Brett Eisenberg on Patreon
2021-06-25 12:04:23 -04:00
strmd
3eb68df9d5 Added a comment 2021-06-25 15:07:55 +00:00
Ilya_Shlyakhter
27df3f8e88 added suggestion re: hardening git-annex against interruptions 2021-06-24 18:16:43 +00:00
Ilya_Shlyakhter
7ad95210d1 Added a comment: resolving merge conflicts 2021-06-24 18:03:40 +00:00
Lukey
fb2e418fa2 Added a comment 2021-06-24 17:43:36 +00:00
Ilya_Shlyakhter
879ac0b713 Added a comment: thanks 2021-06-24 16:40:48 +00:00
Joey Hess
847aa88bcb
Merge branch 'master' of ssh://git-annex.branchable.com 2021-06-23 17:31:09 -04:00
Joey Hess
a3a6e64122
comment 2021-06-23 17:30:50 -04:00
Ilya_Shlyakhter
7bff1dc27c Added a comment: recovering from sqlite db corruption 2021-06-23 18:45:47 +00:00
Joey Hess
af64c184f3
comment 2021-06-23 13:19:34 -04:00
Joey Hess
b980d90b05
close 2021-06-23 12:59:00 -04:00
Joey Hess
689d54cd43
comment 2021-06-23 12:54:48 -04:00
Joey Hess
e701f69ac4
comment and close 2021-06-23 12:53:18 -04:00
datamanager
26511c3166 Added a comment: an alternative 2021-06-23 01:29:23 +00:00
Lukey
d2ba1698c9 2021-06-22 18:40:56 +00:00
Ilya_Shlyakhter
0cc3fc6bbf Added a comment: old comments 2021-06-22 14:03:24 +00:00
Joey Hess
0fe550af75
fix windows build 2021-06-22 09:46:06 -04:00
jwodder
4b93e933db 2021-06-22 13:21:17 +00:00
Joey Hess
4b1b9d7a83
Added annex.freezecontent-command and annex.thawcontent-command configs
Freeze first sets the file perms, and then runs
freezecontent-command. Thaw runs thawcontent-command before
restoring file permissions. This is in case the freeze command
prevents changing file perms, as eg setting a file immutable does.
Also, changing file perms tends to mess up previously set ACLs.

git-annex init's probe for crippled filesystem uses them, so if file perms
don't work, but freezecontent-command manages to prevent write to a file,
it won't treat the filesystem as crippled.

When the the filesystem has been probed as crippled, the hooks are not
used, because there seems to be no point then; git-annex won't be relying
on locking annex objects down. Also, this avoids them being run when the
file perms have not been changed, in case they somehow rely on
git-annex's setting of the file perms in order to work.

Sponsored-by: Dartmouth College's Datalad project
2021-06-21 14:40:52 -04:00
Joey Hess
ba62c3467b
remove dead code 2021-06-21 13:54:12 -04:00
Joey Hess
f23ae9a45b
comment 2021-06-21 13:52:50 -04:00
Joey Hess
ea0835eba6
tag datalad
at yoh's request
2021-06-21 13:23:51 -04:00
Joey Hess
c2347d8bfc
comment 2021-06-21 13:21:43 -04:00