Commit graph

42319 commits

Author SHA1 Message Date
Joey Hess
a9355e74a3
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-22 12:59:53 -04:00
Joey Hess
66bd4f80b3
Improved handling of --time-limit when combined with -J
When concurrency is enabled, there can be worker threads still running
when the time limit is checked. Exiting right there does not
give those threads time to finish what they're doing. Instead, the seeking
is wrapped up, and git-annex then shuts down cleanly.

The whole point of --time-limit existing, rather than using timeout(1)
when running git-annex is to let git-annex finish the action(s) it is
working on when the time limit is reached, and shut down cleanly.

I noticed this problem when investigating why restagePointerFile might
not have run after get/drop of an unlocked file. With --time-limit -J,
a worker thread may have finished updating a work tree file, and be killed
by the time limit check before it can run restagePointerFile. So despite
--time-limit running the shutdown actions, the work tree file didn't get
restaged.

Sponsored-by: Dartmouth College's DANDI project
2022-09-22 12:54:52 -04:00
nobodyinperson
75adb198a1 Added a comment 2022-09-22 06:24:05 +00:00
rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3
c44df3fdf5 removed 2022-09-22 05:08:59 +00:00
jgoerzen
63f8b78e29 Added a comment 2022-09-22 03:40:04 +00:00
yarikoptic
79fa55ce88 Added a comment 2022-09-22 01:33:24 +00:00
yarikoptic
f75641f9f6 Added a comment 2022-09-22 01:03:18 +00:00
Gus
28e71d46e4 Added a comment 2022-09-21 22:30:20 +00:00
Joey Hess
6f0566d704
comment 2022-09-21 18:17:21 -04:00
Joey Hess
2731e3ab38
wording 2022-09-21 16:13:08 -04:00
Joey Hess
05592a2ddb
comment 2022-09-21 16:11:10 -04:00
Joey Hess
af9875765c
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-21 15:17:56 -04:00
Joey Hess
34e313f786
annex.diskreserve default increased from 1 mb to 100 mb
It's hard to know what's a good default for this. But 1 mb seems way too
small, because it's very easy for a git pull or some similar operation
that we don't think of as using much space to use up 1 mb of space.

Most people would want to free up some space if a filesystem only had 100
mb free. But on a small VPS, it's probably not uncommon to have only 1 gb
free. So 1 gb is too large for annex.diskreserve.

While old 1 gb USB keys are around, it's unlikely that anyone is
relying on them to shuttle annex data around; it would be worth anyone's
time to upgrade to a 32 gb or larger cheap modern USB key ($5).

Sponsored-by: Kevin Mueller on Patreon
2022-09-21 15:00:13 -04:00
yarikoptic
ad31c9fc19 Added a comment 2022-09-21 18:49:07 +00:00
yarikoptic
65cd69217c Added a comment 2022-09-21 18:46:50 +00:00
Joey Hess
94216c99a7
comment and todo 2022-09-21 14:32:42 -04:00
Joey Hess
d330c83e2c
comment 2022-09-21 13:42:20 -04:00
Joey Hess
2f16ff8cdc
comment 2022-09-21 13:17:21 -04:00
Joey Hess
90da0a5e59
comment 2022-09-21 13:04:47 -04:00
Joey Hess
b072410e06
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-21 10:41:12 -04:00
rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3
ef61e117aa Added a comment: "not inbackend=URL" is failed with parse error 2022-09-21 07:04:55 +00:00
rinomizu5@5ead4c82685c65d7717dbd5591b80425036ae9e3
cae26503f3 Added a comment: "not inbackend=URL" is failed with parse error 2022-09-21 07:04:35 +00:00
yarikoptic
f3a99db14b Added a comment 2022-09-20 22:50:46 +00:00
yarikoptic
f967316f31 initial report on odd "modified" status 2022-09-20 22:22:42 +00:00
nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9
675f9db76e 2022-09-20 22:16:55 +00:00
Gus
bccc441d9c Added a comment 2022-09-20 22:07:30 +00:00
nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9
0602d659af Added a comment 2022-09-20 21:30:51 +00:00
Joey Hess
24016090b3
wording 2022-09-20 14:55:34 -04:00
Joey Hess
8d26fdd670
skip checkRepoConfigInaccessible when git directory specified explicitly
Fix a reversion that prevented git-annex from working in a repository when
--git-dir or GIT_DIR is specified to relocate the git directory to
somewhere else. (Introduced in version 10.20220525)

checkRepoConfigInaccessible could still run git config --list, just passing
--git-dir. It seems not necessary, because I know that passing --git-dir
bypasses git's check for repo ownership. I suppose it might be that git
eventually changes to check something about the ownership of the working
tree, so passing --git-dir without --work-tree would still be worth doing.
But for now this is the simple fix.

Sponsored-by: Nicholas Golder-Manning on Patreon
2022-09-20 14:52:43 -04:00
Joey Hess
d1467a9b8e
bug report rescued from forum 2022-09-20 14:16:15 -04:00
Joey Hess
3826a553d7
comment 2022-09-20 14:06:09 -04:00
Joey Hess
3d51f866d3
comment 2022-09-20 14:02:09 -04:00
Joey Hess
f5a3a12360
wontfix 2022-09-20 13:54:34 -04:00
Joey Hess
7a1b7ce795
comment 2022-09-20 13:49:21 -04:00
Joey Hess
8479ac6a50
fixed 2022-09-20 13:35:47 -04:00
Joey Hess
0756f4453d
try retrieval from more than one export location when the first fails
Combined with commit 0ffc59d341, this
fixes the case where there are duplicate files on the special remote,
and the first gets modified/deleted, while the second is still present.

directory, adb: Fixed a bug when importtree=yes, and multiple files in the
special remote have the same content, that caused it to refuse to get a
file from the special remote, incorrectly complaining that it had changed,
due to only accepting the inode+mtime of one file (that was since modified
or deleted) and not accepting the inode+mtime of other duplicate files.

Sponsored-by: Max Thoursie on Patreon
2022-09-20 13:33:57 -04:00
Joey Hess
0ffc59d341
change retrieveExportWithContentIdentifier to take a list of ContentIdentifier
This partly fixes an issue where there are duplicate files in the
special remote, and the first file gets swapped with another duplicate,
or deleted. The swap case is fixed by this, the deleted case will need
other changes.

This makes retrieveExportWithContentIdentifier take a list of allowed
ContentIdentifier, same as storeExportWithContentIdentifier,
removeExportWithContentIdentifier, and
checkPresentExportWithContentIdentifier.

Of the special remotes that support importtree, borg is a special case
and does not use content identifiers, S3 I assume can't get mixed up
like this, directory certainly has the problem, and adb also appears to
have had the problem.

Sponsored-by: Graham Spencer on Patreon
2022-09-20 13:19:42 -04:00
Joey Hess
3adf1f24e2
comment 2022-09-20 12:56:16 -04:00
Joey Hess
612d2a8056
comment 2022-09-20 12:39:40 -04:00
Joey Hess
223f03e84c
comment 2022-09-20 12:20:33 -04:00
nobodyinperson
845922024b 2022-09-20 12:11:03 +00:00
jgoerzen
a09a796118 Added a comment 2022-09-19 14:49:02 +00:00
jgoerzen
d3408ec63d Added a comment 2022-09-19 13:55:07 +00:00
Gus
00418e8c8f Added a comment 2022-09-19 13:49:15 +00:00
Lukey
95f9cb1340 Added a comment 2022-09-19 10:03:39 +00:00
Gus
9cca1fb2d2 2022-09-18 12:21:07 +00:00
Gus
8c203331bb 2022-09-18 12:20:16 +00:00
eph@6377f195575d4a04abc70f20e0b00dffcc597d00
d08ff11b97 Added a comment 2022-09-17 21:19:13 +00:00
Joey Hess
1fe9cf7043
deal with ignoreinode config setting
Improve handling of directory special remotes with importtree=yes whose
ignoreinode setting has been changed. (By either enableremote or by
upgrading to commit 3e2f1f73cbc5fc10475745b3c3133267bd1850a7.)

When getting a file from such a remote, accept the content that would have
been accepted with the previous ignoreinode setting.

After a change to ignoreinode, importing a tree from the remote will
re-import and generate new content identifiers using the new config. So
when ignoreinode has changed to no, the inodes will be learned, and after
that point, a change in an inode will be detected as a change. Before
re-importing, a change in an inode will be ignored, as it was before the
ignoreinode change. This seems acceptble, because the user can re-import
immediately if they urgently need to add inodes. And if not, they'll
do it sometime, presumably, and the change will take effect then.

Sponsored-by: Erik Bjäreholt on Patreon
2022-09-16 14:11:25 -04:00
Joey Hess
4a1030d51d
comments 2022-09-16 13:40:27 -04:00