Commit graph

31403 commits

Author SHA1 Message Date
Atemu
6253bd391c Added a comment 2022-03-08 13:24:21 +00:00
Atemu
c7e3414d8d Added a comment 2022-03-08 13:21:03 +00:00
Atemu
327d06eb9d Added a comment 2022-03-08 11:56:42 +00:00
anamyk
a5086345bc Added a comment 2022-03-08 08:11:17 +00:00
anamyk
d4aa2dfb94 Added a comment 2022-03-08 07:49:38 +00:00
yarikoptic
875a04e1e2 Added a comment: still slow 2022-03-08 00:28:00 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
041cea21ba Added a comment 2022-03-07 19:59:18 +00:00
Joey Hess
cd473f071c
comment 2022-03-07 15:30:16 -04:00
Joey Hess
434812bcfb
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 15:25:57 -04:00
Joey Hess
1cbbd23109
comment 2022-03-07 15:25:32 -04:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
85eb26e746 Added a comment 2022-03-07 19:22:54 +00:00
Joey Hess
5fb98d5f29
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 15:20:00 -04:00
Joey Hess
06c2f625a9
comment 2022-03-07 15:18:30 -04:00
tomdhunt
0127e5f4e4 Added a comment 2022-03-07 18:27:38 +00:00
Joey Hess
da698437b6
close 2022-03-07 14:12:11 -04:00
Joey Hess
dab9078ab7
close 2022-03-07 13:23:19 -04:00
Joey Hess
675a491632
document nix-on-droid as an alternative to termux
Sponsored-by: Graham Spencer on Patreon
2022-03-07 12:57:16 -04:00
Joey Hess
5154d15183
comment 2022-03-07 12:11:18 -04:00
Joey Hess
5804ea67a6
comment; close 2022-03-07 12:03:11 -04:00
Joey Hess
568c469e7f
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 12:00:14 -04:00
nowkixide
6a45d7e5d0 Added a comment: Works fine 2022-03-07 12:07:54 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
ce523f7561 Added a comment 2022-03-06 14:37:42 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
6050b839f4 2022-03-06 06:46:01 +00:00
Atemu
54a43f68ed Added a comment 2022-03-05 21:46:23 +00:00
Atemu
1304660e06 Added a comment 2022-03-05 18:47:10 +00:00
Joey Hess
b93e282163
update 2022-03-05 10:22:39 -04:00
Atemu
801673a390 Added a comment 2022-03-04 20:27:31 +00:00
mih
fa5a001ef6 Added a comment: Thanks! 2022-03-04 16:23:16 +00:00
Atemu
e879487dca Added a comment 2022-03-04 14:44:29 +00:00
Atemu
c0164c4750 Added a comment 2022-03-04 14:16:54 +00:00
Atemu
ffc17b0b49 rename bugs/git-annex_is_slow_at_reading_files.mdwn to bugs/git-annex_is_slow_at_reading_file_content.mdwn 2022-03-03 08:40:47 +00:00
Joey Hess
55f71b0ebd
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-02 10:57:41 -04:00
Joey Hess
5e385cb637
add 2022-03-02 10:44:49 -04:00
Atemu
2998c33fcb 2022-03-02 13:18:42 +00:00
Atemu
6d768a8268 2022-03-02 13:16:36 +00:00
Atemu
a7c5913d42 2022-03-02 13:15:26 +00:00
anamyk
de0c6f08c6 Added a comment 2022-03-01 22:52:51 +00:00
Joey Hess
2fc46e1871
git-annex test from standalone speedup
Avoid git-annex test being very slow when run from within the standalone
linux tarball or OSX app.

It may not really be necessary to add to PATH the directory where the
git-annex binary resides, but it can't hurt. Most places where the test
suite or git-annex run git-annex, they use programPath, so won't need
a modified PATH. But I'm not sure if that's always the case.

Sponsored-by: Dartmouth College's Datalad project
2022-03-01 16:08:55 -04:00
Joey Hess
ecf7c29107
update comment 2022-03-01 15:57:13 -04:00
Joey Hess
316a049e96
comment 2022-03-01 15:50:44 -04:00
yarikoptic
ca5834a18c Added a comment: question about backend 2022-02-28 22:42:34 +00:00
yarikoptic
9e6e53af71 initial report on a very slow git annex test on discovery 2022-02-28 20:50:06 +00:00
Joey Hess
525218ef86
commet 2022-02-28 15:42:33 -04:00
yarikoptic
a33b40876d Added a comment 2022-02-28 18:48:51 +00:00
anamyk
2fa5ef6a36 Added a comment 2022-02-28 18:13:29 +00:00
Joey Hess
7a4a1322f5
update 2022-02-28 13:37:05 -04:00
Joey Hess
20875bd5e8
open related todo 2022-02-28 13:26:43 -04:00
Joey Hess
3d433d76a3
remove 127/128 exit status mention
I cannot find any indication that git-annex uses these exit statuses,
and I see I didn't write this doc so I don't know where those numbers
came from. Odd.
2022-02-28 13:26:26 -04:00
Joey Hess
693fec0066
mention --size-limit effect on exit status
Documented elsewhere already.
2022-02-28 13:22:05 -04:00
Joey Hess
ce91f10132
fix annex.skipunknown false error propagation
Propagate nonzero exit status from git ls-files when a specified file does
not exist, or a specified directory does not contain any files checked into
git.

The recent completion of the annex.skipunknown transition exposed this
bug, that has unfortunately been lurking all along.

It is also possible that git ls-files errors out for some other reason
-- perhaps a permission problem -- and this will also fix error propagation
in such situations.

Sponsored-by: Dartmouth College's Datalad project
2022-02-28 12:54:56 -04:00
Joey Hess
a6857ddb79
comment 2022-02-28 12:15:08 -04:00
Joey Hess
f7575e87a0
news 2022-02-28 12:10:50 -04:00
Joey Hess
fb03e38c1a
comment 2022-02-28 12:10:19 -04:00
yarikoptic
5d7b0b6291 initial report on changed (and non-conformant) behavior of annex find 2022-02-25 20:53:25 +00:00
Joey Hess
7de469edd0
comment 2022-02-25 13:32:06 -04:00
Joey Hess
51c528980c
avoid accidentally thawing git-annex symlink
It did nothing, since at this point the link is dangling. But when there
is a thaw hook, it would probably not be happy to be asked to run on a
symlink, or might do something unexpected.

Sponsored-by: Dartmouth College's Datalad project
2022-02-24 14:21:23 -04:00
Joey Hess
f4b046252a
Run annex.thawcontent-command before deleting an object file
In case annex.freezecontent-command did something that would prevent
deletion.

Sponsored-by: Dartmouth College's Datalad project
2022-02-24 14:11:02 -04:00
Joey Hess
28bc5ce232
ignore write bits being set when there is a freeze hook
When annex.freezecontent-command is set, and the filesystem does not
support removing write bits, avoid treating it as a crippled filesystem.

The hook may be enough to prevent writing on its own, and some filesystems
ignore attempts to remove write bits.

Sponsored-by: Dartmouth College's Datalad project
2022-02-24 13:28:31 -04:00
Joey Hess
4144f73c18
comment 2022-02-24 13:10:40 -04:00
Joey Hess
972f41efb7
comment 2022-02-24 13:05:25 -04:00
Joey Hess
4103ae83db
comment 2022-02-24 12:28:55 -04:00
moortgat-pick
0e4f3ad022 Added a comment 2022-02-24 09:41:44 +00:00
yarikoptic
dce6877758 Added a comment: for some reason drop freezes thawn key file before thawing key dir 2022-02-24 03:21:54 +00:00
yarikoptic
ee9cd6309a Added a comment: approach which somewhat works 2022-02-24 02:20:15 +00:00
yarikoptic
1daceb1c16 adjusted with edit 1 observation 2022-02-23 21:08:16 +00:00
yarikoptic
f2331feed1 initial experimentation/complain about freeze/thawing files 2022-02-23 20:53:38 +00:00
Joey Hess
64ccb4734e
smudge: Warn when encountering a pointer file that has other content appended to it
It will then proceed to add the file the same as if it were any other
file containing possibly annexable content. Usually the file is one that
was annexed before, so the new, probably corrupt content will also be added
to the annex. If the file was not annexed before, the content will be added
to git.

It's not possible for the smudge filter to throw an error here, because
git then just adds the file to git anyway.

Sponsored-by: Dartmouth College's Datalad project
2022-02-23 15:17:08 -04:00
Joey Hess
67245ae00f
fully specify the pointer file format
This format is designed to detect accidental appends, while having some
room for future expansion.

Detect when an unlocked file whose content is not present has gotten some
other content appended to it, and avoid treating it as a pointer file, so
that appended content will not be checked into git, but will be annexed
like any other file.

Dropped the max size of a pointer file down to 32kb, it was around 80 kb,
but without any good reason and certianly there are no valid pointer files
anywhere that are larger than 8kb, because it's just been specified what it
means for a pointer file with additional data even looks like.

I assume 32kb will be good enough for anyone. ;-) Really though, it needs
to be some smallish number, because that much of a file in git gets read
into memory when eg, catting pointer files. And since we have no use cases
for the extra lines of a pointer file yet, except possibly to add
some human-visible explanation that it is a git-annex pointer file, 32k
seems as reasonable an arbitrary number as anything. Increasing it would be
possible, eg to 64k, as long as users of such jumbo pointer files didn't
mind upgrading all their git-annex installations to one that supports the
new larger size.

Sponsored-by: Dartmouth College's Datalad project
2022-02-23 14:20:31 -04:00
Joey Hess
38816a9ae9
comment 2022-02-23 11:23:48 -04:00
moortgat-pick
44b9c65af5 2022-02-23 12:53:15 +00:00
moortgat-pick
ebbe840c4c 2022-02-23 12:46:37 +00:00
moortgat-pick
2a68a613a9 2022-02-23 12:45:11 +00:00
https://christian.amsuess.com/chrysn
9d43856075 Added a comment: inodes of git vs. git-annex 2022-02-23 12:18:54 +00:00
Joey Hess
a6b53cb739
add news item for git-annex 10.20220222 2022-02-22 13:34:58 -04:00
Joey Hess
866cad6582
comment 2022-02-22 12:04:10 -04:00
yarikoptic
7e9ebea910 Added a comment 2022-02-21 21:53:03 +00:00
yarikoptic
c59a7b1275 Added a comment 2022-02-21 21:48:59 +00:00
yarikoptic
8ee2661b85 Added a comment 2022-02-21 21:46:15 +00:00
Joey Hess
83827a6822
comment 2022-02-21 15:59:29 -04:00
Joey Hess
5a8b15f6db
comment 2022-02-21 15:46:12 -04:00
Joey Hess
ba907b6682
comment and close, not a bug really 2022-02-21 15:39:38 -04:00
Joey Hess
80f244d7b6
comment 2022-02-21 15:13:51 -04:00
Joey Hess
9c1ff6d024
comment 2022-02-21 15:00:55 -04:00
Joey Hess
ce1b3a9699
info: Allow using matching options in more situations
File matching options like --include will be rejected in situations where
there is no filename to match against. (Or where there is a filename but
it's not relative to the cwd, or otherwise seemed too bothersome to match
against.)

The addition of listKeys' was necessary to avoid using more memory in the
common case of "git-annex info". Adding a filterM would have caused the
list to buffer in memory and not stream. This is an ugly hack, but listKeys
had previously run Annex operations inside unafeInterleaveIO (for direct
mode). And matching against a matcher should hopefully not change any Annex
state.

This does allow for eg `git-annex info somefile --include=*.ext`
although why someone would want to do that I don't really know. But it
seems to make sense to allow it.
But, consider: `git-annex info ./somefile --include=somefile`
This does not match, so will not display info about somefile.
If the user really wants to, they can `--include=./somefile`.

Using matching options like --copies or --in=remote seems likely to be
slower than git-annex find with those options, because unlike such
commands, info does not have optimised streaming through the matcher.

Note that `git-annex info remote` is not the same as
`git-annex info --in remote`. The former shows info about all files in
the remote. The latter shows local keys that are also in that remote.
The output should make that clear, but this still seems like a point
where users could get confused.

Sponsored-by: Jochen Bartl on Patreon
2022-02-21 14:46:07 -04:00
Joey Hess
d36de3edf9
comment 2022-02-21 12:49:36 -04:00
Joey Hess
3187639735
retitle 2022-02-21 12:13:59 -04:00
Atemu
6ca9f5e18a 2022-02-20 18:03:35 +00:00
xloem
499b940dc5 Added a comment 2022-02-19 07:50:53 +00:00
xloem
7976d9d303 removed 2022-02-19 07:48:48 +00:00
xloem
6143695820 Added a comment: free dweb storage services 2022-02-19 07:47:28 +00:00
xloem
32d138b52f Added a comment: free dweb storage services 2022-02-19 07:47:07 +00:00
ycp@f118e050dc106530b9cf62ead031e05eef7b1687
7f167d4be1 Added a comment 2022-02-19 01:15:24 +00:00
yarikoptic
b481ec2738 Added a comment 2022-02-18 21:56:19 +00:00
yarikoptic
9d2e6a60f0 Added a comment 2022-02-18 20:18:04 +00:00
yarikoptic
12fc03a9b2 initial thinking for a possible safe guard 2022-02-18 20:02:40 +00:00
yarikoptic
4643788916 bug/question on the semantic of find --unlocked 2022-02-18 19:17:37 +00:00
Joey Hess
faf84aa5c2
Avoid git status taking a long time after git-annex unlock of many files.
Implemented by making Git.Queue have a FlushAction, which can accumulate
along with another action on files, and runs only once the other action has
run.

This lets git-annex unlock queue up git update-index actions, without
conflicting with the restagePointerFiles FlushActions.

In a repository with filter-process enabled, git-annex unlock will
often not take any more time than before, though it may when the files are
large. Either way, it should always slow down less than git-annex status
speeds up.

When filter-process is not enabled, git-annex unlock will slow down as much
as git status speeds up.

Sponsored-by: Jochen Bartl on Patreon
2022-02-18 15:06:40 -04:00
Joey Hess
c68f52c6a2
restage pointer file after unlock
This avoids a later git status or similar taking a long time to run
as it runs git-annex smudge once per file. While v9 repositories do
avoid that taking long when the files are small, large files can still
make git status take a very long time.

This does make unlock slower, because now git-annex smudge is being run
once per file unlocked. However, the next commit should speed that up in
many cases.

Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2022-02-18 14:55:52 -04:00
Joey Hess
07215cfeb5
complete annex.skipunknown transition
annex.skipunknown now defaults to false, so commands like `git annex get foo*`
will not silently skip over files/dirs that are not checked into git.

Sponsored-by: Brock Spratlen on Patreon
2022-02-18 13:18:05 -04:00
Joey Hess
56d180864f
comment 2022-02-18 12:35:49 -04:00