Commit graph

30917 commits

Author SHA1 Message Date
Joey Hess
b2c48fb86b
Fix using lookupkey inside a subdirectory
Caused by dirContains ".." "foo" being incorrectly False.

Also added a test of dirContains, which includes all the previous bug fixes
I could find and some obvious cases.

Reversion in version 8.20211011

Sponsored-by: Brett Eisenberg on Patreon
2021-10-26 15:00:45 -04:00
Joey Hess
91eb393df4
comment 2021-10-26 14:25:28 -04:00
Joey Hess
3aaf6ade30
review 2021-10-26 14:08:56 -04:00
Joey Hess
5a9e6b1fd4
when private journal file exists, still read from git-annex branch
Fix bug that caused stale git-annex branch information to read when
annex.private or remote.name.annex-private is set.

The private journal file should not prevent reading more current
information from the git-annex branch, but used to.

Note that, overBranchFileContents has to do additional work now, when
there's a private journal file, it reads from the branch redundantly
and more slowly.

Sponsored-by: Jack Hill on Patreon
2021-10-26 13:43:50 -04:00
Lukey
0a04986ac8 Added a comment 2021-10-25 10:25:05 +00:00
Lukey
9f1bf48696 2021-10-25 10:16:35 +00:00
pat
cc877dfe45 Added a comment 2021-10-24 22:42:35 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
b2c797e35b Added a comment 2021-10-24 21:23:56 +00:00
jwodder
be3cf1115e 2021-10-24 19:21:54 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
63313e0b40 2021-10-24 19:12:23 +00:00
jkniiv
4471aae22f still think we should highlight this as new info 2021-10-24 15:10:13 +00:00
username
6f3cadcbda Added a comment 2021-10-23 15:53:57 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
42eaa6f8c9 Moved WSL1 guide to a tips page 2021-10-22 22:14:40 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
ebe022e9cf 2021-10-22 22:13:46 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
29dfc0a9bf Added a comment 2021-10-22 15:58:11 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
201ac941a7 2021-10-21 21:24:24 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
f9e09277c5 2021-10-21 17:17:42 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
efaa1fe94a Added a comment 2021-10-21 08:24:55 +00:00
jwodder
471464e4a0 2021-10-21 04:54:26 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
f8faeabb70 Added a comment 2021-10-21 02:09:02 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
ae80b1214c Added a comment 2021-10-21 02:06:50 +00:00
jwodder
c6d1c7d209 2021-10-21 01:35:48 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
67baa63e5b Added a comment 2021-10-20 22:54:39 +00:00
Joey Hess
6bed8ccbd1
comment 2021-10-20 15:25:35 -04:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
16b08ffcd9 Added a comment: Same problem with git-annex 8.20210223 2021-10-20 19:09:29 +00:00
Joey Hess
f3e33889dd
comment 2021-10-20 14:51:18 -04:00
Joey Hess
53f315db5a
comment 2021-10-20 14:12:08 -04:00
Joey Hess
cb9c54ab25
comment 2021-10-20 13:57:44 -04:00
Joey Hess
dfe5a98286
update 2021-10-20 13:48:37 -04:00
Joey Hess
2801528eb2
oops, I misread, still happens for adjusted branches 2021-10-20 13:45:56 -04:00
Joey Hess
3f3a2d290f
linked bugs 2021-10-20 13:39:13 -04:00
Joey Hess
fe734210eb
comment 2021-10-20 13:24:46 -04:00
Joey Hess
95032bfae7
update per other comment 2021-10-20 13:12:16 -04:00
Joey Hess
77c0892e27
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-20 13:07:09 -04:00
Joey Hess
7b48ce60ef
comment 2021-10-20 13:06:51 -04:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
2ae821e458 Added a comment 2021-10-20 14:42:33 +00:00
gitannexuser2021
cf08ac583a Added a comment: Still seeing the issue. 2021-10-19 22:37:45 +00:00
Joey Hess
47e30f78be
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-19 15:15:29 -04:00
Joey Hess
f4bdecc4ec
improve sqlite MultiWriter handling of read after write
This removes a messy caveat that was easy to forget and caused at least one
bug. The price paid is that, after a write to a MultiWriter db, it has to
close the db connection that it had been using to read, and open a new
connection. So it might be a little bit slower. But, writes are usually
batched together, so there's often only a single write, and so there should
not be much of a slowdown. Notice that SingleWriter already closed the db
connection after a write, so paid the same overhead.

This is the second try at fixing a bug: git-annex get when run as the first
git-annex command in a new repo did not populate all unlocked files.
(Reversion in version 8.20210621)

Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-10-19 15:13:29 -04:00
Lukey
82b54f53ec Added a comment 2021-10-19 17:17:47 +00:00
Joey Hess
ade67b78c5
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-19 13:17:04 -04:00
Joey Hess
c49b7c976d
reopen 2021-10-19 13:16:29 -04:00
Joey Hess
0f38ad9a69
close keys db to possibly work around WSL1 issue 2021-10-19 13:07:49 -04:00
paperbenni
514bbfd7f3 2021-10-19 16:45:12 +00:00
Joey Hess
67a67d740b
comment 2021-10-19 12:43:08 -04:00
Joey Hess
d7bbe01a95
remove xmpp mention 2021-10-19 12:26:22 -04:00
Joey Hess
81ec8508df
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-19 12:03:51 -04:00
Joey Hess
3de3f40c11
comment 2021-10-19 10:26:39 -04:00
Atemu
6824a56d09 Added a comment 2021-10-19 13:05:50 +00:00
Atemu
8626f35898 Added a comment 2021-10-19 12:26:33 +00:00
username
0c216a7cb8 Added a comment 2021-10-18 19:54:29 +00:00
Joey Hess
bd97a0cbd9
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-18 11:50:33 -04:00
Lukey
862854028a Added a comment 2021-10-18 15:36:06 +00:00
Joey Hess
6fe5e31d8c
note about encryption/chunking 2021-10-18 11:11:24 -04:00
username
c47c76feaa Added a comment: directory special remote 2021-10-17 22:15:52 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
6d53f52092 Added a comment 2021-10-17 01:14:18 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
8a20c01775 Added a comment 2021-10-16 15:46:32 +00:00
Lukey
7c40c31210 Added a comment 2021-10-16 15:27:35 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
7ec426734b Note about case sensitivity dirs 2021-10-16 15:25:00 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
8ef65adaca Update WSL1 instructions 2021-10-16 15:16:23 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
4eefcf2c75 2021-10-16 14:45:43 +00:00
athas@60e56fd42a78bbbce444d175865ce4d66ba1a779
9c9bd88cea Added a comment: GitHub Actions 2021-10-15 20:53:06 +00:00
Joey Hess
f9e2d3d900
update 2021-10-15 13:50:57 -04:00
Joey Hess
909e980a74
comment 2021-10-15 13:49:41 -04:00
Joey Hess
cfca6f2d1d
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-15 13:47:43 -04:00
Joey Hess
f42679364b
comment 2021-10-15 13:14:19 -04:00
Joey Hess
647fc90b12
comment 2021-10-15 12:43:23 -04:00
athas@60e56fd42a78bbbce444d175865ce4d66ba1a779
b93f4f65a5 Added a comment: Update 2021-10-15 16:37:34 +00:00
athas@60e56fd42a78bbbce444d175865ce4d66ba1a779
1f8d6543ce removed 2021-10-15 16:21:07 +00:00
athas@60e56fd42a78bbbce444d175865ce4d66ba1a779
00e88a546d Added a comment: Update 2021-10-15 16:13:32 +00:00
athas@60e56fd42a78bbbce444d175865ce4d66ba1a779
a7fa597ad4 2021-10-15 16:09:51 +00:00
Joey Hess
29d687dce9
When retrival from a chunked remote fails, display the error that occurred when downloading the chunk
Rather than the error that occurred when trying to download the unchunked
content, which is less likely to actually be stored in the remote.

Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-10-14 12:45:05 -04:00
Joey Hess
20c375d912
followup 2021-10-14 12:08:54 -04:00
Joey Hess
97dfaabbf0
remove 3 comments that turned out to be about an unrelated problem which got its own bug report 2021-10-14 12:05:07 -04:00
Joey Hess
3b5ac2d0d1
close per comment 2021-10-12 13:42:55 -04:00
Joey Hess
b117f9338d
open todo 2021-10-12 13:42:08 -04:00
Joey Hess
c2a44eab50
move gpg tmp home to system temp dir
test: Put gpg temp home directory in system temp directory, not filesystem
being tested.

Since I've found indications gpg can fail talking to the agent when the
socket ends up on eg, fat. And to hopefully fix this bug report I've
followed up on.

The main risk in using the system temp dir is that TMPDIR could be set to a
long directory path, which is too long to put a unix socket in. To
partially amelorate that risk, it uses either an absolute or a relative
path, whichever is shorter. (Hopefully gpg will not convert it to a longer
form of the path..)

If the user sets TMPDIR to something so long a path to it +
"S.gpg-agent" is too long, I suppose that's their issue to deal with.

Sponsored-by: Dartmouth College's Datalad project
2021-10-12 13:29:56 -04:00
Joey Hess
17a0fa3dbc
negotiate P2P protocol version for tor remotes
This negotiation is not supported by versions of git-annex older
than 6.20180312. Well, maybe really 6.20180227 or so, but using that in
the changelog simplifies things since it was the version for the other
changes as well.

See commit c81768d425 for the back story.

As well as allowing for future protocol improvements, this will result
in negoatiating protocol version 1, which is an improvement over default
version 0.

In fact, it looks like no supported version of git-annex will use
protocol version 0, since version 1 was introduced in 6.20180227.
Still, removing the code for version 0 seems unncessary.
See commit 31e1adc005.

Sponsored-by: Brett Eisenberg on Patreon.
2021-10-11 15:58:51 -04:00
Joey Hess
7bdc7350a5
remove git-annex-shell compat code
* Removed support for accessing git remotes that use versions of
  git-annex older than 6.20180312.
* git-annex-shell: Removed several commands that were only needed to
  support git-annex versions older than 6.20180312.
  (lockcontent, recvkey, sendkey, transferinfo, commit)

The P2P protocol was added in that version, and used ever since, so
this code was only needed for interop with older versions.

"git-annex-shell commit" is used by newer git-annex versions, though
unnecessarily so, because the p2pstdio command makes a single commit at
shutdown. Luckily, it was run with stderr and stdout sent to /dev/null,
and non-zero exit status or other exceptions are caught and ignored. So,
that was able to be removed from git-annex-shell too.

git-annex-shell inannex, recvkey, sendkey, and dropkey are still used by
gcrypt special remotes accessed over ssh, so those had to be kept.
It would probably be possible to convert that to using the P2P protocol,
but it would be another multi-year transition.

Some git-annex-shell fields were able to be removed. I hoped to remove
all of them, and the very concept of them, but unfortunately autoinit
is used by git-annex sync, and gcrypt uses remoteuuid.

The main win here is really in Remote.Git, removing piles of hairy fallback
code.

Sponsored-by: Luke Shumaker
2021-10-11 15:36:51 -04:00
Joey Hess
5e3fe816ef
add news item for git-annex 8.20211011 2021-10-11 12:53:40 -04:00
Joey Hess
8deef700d5
document how to resume downloads 2021-10-11 12:40:16 -04:00
Joey Hess
e42215177e
comment 2021-10-11 12:22:38 -04:00
Joey Hess
0c12d01233
update 2021-10-11 10:06:36 -04:00
Joey Hess
1b79f2404d
Merge branch 'master' of ssh://git-annex.branchable.com 2021-10-08 13:27:23 -04:00
Joey Hess
022bb6174c
Merge branch 'borgchunks' 2021-10-08 13:26:45 -04:00
Joey Hess
7ae7820ac0
todo 2021-10-08 13:26:40 -04:00
Joey Hess
69f8e6c7c0
ImportableContentsChunkable
This improves the borg special remote memory usage, by
letting it only load one archive's worth of filenames into memory at a
time, and building up a larger tree out of the chunks.

When a borg repository has many archives, git-annex could easily OOM
before. Now, it will use only memory proportional to the number of
annexed keys in an archive.

Minor implementation wart: Each new chunk re-opens the content
identifier database, and also a new vector clock is used for each chunk.
This is a minor innefficiency only; the use of continuations makes
it hard to avoid, although putting the database handle into a Reader
monad would be one way to fix it.

It may later be possible to extend the ImportableContentsChunkable
interface to remotes that are not third-party populated. However, that
would perhaps need an interface that does not use continuations.

The ImportableContentsChunkable interface currently does not allow
populating the top of the tree with anything other than subtrees. It
would be easy to extend it to allow putting files in that tree, but borg
doesn't need that so I left it out for now.

Sponsored-by: Noam Kremen on Patreon
2021-10-08 13:15:22 -04:00
alex@f04d0d3c452a2a99b27ccc93c1543bee4a1bf5be
c5dd89f8de Added a comment: Re: Resuming an interrupted download 2021-10-08 03:06:57 +00:00
jkniiv
921e736953 Added a comment 2021-10-07 18:27:19 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
5bbbc87e99 Added a comment 2021-10-07 16:27:39 +00:00
jkniiv
04cb121b16 add footnote about the permissions required and Microsoft blog post 2021-10-07 16:25:37 +00:00
jkniiv
5135730467 remove mention of mount option case=dir (turned out to be unnecessary) 2021-10-07 11:40:25 +00:00
jkniiv
d9c2a632f5 Added a comment 2021-10-07 11:36:49 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
392a886263 Added a comment 2021-10-07 06:21:25 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
ae00d385fe Added a comment 2021-10-07 06:17:30 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
bcf7bd8505 2021-10-07 05:53:23 +00:00
jkniiv
f36272be0c Added a comment: the WSL1 use case 2021-10-07 04:12:48 +00:00
jkniiv
2b06612ed5 rename WSL1 section to highlight date, add wording about being experimental, reword some awkwardness, add further directions 2021-10-07 03:56:37 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
c3afda1699 Add steps for WSL1 2021-10-06 22:35:27 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
50c2bd49e5 Added a comment 2021-10-06 21:10:34 +00:00