Commit graph

45004 commits

Author SHA1 Message Date
Joey Hess
783eb8879a
notes on behavior 2024-06-10 11:07:04 -04:00
jlueters@79a910340cdff27611c6a650c108afbe2f61c5f6
daa2c6cce1 2024-06-10 14:24:34 +00:00
Joey Hess
b1cc8c6837
Merge branch 'master' of ssh://git-annex.branchable.com 2024-06-07 16:52:04 -04:00
Joey Hess
25a6ab6f11
Avoid grafting in export tree objects that are missing
They could be missing due to an interrupted git-annex at just the wrong
time during a prior graft, after which the tree objects got garbage
collected.

Or they could be missing because of manual messing with the git-annex
branch, eg resetting it to back before the graft commit.

Sponsored-by: Dartmouth College's OpenNeuro project
2024-06-07 16:51:50 -04:00
emilymaers
3947a51cc8 removed 2024-06-07 20:46:34 +00:00
emilymaers
4fbfc5e5ac Added a comment: blockchain 2024-06-07 20:46:20 +00:00
Joey Hess
b32c4c2e98
atomic git-annex branch update when regrafting in transition
Fix a bug where interrupting git-annex while it is updating the git-annex
branch could lead to git fsck complaining about missing tree objects.

Interrupting git-annex while regraftexports is running in a transition
that is forgetting git-annex branch history would leave the
repository with a git-annex branch that did not contain the tree shas
listed in export.log. That lets those trees be garbage collected.

A subsequent run of the same transition then regrafts the trees listed
in export.log into the git-annex branch. But those trees have been lost.

Note that both sides of `if neednewlocalbranch` are atomic now. I had
thought only the True side needed to be, but I do think there may be
cases where the False side needs to be as well.

Sponsored-by: Dartmouth College's OpenNeuro project
2024-06-07 16:34:10 -04:00
Joey Hess
f5532be954
graft in exported tree before updating the export log
It was possible for the export.log to get written and then git-annex was
interrupted, before it could graft in the exported tree. Which could
result in export.log referencing a tree that got garbage collected.
2024-06-07 15:25:02 -04:00
Joey Hess
6568ba4904
Merge branch 'master' into proxy 2024-06-07 12:35:47 -04:00
Joey Hess
43ff697f25
update status and design work on proxy encryption and chunking 2024-06-07 12:35:04 -04:00
Joey Hess
a0e59c1d17
comment 2024-06-07 12:35:00 -04:00
Joey Hess
4b940c92bb
proxied remotes working on client side
Got the right git config settings inherited now.

Note that the url config is not passed on to git, so it won't be able
to access the proxied remote. That would need some kind of
git-remote-annex but for proxied remotes anyway. Unsure yet if that will
be needed.
2024-06-07 12:11:46 -04:00
Joey Hess
5aaa285083
Merge branch 'master' into proxy 2024-06-07 10:43:13 -04:00
Joey Hess
058726ee86
next step identified 2024-06-06 18:06:45 -04:00
Joey Hess
d59383beaf
update 2024-06-06 17:25:22 -04:00
Joey Hess
9bc4dd635c
update 2024-06-06 17:23:51 -04:00
Joey Hess
b43c835def
instantiate remotes that are behind a proxy remote
Untested, but this should be close to working. The proxied remotes have
the same url but a different uuid. When talking to current
git-annex-shell, it will fail due to a uuid mismatch. Once it supports
proxies, it will know that the presented uuid is for a remote that it
proxies for.

The check for any git config settings for a remote with the same name as
the proxied remote is there for several reasons. One is security:
Writing a name to the proxy log should not cause changes to
how an existing, configured git remote operates in a different clone of
the repo.

It's possible that the user has been using a proxied remote, and decides
to set a git config for it. We can't tell the difference between that
scenario and an evil remote trying to eg, intercept a file upload
by replacing their remote with a proxied remote.

Also, if the user sets some git config, does it override the config
inherited from the proxy remote? Seems a difficult question. Luckily,
the above means we don't need to think through it.

This does mean though, that in order for a user to change the config of
a proxy remote, they have to manually set its annex-uuid and url, as
well as the config they want to change. They may also have to set any of
the inherited configs that they were relying on.
2024-06-06 17:15:32 -04:00
Joey Hess
7f1cdb3107
remote git config inheritance for proxied remotes
When there is a proxy remote, remotes that it proxies need to be
constructed with the right subset of the remote git-config settings.
Obviously, the url is the same, and the uuid is different.

Added proxyInheritedFields that lists all the fields that should be
inherited. These will be copied into the proxied remote when instantiating it.

There were a lot of decisions here, made without certainty in some
cases. May need to revisit them.

The RemoteGitConfigField type was added to make sure that every config
used in extractRemoteGitConfig gets considered for proxy inheritance,
including new ones that get added going forward. And to avoid needing to
write the field string more than once.
2024-06-06 16:30:40 -04:00
Joey Hess
a72d0f69d0
filter out illegal remote names when reading proxy log 2024-06-06 12:51:30 -04:00
Joey Hess
d208b03e5d
Merge branch 'master' into proxy 2024-06-06 12:42:18 -04:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
1e6b4f324a removed 2024-06-06 13:40:26 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
6274d16102 Added a comment 2024-06-06 11:23:55 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
d4993248eb Added a comment 2024-06-06 11:23:34 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
a1e1af35af 2024-06-06 10:29:21 +00:00
nobodyinperson
6985c62a47 Added a comment 2024-06-06 09:09:03 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
7dbfb16415 2024-06-05 17:45:49 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
93b11da4db Added a comment 2024-06-05 17:34:32 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
6b4ae7b635 2024-06-05 17:22:04 +00:00
ruslan@302cb7f8d398fcce72f88b26b0c2f3a53aaf0bcd
ca687413ef Added a comment 2024-06-05 16:53:51 +00:00
Joey Hess
1761e971ee
status update after day 1 of new project 2024-06-04 14:55:54 -04:00
Joey Hess
f97f4b8bdb
Added updateproxy command and remote.name.annex-proxy configuration
So far this only records proxy information on the git-annex branch.
2024-06-04 14:52:03 -04:00
Joey Hess
f3f40e03b4
clarify comment
Since remotes not being available is a thing, this was confusing.
2024-06-04 14:29:24 -04:00
Joey Hess
8dbee28963
reorder imports for consistency 2024-06-04 13:25:48 -04:00
Joey Hess
a31770c350
reorder constructors for consistency 2024-06-04 13:11:27 -04:00
Joey Hess
3df70c5c0c
implementation plan 2024-06-04 07:51:33 -04:00
Joey Hess
6375e3be3b
recieved funding to work on this, which comes with a schedule 2024-06-04 06:53:59 -04:00
Joey Hess
ac3fe92956
comment 2024-06-04 06:41:14 -04:00
Joey Hess
3db94f1b71
Merge branch 'master' of ssh://git-annex.branchable.com 2024-06-04 06:40:08 -04:00
Joey Hess
3be7163771
update 2024-06-04 06:40:04 -04:00
Joey Hess
5992e1729a
fixed by git release 2024-06-04 06:39:08 -04:00
nobodyinperson
c606b6a35d Added a comment: Yes, GitLab fixed! 2024-06-04 07:38:47 +00:00
datamanager
82b891de7a Added a comment: GitLab fixed? 2024-06-04 01:18:25 +00:00
Joey Hess
61ed0b3f03
root cause analysis 2024-06-03 13:56:43 -04:00
yarikoptic
4a48933867 Added a comment 2024-06-03 17:54:43 +00:00
Joey Hess
da2c02162c
Fix Windows build with Win32 2.13.4+
Thanks, Oleg Tolmatcev
2024-06-03 13:04:15 -04:00
Yaroslav Halchenko
176eb9c9c6
ENH: add one more mailmap for yarikoptic 2024-06-03 13:00:45 -04:00
Joey Hess
c382555cf8
comment 2024-06-03 12:31:55 -04:00
jkniiv
313a0285e5 a small clarification 2024-06-01 22:11:32 +00:00
jkniiv
5badd2ae4e report on git-remote-annex on Windows not quite working 2024-06-01 21:59:27 +00:00
Joey Hess
0e96f0acd8
add news item for git-annex 10.20240531 2024-05-31 12:32:42 -04:00