That is not allowed, so it's not a bug in git-annex when it happens and
instead tell the special remote developer how it's messed up.
Note that currently only Remote.External can overwrite the parsed remote
config with a PassedThrough value. PassedThrough values are otherwise
only generated for configs that are not parsed by the remote config
parser.
Sponsored-by: Joshua Antonishen
p2phttp: Fix a hang that could occur when used with --directory, and a
repository in the repository got removed.
It could leak up to -J number of worker threads, but this only affected a
client trying to access the deleted repository.
It may be that this could also affect a non-deleted repository, and also
leak a worker thread, if invalid p2p protocol is sent.
git write-tree was being run once per file git-annex acts on when eg,
getting files, which is slow when the remote repository has a large
tree.
onLocal calls quiesce after each action, and quiesce closes the keys db
since [[!commit ba7ecbc6a9c]]. Which has a relevant comment about
performance. I have not addressed that, the keys db still gets closed and
reopened after each file.
Turns out that, since git write-tree was run by each call to
reconcileStaged, the .git/annex/keysdb.cache value was never the
same as the git index's inode. Because git write-tree updates the index's
mtime even when no changes have been made.
And so, when the database got closed and reopened, reconcileStaged would
see a changed index, and run git write-tree again. Over and over.
I considered writing the index's new inodecache after write-tree to the
keysdb.cache, but that would be vulnerable to a race, if the index was
changed just after write-tree.
The fix was to stop using keysb.cache at all. When the database is closed
and later reopened by the same process, avoid re-doing reconcileStaged.
Now that .git/annex/keysdb.cache is no longer used. It could be removed,
but the time overhead of removing it would be more than the space overhead
of keeping it. Defferred removal to the v11 upgrade.
Sponsored-by: unqueued
It was already default in stack builds, now it is default in cabal
builds as well.
Add build warnings when git-annex is built without the OsPath build flag.
git-annex version: Report on whether it was built with the OsPath build flag.
Having the flag on by default was always the plan, and this is a good time to
make the change. A bit of added urgency comes from the close-on-exec leak
issue. Fixing that is going to need reimplentation of things like openFile.
Needing to reimplenment it twice is not very appealing, especially since the
FilePath version of it has an implementation that cannot be easily copied and
tweaked. If OsPath is on by default, I can start with only implementing
openFile for it, and fix the bug in that build. And perhaps avoid doing the
extra work that will later get thrown away when this transition finishes.
Note that at this point, Debian still needs to package file-io. Hopefully, they
will package it, rather than turning off the OsPath build flag.
This is the same as --not --in $remote, but easier to type. And the
documentation of --fast helps also document that drop can do extra work
when used without --fast.
Sponsored-by: Nicholas Golder-Manning
initremote: When onlyencryptcreds=yes is used along with embedcreds=yes,
and encryption is enabled, only encrypt the embedded creds, without
encrypting the content of the special remote.
Useful for exporttree=yes/importtree=yes remotes.
Sponsored-by: Joshua Antonishen
To match content that is recorded as present in an url.
Note that, this cannot ask remotes to provide an url using whereisKey, like
whereis does. Because preferred content expressions need to match the same
from multiple perspectives, and the remote would not always be available.
That's why the docs say "recorded as present", but still this may be
surprising to some who see an url in whereis output and are surprised they
cannot match on it.
The use of getDownloader is to strip the downloader prefix from urls like
"yt:". Note that, when OtherDownloader is used, this strips the ":" prefix,
and allows matching on those urls too.
Used protectedOutput to set up a umask that makes the socket only
accessible by the current user.
Authentication is still needed when using this option unless it is combined
with --wideopen. It was just simpler to keep authentication separate from
this.
sync: Push the current branch first, rather than a synced branch, to better
support git forges (gitlab, gitea, forgejo, etc.) which use push-to-create
with the first pushed branch becoming the default branch.
With considerable complication to filter out warning message about
receive.denyCurrentBranch when pushing to a non-bare repository. Localization
may break it in the future, but it seems like the best way to handle this. See
my comments for the gory details.
Added annex.fastcopy and remote.name.annex-fastcopy config setting. When
set, this allows the copy_file_range syscall to be used, which can eg allow
for server-side copies on NFS. (For fastest copying, also disable
annex.verify or remote.name.annex-verify.)
This is a simple implementation, that does not handle resuming as well as
it possibly could.
It can be used with both local git remotes (including on NFS), and
directory special remotes. Other types of remotes could in theory also
support it, so I've left the config documented as a general thing.
