Made all uses of openFd and dup set the close-on-exec flag, with a few
exceptions when starting a git-annex daemon.
Made openFdWithMode be used everywhere, rather than openFd.
Adding a new parameter to it ensures I checked everything.
And will help to make sure this gets considered in the future when
opening fds.
In lockPidFile, the only thing that keeps the pid file locked, once
daemonize re-runs the command in a new session, is that the fd is
inherited.
In Utility.LogFile.redir, the new fd it dups to does not have the
close-on-exec flag set, because this is used to set up the stdout and
stderr fds, which need to be inherited by child processes.
Same in Assistant.startDaemon where the browser gets started with the
original stdout and stderr.
This does nothing about uses of openFile and similar!
Sponsored-By: mycroft
add: Fix crash adding filenames that are exactly 21 bytes long and begin
with a utf-8 character.
Also longer filenames that start with "....." would cause the same crash.
I also audited for other calls to truncateFilePath that could truncate it
to "". Most use pathmax so are not a problem. Backend.Utilities.genKeyName
could possibly truncate it like that, but appends the md5 so would not be a
problem either.
Sponsored-by: Kevin Mueller
eg, git-annex enableremote foo encryption=none will not remove encryption,
and other encryption= settings don't change the type of encryption used.
Either of which would render data stored in a special remote inaccessible.
Probably fixes reversion introduced in
71f78fe45d.
That commit got rid of the hasEncryptionConfig check, which I think would
have detected this before. I've not gone back to verify that.
Sponsored-by: mycroft
initremote: When onlyencryptcreds=yes is used along with embedcreds=yes,
and encryption is enabled, only encrypt the embedded creds, without
encrypting the content of the special remote.
Useful for exporttree=yes/importtree=yes remotes.
Sponsored-by: Joshua Antonishen
This avoids losing the merge commit when re-running git-annex adjust in the
adjusted branch.
It also makes git-annex sync error out, rather than displaying a warning
and exiting successfully.
Sponsored-by: Leon Schuermann on Patreon
Improve behavior when there are special remotes configured with
autoenable=yes with names that conflict with other remotes.
The use of remoteList' is to avoid using the cached remote list in the case
where there are two special remotes both configured to autoenable and both
with the same name. Once the 1st is autoenabled, this makes reload the
remote list and so see the 1st, and so refuse to autoenable the second.
This adds a little bit of overhead, but it should be sufficiently small not
to need optimising.
Sponsored-by: Dartmouth College's OpenNeuro project
S3: When initremote is given the name of a bucket that already exists,
automatically set datacenter to the right value, rather than needing it to
be explicitly set.
This needs aws-0.23. But, initremote stores the datacenter value, so
a remote set up this way can be used with git-annex built with an older aws.
This is not done when signature=anonymous, because in that case,
using AWS.defaultRegion works fine for accessing buckets on other
datacenters.
It feels a bit round-about to need to do this probing. But without it,
the problem seems to be that, with a v4 signature, the location constraint
is included in the Authorization header. When that is the wrong location,
AWS S3 rejects it. I do wonder though if there is an easier way that I
am currently missing.
Sponsored-by: Dartmouth College's DANDI project
* S3: Default to signature=v4 when using an AWS endpoint, since some
AWS regions need v4 and all support it. When host= is used to specify
a different S3 host, the default remains signature=v2.
* webapp: Support setting up S3 buckets in regions that need v4
signatures.
For the webapp, went ahead and added all current S3 regions
(except govcloud, which is not usable by everyone).
Sponsored-by: Dartmouth College's DANDI project
