Commit graph

1796 commits

Author SHA1 Message Date
Joey Hess
c15dad6040
improve AuthToken display in P2P --debug
Using an empty string to obscure the AuthToken made it impossible to
tell if one was really being presented or not.
2025-08-01 12:58:03 -04:00
Joey Hess
f1781d01d8
remotedaemon support for generic P2P transports
RemoteDaemon.Transport.Tor was refactored into this, and most of the
code is reused between them.

getSocketFile does not yet deal with repositories on crippled
filesystems that don't support sockets. Annex.Ssh detects that and
allows the user to set an environment variable, and something similar
could be done here.

And it does not deal with a situation where there is no path to the
socket file that is not too long. In that situation it would crash out
I suppose. Probably though, remotedaemon is ran from the top of the
repo, and in that case the path is just ".git/annex/p2p/<md5>" so nice
and short.

This seems to mostly work. But I don't yet have a working git-annex-p2p-
command to test it with.

And with my not quite working git-annex-p2p-foo test script, running
remotedaemon results in an ever-growing number of zombie processes
that it's not waiting on.
2025-07-31 14:45:32 -04:00
Joey Hess
2007507e2b
prevent relatedTemplate from truncating a filename to end in whitespace
Avoid a problem with temp file names ending in whitespace on filesystems
like VFAT that don't support such filenames.

See a6eb7d7339 previously for the same but
with "."

At some point relatedTemplate is more bother than it's worth and it would
be simpler to just use "temp" as the basename of all temp files. We seem to
be approaching that point, since my interest in absurd ancient filesystem
limitations is limited.

Sponsored-by: unqueued on Patreon
2025-06-23 13:31:15 -04:00
Joey Hess
73060eea51
annex.fastcopy
Added annex.fastcopy and remote.name.annex-fastcopy config setting. When
set, this allows the copy_file_range syscall to be used, which can eg allow
for server-side copies on NFS. (For fastest copying, also disable
annex.verify or remote.name.annex-verify.)

This is a simple implementation, that does not handle resuming as well as
it possibly could.

It can be used with both local git remotes (including on NFS), and
directory special remotes. Other types of remotes could in theory also
support it, so I've left the config documented as a general thing.
2025-06-03 15:01:38 -04:00
Joey Hess
b111009868
moveFile on windows forgot to delete src file in fallback case
This dates back to commit 625303226d,
where a cross-device moveFile on Windows was made to fall back to copying
to the destination, but forgot to delete the source file.

Should fix the following test suite failure on Windows:

    import:                                FAIL (2.52s)
      .\Test\Framework.hs:383:
      C:\Users\RUNNER~1\AppData\Local\Temp\importtest.0\import1\f exists unexpectedly
      Use -p '/import/' to rerun this test only.

Which was seen here, running the test suite in the github action environment.
https://github.com/psychoinformatics-de/git-annex-wheel/issues/5
2025-05-30 13:18:26 -04:00
Joey Hess
dbc00906bf
Fix build without the assistant. 2025-03-26 10:17:03 -04:00
Joey Hess
81ce4264df
compute: add response to OUTPUT
This allows rejecting output filenames that are outside the repository,
and also handles converting eg "-foo" to "./-foo" to prevent a command
that it's passed to interpreting the output filename as a dashed option.
2025-03-07 14:47:34 -04:00
Joey Hess
9f2ce19858
OSX build fix 2025-02-17 14:01:54 -04:00
Joey Hess
66b8ba8fb0
OSX build fixes 2025-02-17 13:59:52 -04:00
Joey Hess
25e4f84e8f
push down OsPath into CopyFile 2025-02-12 13:11:27 -04:00
Joey Hess
0f47eceacf
stop exporting RawFilePath 2025-02-12 12:59:30 -04:00
Joey Hess
fb0ecb997f
Merge branch 'ospathwin2' into ospath 2025-02-11 16:46:01 -04:00
Joey Hess
6653e000f0 fix convertToWindowsNativeNameSpace bug
This fixes a test suite failure. The OsPath conversion made that be used
in more places, including addurl, which exposed an existing bug.
2025-02-11 20:37:40 -08:00
Joey Hess
769c7b3e65
avoid build warning on windows 2025-02-11 16:31:35 -04:00
Joey Hess
a149336a59
OsPath transition Windows build fixes
This gets it building on Windows again, with 1 test suite failure
(addurl).

Sponsored-by: Kevin Mueller
2025-02-11 15:40:53 -04:00
Joey Hess
19dea33f57
more OsPath conversion 2025-02-11 14:00:01 -04:00
Joey Hess
a5eaf3d4d2
use to/fromOsPath
Just to reduce the number of from/toRawFilePath calls, which I would
like to minimize.

In this build path, the two are the same though.
2025-02-11 13:54:51 -04:00
Joey Hess
780a379ab1
remove unused functions from Utility.RawFilePath 2025-02-11 13:49:17 -04:00
Joey Hess
c85d5a0dc8
OsPath conversion of linuxstandalone builder
Sponsored-by: Joshua Antonishen
2025-02-11 12:12:27 -04:00
Joey Hess
2ff716be30
OsPath build flag no longer depends on filepath-bytestring
However, filepath-bytestring is still in Setup-Depends.
That's because Utility.OsPath uses it when not built with OsPath.
It would be maybe possible to make Utility.OsPath fall back to using
filepath, and eliminate that dependency too, but it would mean either
wrapping all of System.FilePath's functions, or using `type OsPath = FilePath`

Annex.Import uses ifdefs to avoid converting back to FilePath when not
on windows. On windows it's a bit slower due to that conversion.
Utility.Path.Windows.convertToWindowsNativeNamespace got a bit
slower too, but not really worth optimising I think.

Note that importing Utility.FileSystemEncoding at the same time as
System.Posix.ByteString will result in conflicting definitions for
RawFilePath. filepath-bytestring avoids that by importing RawFilePath
from System.Posix.ByteString, but that's not possible in
Utility.FileSystemEncoding, since Setup-Depends does not include unix.
This turned out not to affect any code in git-annex though.

Sponsored-by: Leon Schuermann
2025-02-10 16:39:55 -04:00
Joey Hess
c730d00b6e
more OsPath conversion (749/749)
Builds with and without OsPath build flag.

Unfortunately, the test suite fails.

Sponsored-by: unqueued on Patreon
2025-02-10 14:59:20 -04:00
Joey Hess
20ed039d59
don't export pack and unpack
These are too widly used for other things to make sense to export OsPath
versions of them. And OsString also provides them and gets imported
qualified when needed.
2025-02-10 12:34:13 -04:00
Joey Hess
2d224e0d28
more OsPath conversion (658/749)
At this point the test suite builds, and mostly the assistant is left.

Sponsored-by: unqueued
2025-02-08 15:27:44 -04:00
Joey Hess
c74c75b352
more OsPath conversion (639/749)
Sponsored-by: k0ld
2025-02-07 16:07:05 -04:00
Joey Hess
a5d48edd94
more OsPath conversion (602/749)
Sponsored-by: Brock Spratlen
2025-02-07 14:46:11 -04:00
Joey Hess
2d1db7986c
more OsPath conversion (572/749)
Sponsored-by: Jack Hill
2025-02-06 16:18:52 -04:00
Joey Hess
0811531b59
more OsPath conversion (542/749)
Sponsored-by: Luke T. Shumaker
2025-02-06 11:38:14 -04:00
Joey Hess
9394197621
Merge branch 'master' into ospath 2025-02-05 13:31:07 -04:00
Joey Hess
4dc904bbad
more OsPath conversion
Sponsored-by: Leon Schuermann
2025-02-04 16:09:47 -04:00
Joey Hess
5cc8d9d03b
replace removeLink with removeFile
removeFile calls unlink so removes anything not a directory. So these
are replaceable in order to convert to OsPath.
2025-02-02 14:16:58 -04:00
Joey Hess
71195cce13
more OsPath conversion
Sponsored-by: k0ld
2025-02-01 14:06:38 -04:00
Joey Hess
474cf3bc8b
more OsPath conversion
Sponsored-by: Brock Spratlen
2025-02-01 11:54:19 -04:00
Joey Hess
0c0f4a20f4
move windows test suite workaround to relatedTemplate 2025-01-30 19:06:06 -04:00
Joey Hess
773115fd5f
fix truncateFilePath edge case on windows
If the filepath starts with something that is not valid utf-8, it would
have returned "". And if the filepath was all non-valid utf-8, it would
also return "".
2025-01-30 16:53:10 -04:00
Joey Hess
c69e57aede
more OsPath conversion
Sponsored-by: Jack Hill
2025-01-30 15:46:32 -04:00
Joey Hess
96c953a4b7
Merge branch 'master' into ospath 2025-01-30 15:06:05 -04:00
Joey Hess
c2a48f1fa0
Merge branch 'ospath-mk1' 2025-01-30 15:05:18 -04:00
Joey Hess
c926de73b9
avoid build warning
An odd one, it complained about security_attributes being defined but
not used, but it was used..
2025-01-30 14:52:41 -04:00
Joey Hess
3a08881214
avoid build warnings on windows 2025-01-30 14:50:45 -04:00
Joey Hess
84291b6014
bring back OsPath changes
I hope that the windows test suite failure on appveyor was fixed by
updating to a newer windows there. I have not been able to reproduce
that failure in a windows 11 VM run locally.
2025-01-30 14:34:21 -04:00
Joey Hess
bd5d2b0782
fix FileIO openTempFile on Windows
When an UNC-style path is passed into openTempFile, the returned file
starts with that same style of path. Which can cause problems, eg piping
that filename to git failed. So, convert the output filename to be
relative to the input temp directory.
2025-01-30 14:32:14 -04:00
Joey Hess
c1e90767da
fix FileIO openTempFile on Windows
When an UNC-style path is passed into openTempFile, the returned file
starts with that same style of path. Which can cause problems, eg piping
that filename to git failed. So, convert the output filename to be
relative to the input temp directory.
2025-01-30 14:31:01 -04:00
Joey Hess
aaf4dd3b9c
fix liuxstandalone build 2025-01-29 18:12:00 -04:00
Joey Hess
c309edb8fb
Merge branch 'master' into ospath 2025-01-29 15:37:19 -04:00
Joey Hess
1b7293fb67
fix windows build 2025-01-29 15:36:41 -04:00
Joey Hess
4d7bdd0047
windows build fix 2025-01-29 14:59:00 -04:00
Joey Hess
4e999af7ac
fix windows build and clarify comment 2025-01-29 14:52:20 -04:00
Joey Hess
76bff3e8d1
Merge branch 'master' into ospath 2025-01-29 14:29:34 -04:00
Joey Hess
2940cfcd18
fix fromOsPath on windows
Same reasons as 9c9baf7c65
2025-01-29 14:19:25 -04:00
Joey Hess
9c9baf7c65
fix toOsPath on windows
After recent changes to use OsPath, test suite fails on windows with:

Exception: Uneven number of bytes: 5. This is not a Word16 bytestream.

Hopefully this fixes it. The innefficiency of the conversion is
unfortunate.
2025-01-29 14:06:54 -04:00