Commit graph

4969 commits

Author SHA1 Message Date
Joey Hess
a673fc7cfd
recompute: stage new version of file in git
When writing doc/tips/computing_annexed_files.mdwn, I noticed
that a recompute --reproducible followed by a drop and a re-get did not
actually test if the file could be reproducible computed again.

Turns out that get and drop both operate on staged files. If there is an
unstaged modification in the work tree, that's ignored. Somewhat
surprisingly, other commands like info do operate on staged files. So
behavior is inconsistent, and fairly surprising really, when there are
unstaged modifications to files.

Probably this is rarely noticed because `git-annex add` is used to add a
new version of a file, and then it's staged. Or `git mv` is used to move
a file, rather than `mv` of a file over top of an existing file. So it's
uncommon to have an unstaged annexed file in a worktree.

It might be worth making things more consistent, but that's out of scope
for what I'm working on currently.

Also, I anticipate that supporting unlocked files with recompute will
require it to stage changes anyway.

So, make recompute stage the new version of the file.

I considered having recompute refuse to overwrite an existing staged
file. After all, whatever version was staged before will get lost when
the new version is staged over top of it. But, that's no different than
`git-annex addcomputed` being run with the name of an existing staged
file. Or `git-annex add` being run with a new file content when there is
an existing staged file. Or, for that matter, `git add` being ran with a
new content when there is an existing staged file.
2025-03-12 13:42:00 -04:00
Joey Hess
21b45da406
todo 2025-03-12 12:11:39 -04:00
Joey Hess
5f269513af
buffer responses to compute programs in a TQueue
This avoids a potential problem where the program sends several INPUT
before reading responses, so flushing the respose to the pipe could
block. It's unlikely, but seemed worth making sure it can't happen.
2025-03-11 12:40:21 -04:00
Joey Hess
0ee644b417
close off newline injection attacks against compute special remote protocol 2025-03-11 12:04:58 -04:00
Joey Hess
73a8ba5307
update 2025-03-11 11:53:14 -04:00
Joey Hess
0477a8d098
add INPUT-REQUIRED
Used by git-annex-compute-singularity to make addcomputed --fast work.

Also, simplified git-annex-compute-singularity; there is no need to hard
link the container into place. singularity does not care about the
extension of the container, so can just pass it the annex object file.
2025-03-11 11:46:31 -04:00
Joey Hess
e0b7653495
added git-annex-compute-singularity
And implemented SANDBOX, which it needs.
2025-03-10 16:41:26 -04:00
Joey Hess
6a8e57f0e9
remove todo I just added
If a compute program does this, it has a security hole. Not git-annex.
2025-03-07 13:29:57 -04:00
Joey Hess
78045f8e4f
todo 2025-03-07 13:24:11 -04:00
Joey Hess
6f78341fbf
Merge branch 'compute' 2025-03-06 14:23:58 -04:00
Joey Hess
e952753846
preparing to merge compute 2025-03-06 14:22:45 -04:00
bpoldrack
9f045ed494 Added a comment 2025-03-05 14:23:57 +00:00
msz
62ab16aef3 Tag copy_file_range todo with projects/INM7 (came from our cluster) 2025-03-05 13:35:19 +00:00
msz
f1efad3b94 Added a comment: DataLad exploration of the compute on demand space 2025-03-05 13:31:41 +00:00
msz
e4232791dd Added a comment 2025-03-05 11:27:39 +00:00
yarikoptic
b567aac217 map --json wishlist 2025-02-21 15:31:35 +00:00
Spencer
4414e97b9b Added a comment: For Those Who Stumble Here 2025-02-19 23:08:42 +00:00
Joey Hess
a2fa2a8c5f
update 2025-02-19 16:03:34 -04:00
Joey Hess
2f11c65491
comments 2025-02-19 15:14:52 -04:00
Joey Hess
e8b00faea8
Merge branch 'master' into ospath 2025-02-14 16:28:43 -04:00
Joey Hess
e6e69f8f93
draft 2025-02-13 16:12:07 -04:00
Joey Hess
bf6446528d
comment 2025-02-13 13:51:21 -04:00
Joey Hess
2ff0adba0b
comment 2025-02-13 13:01:15 -04:00
Joey Hess
bab26da74b
Merge branch 'master' into ospath 2025-02-11 16:56:17 -04:00
Joey Hess
90eb1e2da6
update todo 2025-02-11 13:01:13 -04:00
thk
09d47726b7 Added a comment: iroh 2025-02-08 06:56:32 +00:00
Joey Hess
a03c609268
Merge branch 'master' into ospath 2025-01-30 15:07:17 -04:00
Joey Hess
c3e6cbb432
update 2025-01-30 15:06:37 -04:00
Joey Hess
94e177db9d
document a SNAFU 2025-01-29 19:04:13 -04:00
Joey Hess
cbb6df35aa
merge in doc changes from master 2025-01-29 18:57:25 -04:00
Joey Hess
76bff3e8d1
Merge branch 'master' into ospath 2025-01-29 14:29:34 -04:00
matrss
1e0a48fad0 Added a comment 2025-01-29 10:13:59 +00:00
matrss
acdefd77a6 Added a comment 2025-01-29 09:56:13 +00:00
Joey Hess
917c43f31f
Merge /home/joey/tmp/git-annex into ospath 2025-01-28 15:29:58 -04:00
Joey Hess
da9ca7475e
comment 2025-01-28 11:57:03 -04:00
Joey Hess
67034a02ea
update 2025-01-28 11:38:04 -04:00
Joey Hess
4f0e64b6de
update 2025-01-28 11:36:02 -04:00
Joey Hess
24d5dbe30b
comment 2025-01-28 11:12:02 -04:00
Joey Hess
6fb1dd6afa
comment 2025-01-28 10:28:35 -04:00
Joey Hess
0529f9a84a
Merge branch 'master' of ssh://git-annex.branchable.com 2025-01-27 12:23:13 -04:00
Joey Hess
754c0a001b
comment 2025-01-27 12:19:16 -04:00
matrss
7adf1f45fa Added a comment 2025-01-27 15:26:15 +00:00
Joey Hess
02c792b724
thoughts 2025-01-27 10:37:35 -04:00
Joey Hess
71206a8603
update comment 2025-01-27 10:25:55 -04:00
Joey Hess
b61d316c36
fix link 2025-01-27 09:35:33 -04:00
Joey Hess
f9d42c37c0
more OsPath conversion
Finally reached Annex code in this conversion.

Sponsored-by: Graham Spencer
2025-01-25 10:54:51 -04:00
Joey Hess
c1ecb80716
update 2025-01-23 10:27:18 -04:00
Joey Hess
77e9781ae2
parsePOSIXTime ByteString conversion
Some easy (though tiny) speed wins.

Sponsored-by: Luke T. Shumaker on Patreon
2025-01-22 16:42:09 -04:00
Joey Hess
6e27b0d4d1
convert from readFileStrict
This removes that function, using file-io readFile' instead.

Had to deal with newline conversion, which readFileStrict does on
Windows. In a few cases, that was pretty ugly to deal with.

Sponsored-by: Kevin Mueller
2025-01-22 16:20:36 -04:00
Joey Hess
9b79f0f43d
use file-io for readFile/writeFile/appendFile on ByteStrings
These are all straightforward, and easy small performance wins.

Sponsored-by: Nicholas Golder-Manning
2025-01-22 14:30:25 -04:00