Joey Hess
02c894958b
Merge branch 'master' of ssh://git-annex.branchable.com
2025-03-11 12:42:10 -04:00
Joey Hess
5f269513af
buffer responses to compute programs in a TQueue
...
This avoids a potential problem where the program sends several INPUT
before reading responses, so flushing the respose to the pipe could
block. It's unlikely, but seemed worth making sure it can't happen.
2025-03-11 12:40:21 -04:00
Joey Hess
0ee644b417
close off newline injection attacks against compute special remote protocol
2025-03-11 12:04:58 -04:00
Joey Hess
73a8ba5307
update
2025-03-11 11:53:14 -04:00
Joey Hess
5760a15c7c
avoid error on missing compute state in checkKey
...
This improves eg `git-annex move --to` a compute remote that does not
contain the key. Rather than erroring with "Missing compute state" when
it checks if the key is in the remote, it proceeds to trying to store to
it, which has a nice error message.
2025-03-11 11:49:47 -04:00
Joey Hess
0477a8d098
add INPUT-REQUIRED
...
Used by git-annex-compute-singularity to make addcomputed --fast work.
Also, simplified git-annex-compute-singularity; there is no need to hard
link the container into place. singularity does not care about the
extension of the container, so can just pass it the annex object file.
2025-03-11 11:46:31 -04:00
yarikoptic
89534f2890
Added a comment: just thinking out loud
2025-03-11 15:15:15 +00:00
Joey Hess
bb0bc078fc
Merge branch 'master' of ssh://git-annex.branchable.com
2025-03-11 11:13:21 -04:00
Joey Hess
b02aca8627
reorg and expand security section
2025-03-11 11:12:59 -04:00
yarikoptic
cb9c79c26c
Added a comment
2025-03-11 15:09:20 +00:00
Joey Hess
a9df446d5d
expand
2025-03-10 17:35:34 -04:00
Joey Hess
106373c53b
response
2025-03-10 16:46:55 -04:00
Joey Hess
24b6f50b79
Merge branch 'master' of ssh://git-annex.branchable.com
2025-03-10 16:42:24 -04:00
Joey Hess
e0b7653495
added git-annex-compute-singularity
...
And implemented SANDBOX, which it needs.
2025-03-10 16:41:26 -04:00
Joey Hess
657ff9a32e
compute protocol debugging
2025-03-10 15:14:59 -04:00
Joey Hess
7bda5f470c
document output files must be regular files
2025-03-10 14:15:07 -04:00
Joey Hess
f59c0d1f07
make usage an error
2025-03-10 14:13:32 -04:00
Joey Hess
9d9e34c187
compute: disallow output files that are not regular files
...
Use case where this came up is a compute program using singularity,
where the process inside the container will be allowed to write to the temp
directory, so could make eg a /etc/shadow symlink, which could then be
used to exfiltrate that from the system to wherever the annex object
might be pushed to.
It seemed better to fix this once in git-annex rather than in any such
compute program.
2025-03-10 12:55:03 -04:00
yarikoptic
f36da19adb
Added a comment
2025-03-09 01:02:55 +00:00
yarikoptic
1e6324c179
Added a comment: Any way to annotate what are input files?
2025-03-08 14:51:20 +00:00
Joey Hess
9d6c052c27
symlink, don't hardlink
...
hardlink can cause problems with unlocked files
2025-03-07 17:15:54 -04:00
Joey Hess
45d7f3ca4b
disconnect stdio for wasm binaries
2025-03-07 17:15:21 -04:00
Joey Hess
18be4910d8
use pwd and quote it
...
Seems more portable and safe
2025-03-07 16:06:37 -04:00
Joey Hess
5ef1c44e07
case
2025-03-07 16:03:35 -04:00
Joey Hess
10e36759bf
layout
2025-03-07 16:03:09 -04:00
Joey Hess
dcd7c207a8
layout
2025-03-07 16:02:43 -04:00
Joey Hess
2391c2802a
add git-annex-compute-wasmedge
2025-03-07 16:02:11 -04:00
Joey Hess
ed51924211
redirect command stdout to stderr
...
Otherwise it will be interpreted as compute program protocol
2025-03-07 16:01:27 -04:00
Joey Hess
2c6dce83de
make OUTPUT subdirs
...
Simplifies compute programs.
2025-03-07 14:57:12 -04:00
Joey Hess
b4becb7167
Merge branch 'master' of ssh://git-annex.branchable.com
2025-03-07 14:50:11 -04:00
Joey Hess
81ce4264df
compute: add response to OUTPUT
...
This allows rejecting output filenames that are outside the repository,
and also handles converting eg "-foo" to "./-foo" to prevent a command
that it's passed to interpreting the output filename as a dashed option.
2025-03-07 14:47:34 -04:00
Joey Hess
6a8e57f0e9
remove todo I just added
...
If a compute program does this, it has a security hole. Not git-annex.
2025-03-07 13:29:57 -04:00
Joey Hess
78045f8e4f
todo
2025-03-07 13:24:11 -04:00
jasonb@ab4484d9961a46440958fa1a528e0fc435599057
b0d4fe5dd0
2025-03-07 04:13:24 +00:00
yarikoptic
27ef1a47df
initial report on slow thaw
2025-03-06 22:40:35 +00:00
Joey Hess
1f59545ad0
improve
2025-03-06 14:54:05 -04:00
Joey Hess
138421449e
add git-annex-compute-imageconvert
2025-03-06 14:47:22 -04:00
Joey Hess
825a648670
prefix output with ./ in example
2025-03-06 14:42:07 -04:00
Joey Hess
b835c8c937
no longer a draft
2025-03-06 14:29:07 -04:00
Joey Hess
6f78341fbf
Merge branch 'compute'
2025-03-06 14:23:58 -04:00
Joey Hess
e952753846
preparing to merge compute
2025-03-06 14:22:45 -04:00
Joey Hess
4979df54d5
update
2025-03-06 13:34:51 -04:00
jerome.charousset@86fd8ed1bf55902989d7e70a11c38cb3a444b72d
203a730e28
Added a comment: Special use case for Scientific application
2025-03-06 17:02:22 +00:00
Joey Hess
1e9bb30c4e
update
2025-03-06 12:52:12 -04:00
Joey Hess
c6c6e2632d
avoid unncessary git-annex branch changes for recompute and addcomputed
2025-03-06 12:41:30 -04:00
Joey Hess
ccc454a791
computation progress display
2025-03-05 13:46:06 -04:00
matrss
629ab3f836
Added a comment
2025-03-05 15:40:44 +00:00
bpoldrack
9f045ed494
Added a comment
2025-03-05 14:23:57 +00:00
msz
62ab16aef3
Tag copy_file_range todo with projects/INM7 (came from our cluster)
2025-03-05 13:35:19 +00:00
msz
f1efad3b94
Added a comment: DataLad exploration of the compute on demand space
2025-03-05 13:31:41 +00:00
