Commit graph

42775 commits

Author SHA1 Message Date
Joey Hess
26dea5641e
avoid name shadowing warning 2022-09-26 16:20:56 -04:00
Joey Hess
7059322a6c
Support "inbackend" in preferred content expressions
Well, actually, fix a typo that has always been in the implementation of
that. "inbacked" used to work, but let's not tell users about that; they
might try to use it and expect git-annex to keep supporting the typo..

Sponsored-by: Jack Hill on Patreon
2022-09-26 16:06:49 -04:00
Joey Hess
640e2646a1
add comment 2022-09-26 16:05:46 -04:00
Joey Hess
dcc2957d9c
improve documentation about backends
I noticed that, using just the man pages, there is no real description
of what backends are, or what ones are available. Except for some
examples.

Added a git-annex-backends man page, that is just a stub, but at least
describes what they basically are, and tells how to find the supported
ons, and links to the backends web page.

Sponsored-by: Brett Eisenberg on Patreon
2022-09-26 15:59:10 -04:00
Joey Hess
f1c85ac11b
fix breakage in wormhole's sendFile
Commit ff0927bde9 broke this, it made it
try to read all of the input before looking for the code. But, wormhole
keeps running until it sends the file, so that caused a deadlock. Oops.

Sponsored-by: Luke Shumaker on Patreon
2022-09-26 15:26:29 -04:00
Joey Hess
17129fed66
fix wormhole --appid option position
p2p: Pass wormhole the --appid option before the receive/send command, as
it does not accept that option after the command

I'm left wondering, did I get this wrong from the beginning, or did
wormhole change its option parser? I'm reminded of the change in 0.8.2
where it silently changed what FD the pairing code was output to.
But, looking at the wormhole source, it was at least putting --appid before
send in its test suite from the introduction of the option.

So I think probably this has always been broken. On 2021-12-31 the --appid
option was enabled, and it took until now for someone to try
git-annex p2p --pair and notice that flag day broke it..

Sponsored-by: Svenne Krap on Patreon
2022-09-26 15:11:38 -04:00
Joey Hess
ce65f11de0
enable-tor: Fix breakage caused by git's fix for CVE-2022-24765
This relies on bfa451fc4e and is a bit of an
ugly hack.

Sponsored-by: Noam Kremen on Patreon
2022-09-26 14:48:58 -04:00
Joey Hess
bfa451fc4e
pass --git-dir when reading git config when it was specified explicitly
Let GIT_DIR and --git-dir override git's protection against operating in a
repository owned by another user.

This is the same behavior other git commands have.

Sponsored-by: Jarkko Kniivilä on Patreon
2022-09-26 14:38:34 -04:00
Joey Hess
1d47a7e7e6
Revert "fix comment"
This reverts commit c39b057b2d.
2022-09-26 13:39:10 -04:00
Joey Hess
c39b057b2d
fix comment 2022-09-26 13:37:47 -04:00
Joey Hess
fc9212e6ea
comment 2022-09-26 13:25:04 -04:00
Joey Hess
6e21bd54d4
comment 2022-09-26 13:23:04 -04:00
Joey Hess
79aadf63d4
changelog and close 2022-09-26 13:11:23 -04:00
Joey Hess
2e40eb07db
note that hooks are also run when on a crippled filesystem now 2022-09-26 13:10:47 -04:00
Joey Hess
b411a1ce74
remove unncessary do block
Left by Reiko's patch
2022-09-26 13:10:25 -04:00
Reiko Asakura
1d48153bb8
Run freeze and thaw hooks on crippled filesystems
The user sets these hooks deliberately so they should always be run. For
example this allows hooks to be used to manage file permissions on NTFS
volumes in WSL1.
2022-09-26 13:05:39 -04:00
Joey Hess
e05dd70544
handle upgrading repositories initialized with --version=9
As was attempted earlier in the buggy commit 0d2e3058ee

Avoided the bug that had by making the upgrade log be updated after each
upgrade step. So, after upgrade from v8 to v9, the log is updated, and
so Upgrade.V9's timeOfUpgrade check will find that it was upgraded
recently and so won't let it skip ahead to v10.

Sponsored-by: k0ld on Patreon
2022-09-26 12:59:51 -04:00
Joey Hess
8230f4a6f1
changelog for problem fixed by earlier revert 2022-09-26 12:27:32 -04:00
Joey Hess
6b658a7f54
comment 2022-09-26 12:20:11 -04:00
Joey Hess
d88f3c6ad8
Revert "handle upgrading repositories initialized with --version=9"
This reverts commit 0d2e3058ee.

This commit accidentially caused repos that were initialized at v8 to
also upgrade to v10. There is nothing in the upgrade.log for such a
repo..
2022-09-26 12:17:50 -04:00
Joey Hess
98eb5ff84f
fix windows build 2022-09-26 12:08:04 -04:00
Joey Hess
aff5e4376b
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-26 12:07:46 -04:00
yarikoptic
87b62210d7 report on windows FTBFS 2022-09-26 15:47:23 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
d1762cc8e4 Added a comment 2022-09-25 19:43:31 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
f105bed5d9 2022-09-25 19:39:22 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
ba6eabd76b 2022-09-25 02:01:26 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
cc9634ce12 2022-09-24 23:28:09 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
5c6b875f10 Fix some markdown formatting problems 2022-09-24 23:27:17 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
00843b296b Add patches for WSL1 2022-09-24 23:23:40 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
095a7bf800 Fix WSL1 instructions 2022-09-24 22:31:54 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
bc26c85745 close bug "git commit smudges unncessarily" 2022-09-24 20:37:56 +00:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
a7e3e9d0c9 Added a comment 2022-09-24 19:43:47 +00:00
dukeofcool199
a887de360b 2022-09-24 19:24:19 +00:00
dukeofcool199
3a3e17cee5 2022-09-24 19:21:09 +00:00
Joey Hess
e62e4eaaf2
refector for legibility 2022-09-23 18:53:06 -04:00
Joey Hess
2478e9e03a
restage: New git-annex command, handles restaging unlocked files
This is much easier and less failure-prone than having the user run
git update-index --refresh themselves.

Sponsored-by: Dartmouth College's DANDI project
2022-09-23 16:29:59 -04:00
Joey Hess
f7146c153b
fix restaging of transferred files after stalldetection kicks in
Sponsored-by: Dartmouth College's DANDI project
2022-09-23 15:55:40 -04:00
Joey Hess
6a3bd283b8
add restage log
When pointer files need to be restaged, they're first written to the
log, and then when the restage operation runs, it reads the log. This
way, if the git-annex process is interrupted before it can do the
restaging, a later git-annex process can do it.

Currently, this lets a git-annex get/drop command be interrupted and
then re-ran, and as long as it gets/drops additional files, it will
clean up after the interrupted command. But more changes are
needed to make it easier to restage after an interrupted process.

Kept using the git queue to run the restage action, even though the
list of files that it builds up for that action is not actually used by
the action. This could perhaps be simplified to make restaging a cleanup
action that gets registered, rather than using the git queue for it. But
I wasn't sure if that would cause visible behavior changes, when eg
dropping a large number of files, currently the git queue flushes
periodically, and so it restages incrementally, rather than all at the
end.

In restagePointerFiles, it reads the restage log twice, once to get
the number of files and size, and a second time to process it.
This seemed better than reading the whole file into memory, since
potentially a huge number of files could be in there. Probably the OS
will cache the file in memory and there will not be much performance
impact. It might be better to keep running tallies in another file
though. But updating that atomically with the log seems hard.

Also note that it's possible for calcRestageLog to see a different file
than streamRestageLog does. More files may be added to the log in
between. That is ok, it will only cause the filterprocessfaster heuristic to
operate with slightly out of date information, so it may make the wrong
choice for the files that got added and be a little slower than ideal.

Sponsored-by: Dartmouth College's DANDI project
2022-09-23 15:47:24 -04:00
Joey Hess
9c76e503cf
generalize refreshIndex to MonadIO
Sponsored-by: Dartmouth College's DANDI project
2022-09-23 14:28:52 -04:00
Joey Hess
b17e328175
avoid unncessary locking by checkLogFile
Like the comment says, this works without locking. It looks like I
originally copied another function and forgot to remove the locking.

Sponsored-by: Dartmouth College's DANDI project
2022-09-23 14:01:43 -04:00
Joey Hess
ed8afacc39
add a finalizer to streamLogFile
Sponsored-by: Dartmouth College's DANDI project
2022-09-23 13:49:01 -04:00
Joey Hess
8718125ae4
refactor the restage runner
Sponsored-by: Dartmouth College's DANDI project
2022-09-23 13:12:17 -04:00
nick.guenther@e418ed3c763dff37995c2ed5da4232a7c6cee0a9
4822121728 Added a comment 2022-09-23 01:40:56 +00:00
Joey Hess
f64eff9355
test: Added --test-with-git-config option
Sponsored-by: Dartmouth College's DANDI project
2022-09-22 15:58:45 -04:00
Joey Hess
d049228fd3
comment and todo 2022-09-22 14:48:32 -04:00
Joey Hess
0139a732fe
Merge branch 'master' of ssh://git-annex.branchable.com 2022-09-22 14:40:29 -04:00
Joey Hess
6e3c9bea2e
drain transferrer read handle when shutting it down
Fixes updating git index file after getting an unlocked file when
annex.stalldetection is set.

The transferrer may want to send additional protocol messages when it's
shut down. Closing the read handle prevented it from doing that, and caused
it to crash rather than cleanly shutting down.

Draining the handle without processing the protocol seemed ok to do,
because anything it outputs is going to be some side message displayed
at shutdown. Displaying those once per transferrer process that is running
seems unncessary.

Sponsored-by: Dartmouth College's DANDI project
2022-09-22 14:39:39 -04:00
yarikoptic
ea4a466b8b Added a comment 2022-09-22 18:14:15 +00:00
Joey Hess
f015f3e8e3
comment 2022-09-22 14:03:31 -04:00
Joey Hess
5ed774b701
comment 2022-09-22 13:40:13 -04:00