git-annex/Remote
Joey Hess 202ea3ff2a
don't sync with cluster nodes by default
Avoid `git-annex sync --content` etc from operating on cluster nodes by default
since syncing with a cluster implicitly syncs with its nodes. This avoids a
lot of unncessary work when a cluster has a lot of nodes just in checking
if each node's preferred content is satisfied. And it avoids content
being sent to nodes individually, so instead syncing with clusters always
fanout uploads to nodes.

The downside is that there are situations where a cluster's preferred content
settings can be met, but those of its nodes are not. Or where a node does not
contain a key, but the cluster does, and there are not enough copies of the key
yet, so it would be desirable the send it there. I think that's an acceptable
tradeoff. These kind of situations are ones where the cluster itself should
probably be responsible for copying content to the node. Which it can do much
less expensively than a client can. Part of the balanced preferred content
design that I will be working on in a couple of months involves rebalancing
clusters, so I expect to revisit this.

The use of annex-sync config does allow running git-annex sync with a specific
node, or nodes, and it will sync with it. And it's also possible to set
annex-sync git configs to make it sync with a node by default. (Although that
will require setting up an explicit git remote for the node rather than relying
on the proxied remote.)

Logs.Cluster.Basic is needed because Remote.Git cannot import Logs.Cluster
due to a cycle. And the Annex.Startup load of clusters happens
too late for Remote.Git to use that. This does mean one redundant load
of the cluster log, though only when there is a proxy.
2024-06-25 10:24:38 -04:00
..
Directory improve createDirectoryUnder to allow alternate top directories 2022-08-12 12:52:37 -04:00
External rclone special remote 2024-04-17 15:20:37 -04:00
Helper add annex-proxied 2024-06-24 10:16:59 -04:00
List auto-init autoenable=yes 2020-05-27 12:40:35 -04:00
Rsync add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
WebDAV Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Adb.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
BitTorrent.hs support VURL backend 2024-02-29 13:48:51 -04:00
Borg.hs implement Unavilable for borg bup ddar directory rsync 2023-08-16 15:48:09 -04:00
Bup.hs implement Unavilable for borg bup ddar directory rsync 2023-08-16 15:48:09 -04:00
Ddar.hs implement Unavilable for borg bup ddar directory rsync 2023-08-16 15:48:09 -04:00
Directory.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
External.hs avoid accepting externaltype= and readonly= parameters for rclone 2024-04-17 15:41:55 -04:00
GCrypt.hs eliminate s2w8 and w82s 2023-10-26 13:12:57 -04:00
Git.hs don't sync with cluster nodes by default 2024-06-25 10:24:38 -04:00
GitLFS.hs Removed the vendored git-lfs and the GitLfs build flag 2023-08-28 13:12:31 -04:00
Glacier.hs let Remote.availability return Unavilable 2023-08-16 14:31:31 -04:00
Hook.hs let Remote.availability return Unavilable 2023-08-16 14:31:31 -04:00
HttpAlso.hs let Remote.availability return Unavilable 2023-08-16 14:31:31 -04:00
List.hs clarify comment 2024-06-04 14:29:24 -04:00
P2P.hs dropping from clusters 2024-06-23 09:43:40 -04:00
Rclone.hs rclone special remote 2024-04-17 15:20:37 -04:00
Rsync.hs export tree: avoid confusing output about renaming files 2024-03-09 13:50:26 -04:00
S3.hs export tree: avoid confusing output about renaming files 2024-03-09 13:50:26 -04:00
Tahoe.hs disk free checking for unsized keys 2024-01-16 14:29:10 -04:00
Web.hs implement URL to VURL migration 2024-03-01 16:42:02 -04:00
WebDAV.hs export tree: avoid confusing output about renaming files 2024-03-09 13:50:26 -04:00