git-annex/Logs
Joey Hess 202ea3ff2a
don't sync with cluster nodes by default
Avoid `git-annex sync --content` etc from operating on cluster nodes by default
since syncing with a cluster implicitly syncs with its nodes. This avoids a
lot of unncessary work when a cluster has a lot of nodes just in checking
if each node's preferred content is satisfied. And it avoids content
being sent to nodes individually, so instead syncing with clusters always
fanout uploads to nodes.

The downside is that there are situations where a cluster's preferred content
settings can be met, but those of its nodes are not. Or where a node does not
contain a key, but the cluster does, and there are not enough copies of the key
yet, so it would be desirable the send it there. I think that's an acceptable
tradeoff. These kind of situations are ones where the cluster itself should
probably be responsible for copying content to the node. Which it can do much
less expensively than a client can. Part of the balanced preferred content
design that I will be working on in a couple of months involves rebalancing
clusters, so I expect to revisit this.

The use of annex-sync config does allow running git-annex sync with a specific
node, or nodes, and it will sync with it. And it's also possible to set
annex-sync git configs to make it sync with a node by default. (Although that
will require setting up an explicit git remote for the node rather than relying
on the proxied remote.)

Logs.Cluster.Basic is needed because Remote.Git cannot import Logs.Cluster
due to a cycle. And the Annex.Startup load of clusters happens
too late for Remote.Git to use that. This does mean one redundant load
of the cluster log, though only when there is a proxy.
2024-06-25 10:24:38 -04:00
..
Chunk
Cluster don't sync with cluster nodes by default 2024-06-25 10:24:38 -04:00
ContentIdentifier simplify base64 to only use ByteString 2023-10-26 13:10:05 -04:00
Difference
Export Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
MetaData
PreferredContent start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
Presence migrate --apply 2023-12-08 13:23:46 -04:00
Remote
SingleValue deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Trust newtype MapLog 2023-11-13 14:37:22 -04:00
Activity.hs Future proof activity log parsing 2021-06-14 14:18:19 -04:00
AdjustedBranchUpdate.hs sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
Chunk.hs newtype MapLog 2023-11-13 14:37:22 -04:00
Cluster.hs don't sync with cluster nodes by default 2024-06-25 10:24:38 -04:00
Config.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
ContentIdentifier.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
Difference.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
EquivilantKeys.hs run codespell throughout fixing typos automagically 2024-05-01 15:46:21 -04:00
Export.hs Merge branch 'master' into proxy 2024-06-10 14:26:18 -04:00
File.hs Windows: Fix CRLF handling in some log files 2023-10-30 14:23:23 -04:00
FsckResults.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Group.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
Import.hs implement importChanges optimisaton (not used yet) 2023-05-31 16:01:34 -04:00
Line.hs
Location.hs remove dead nodes when loading the cluster log 2024-06-16 14:39:44 -04:00
MapLog.hs newtype MapLog 2023-11-13 14:37:22 -04:00
MetaData.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Migrate.hs migrate --apply 2023-12-08 13:23:46 -04:00
Multicast.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
NumCopies.hs cache negative lookups of global numcopies and mincopies 2023-06-06 14:43:25 -04:00
PreferredContent.hs importtree: support preferred content expressions needing keys 2023-12-18 16:27:59 -04:00
Presence.hs migrate --apply 2023-12-08 13:23:46 -04:00
Proxy.hs add git-annex updatecluster command 2024-06-14 15:02:01 -04:00
Remote.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
RemoteState.hs newtype MapLog 2023-11-13 14:37:22 -04:00
Restage.hs fix deadlock in restagePointerFiles 2022-12-08 14:36:11 -04:00
Schedule.hs start implementing hidden git-annex repositories 2021-04-20 15:04:53 -04:00
SingleValue.hs deal better with clock skew situations, using vector clocks 2021-08-04 12:33:46 -04:00
Smudge.hs add a finalizer to streamLogFile 2022-09-23 13:49:01 -04:00
Transfer.hs fix build on windows 2024-03-26 13:12:58 -04:00
Transitions.hs fix obviously wrong attoparsec parser 2022-02-07 14:15:17 -04:00
Trust.hs info: Added --autoenable option 2022-06-01 14:20:38 -04:00
Unused.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Upgrade.hs add annex.dbdir (WIP) 2022-08-11 16:58:53 -04:00
UUID.hs one-way escaping of newlines in uuid.log 2023-03-13 14:19:32 -04:00
UUIDBased.hs newtype MapLog 2023-11-13 14:37:22 -04:00
View.hs understand adjusted view branch names 2023-02-27 14:57:58 -04:00
Web.hs migrate --apply 2023-12-08 13:23:46 -04:00