Measuring route_controller_route_sync_total in Kubernetes

What the counter tells you about route reconciliation

With the default fixed-interval loop, the counter keeps climbing even when nothing in the node set changes. That is the normal shape of a polling controller: it keeps asking the cloud provider the same question until you get bored or the quota bill starts looking rude.

When CloudControllerManagerWatchBasedRoutesReconciliation is enabled, the route controller behaves differently. The counter only moves when node objects change, so a quiet cluster should look quiet. A rising line with no node activity is a sign that the interval loop is still in charge.

Fixed-interval syncs keep the number moving even when nodes stay put

In fixed-interval mode, the value will increase on schedule whether the cluster has been busy or not. Ten minutes with no node changes and a rising counter is perfectly consistent with the old behaviour. That makes the metric useful for proving that the controller is still polling, not waiting on events.

If you are checking a live cluster, look for a steady slope rather than spikes. A route controller that syncs every few seconds or minutes will paint a regular staircase even when the cluster is otherwise flat.

Watch-based reconciliation only bumps the metric when nodes change

Watch-based reconciliation ties route syncs to node add, remove, and update events. That means the counter should stay still for long periods if the node set does not move. When it does jump, the jump should line up with a real cluster event rather than an arbitrary timer.

That pattern is the point of the feature. It cuts down needless cloud provider API calls and makes the controller less eager to spend quota on nothing much.

Checking the Cloud Controller Manager against real cluster activity

The only useful way to read the metric is against actual node churn. A counter value by itself tells you very little. A counter value beside node events tells you whether the Cloud Controller Manager is polling or watching.

Compare the metric rate before and after enabling CloudControllerManagerWatchBasedRoutesReconciliation

Run the same cluster through both modes and compare the rate of route_controller_route_sync_total. With the gate off, the rate should keep moving at the controller’s normal interval. With the gate on, the rate should flatten during quiet periods and jump only when node events arrive.

A clean A/B check is easy to spot in a graph. The disabled case produces a steady rise. The enabled case should look sparse, with long flat sections and small jumps when the node set changes.

Read the change pattern against node add, remove, and update events

The counter should track the moments when the route table actually needs a refresh. A new node joining the cluster should cause a sync. A node leaving should do the same. A node update can also trigger reconciliation, which is useful when labels or addresses change.

If the counter rises without matching node activity, the watch-based path is not doing what you want. If the counter stays flat while nodes are changing, the controller is missing events or the feature gate is not in the state you think it is.

Using the metric to judge API pressure and rollout safety

The metric is not just a neat number for dashboards. It is a crude but useful way to spot cloud provider API pressure before the quota starts complaining. Route reconciliation that happens for no real reason eats calls for no real gain.

Spot unnecessary cloud provider API calls before quota starts to bite

Fixed-interval syncs can burn API capacity even in a stable cluster. That matters when the provider is rate-limited or when other controllers are competing for the same quota. route_controller_route_sync_total gives you a count of those syncs, which is enough to show whether the controller is busy because it needs to be, or because it has been told to keep asking.

A quiet node set with a busy counter is a clue worth taking seriously. It usually means the route controller is making calls that do not change the outcome.

Keep the feature gate off-by-default behaviour in mind while testing v1.36

CloudControllerManagerWatchBasedRoutesReconciliation is off by default. That is handy for testing, because the baseline behaviour is still the old interval loop unless you change it. It also means a test cluster can look unchanged after an upgrade if you forget to turn the gate on.

In Kubernetes v1.36, the counter helps separate the two behaviours without guesswork. If the metric keeps climbing while the cluster sits still, you are still on polling. If it only moves with node events, watch-based reconciliation is doing the work.

Related posts

Weekly Tech Digest | 06 Jul 2026

Stay updated with the latest in tech! This digest covers AI ethics, auto industry shifts, and the impact of politics on technology, exploring today's pressing issues.

wolfCOSE zero-allocation parsing in embedded C

wolfCOSE looks sensible only if you care about what your firmware actually has to carry. I like that, because on small targets the wrong crypto feature can cost more than the message itself, and there...

restic | v0.19.1

restic v0 19 1: safer FUSE mounts and mountpoint checks, robust backup source and exclude handling, clearer CLI JSON output, Windows SFTP deletion fixes