Summary:

Redpanda v26.1.16 and v25.3.16 contain a regression in the c-ares 1.34.7 DNS library. When a DNS query is issued while another query on the same channel is in a completing or timing-out state, and the two happen to share the same randomly assigned query ID, c-ares silently discards the new query without invoking its callback, which results in a blocked semaphore.

This causes an internal RPC transport between one shard and one peer broker to become permanently blocked after any broker restart. This manifests as leaderless or under-replicated partitions and heartbeat errors. The cluster health monitor and readiness probes report the cluster unhealthy cluster-wide, even though Raft has a healthy quorum and produce/consume continue normally through the surviving leader.

Some clusters on v26.1.16 have additionally shown Kafka connection-slot exhaustion: a broker accumulates logical connections that are no longer serving clients, eventually reaching kafka_connections_max and dropping new connections without returning a Kafka protocol error. rpk cluster health continues to report healthy with no leaderless partitions. If the affected broker holds controller or Cloud Topics leadership, other brokers cannot reach its Kafka listener and writes to dependent partitions also fail.

Do not run 25.3.16 or 26.1.16. A fix was released in 25.3.17 and v26.1.17 while v26.2.1 remains unaffected by the issue, and Redpanda Cloud (BYOC, Dedicated and Serverless) clusters have already been reverted.

Update:

CVE-2026-33630 is fixed in c-ares 1.34.7, but that version contains the upstream regression (https://github.com/c-ares/c-ares/issues/1256) described in this TSB. The c-ares 1.34.8 release does not fix the regression, and the CVE-2026-33630 fix will be re-applied once an upstream release resolves it.

Severity:

High

Redpanda Products Affected:

• Redpanda Self-Managed - Enterprise

• Redpanda Self-Managed - Community

• Redpanda Cloud (including Redpanda Serverless)

Release Affected:

• 26.1.16

• 25.3.16

Fixed in: 26.1.17 / 25.3.17. 

Downgrades

26.1.16 → 26.1.14 is a supported downgrade path.

25.3.16 → 25.3.15 is a supported downgrade path.

Identification:

If you are running 25.3.16 or 26.1.16, you are at risk. Check your version first:

rpk version

The following indicate the condition is already developing:

• A broker at or near its Kafka connection limit while other brokers in the same cluster are well below it. Compare vectorized_kafka_rpc_active_connections per broker against rpk cluster config get kafka_connections_max.

• vectorized_kafka_rpc_connections_rejected increasing on that broker, while vectorized_kafka_rpc_connections_rejected_rate_limit and vectorized_kafka_rpc_connection_close_errors stay at zero.

• Clients reporting connection failures or timeouts with no Kafka-level error.

rpk cluster health may report healthy throughout. Do not use it to rule this out.

Impact:

• Clients cannot establish new connections to an affected broker.

• The condition can spread to every broker in the cluster.

• If the affected broker holds controller or Cloud Topics leadership, writes to dependent partitions fail.

• Restarting the affected broker releases the retained slots and restores connectivity, but the condition recurs while the cluster remains on 25.3.16 / 26.1.16.

Action required:

Self Hosted

Immediate action: Do not upgrade to Redpanda 25.3.16 / 26.1.16. Halt any upgrade to 25.3.16 / 26.1.16 that is scheduled or in progress.

Remediation:  If you are running one of the affected versions, please roll back to either 25.3.15 or 26.1.14 as appropriate for your major version. Roll back every broker — a cluster showing symptoms can appear healthy after a single broker restart while other brokers still run the affected version.

If you have questions about a minor version downgrade please reach out to Redpanda Support

You can also upgrade to any v26.2.x immediately from 26.1.16, or to 26.1.14 from 25.3.16 to mitigate the issue.  Once 26.1.17 and 25.3.17 are released those are also safe target versions for upgrades.

If a broker is currently refusing connections: Restart that broker to release the retained connection slots and restore client connectivity, then roll back. Confirm which broker is affected before restarting it. If the cluster is already showing leaderless or under-replicated partitions, contact Redpanda Support before restarting.

Redpanda Cloud

No action is needed, any impacted cluster is already either downgraded, or had its upgrade halted until a patch version is out. 

Questions? If you have any questions on this TSB, or need further guidance, please contact Redpanda Support