Problem Statement

Suppose you have existing topics in a v23.3.x cluster, once you enable Tiered Storage by enabling cloud_storage_enabled, brokers start complaining below and existing topics may end up with unavailable partially.

WARN  2024-03-14 14:10:40,229 [shard 1:main] raft - [group_id:1, {kafka/foo/0}] state_machine_manager.cc:419 - exception thrown from background apply fiber for archival_metadata_stm - std::runtime_error (Reader cannot read before start of the log 0 < 11)

For example, producing to a topic works but consuming from the topic won't work.

 

This applies all versions lower than v23.3.9 (inclusive).

 

Solution

Upgrade the cluster to v23.3.10 or above.

 

 

Notes

Relevant fix: https://github.com/redpanda-data/redpanda/issues/17086