Time-Based Consumption Fails Due to Missing MaxTimestamp in IBM Sarama Batches

Symptom

When a topic is configured with message.timestamp.type=CreateTime and records are produced as batches using IBM Sarama, consuming records using a specific timestamp (i.e., timequery) fails to retrieve the correct records corresponding to that timestamp.

For example, although offset 10 has the timestamp 1743059592562, consuming at this exact timestamp returns offset 30, which is associated with a different timestamp, 1743061521512.

$ rpk topic consume panda -o10 -n 1 { "topic": "panda", "value": "Batch message \u0000", "timestamp": 1743059592562, "partition": 0, "offset": 10 } $ rpk topic consume panda -o@1743059592562 -n 1 { "topic": "panda", "value": "hello?", "timestamp": 1743061521512, "partition": 0, "offset": 30 }

 

Cause

This issue occurs because IBM Sarama (version 1.45.0 and earlier) does not set a MaxTimestamp for record batches during producing.

 

Resolution

Use IBM Sarama version 1.45.1 or later, which includes a fix to correctly set the MaxTimestamp in record batches. For details, refer to this pull request.

 

If you need further assistance please reach out to Redpanda Support.