Problem Statement

Since v23.1, Redpanda has some properties that carry configuration values as a map object. This article describes how to update the value and a known issue with it.

Solution

As of v23.1.7, the following properties carry values as a map object. Both properties have an empty value by default. Please visit https://docs.redpanda.com/docs/reference/cluster-properties/ for more details about the properties.

• kafka_client_group_byte_rate_quota
• kafka_client_group_fetch_byte_rate_quota

Let's pick up kafka_client_group_byte_rate_quota as an example here.

$ rpk cluster config get kafka_client_group_byte_rate_quota
[]

To add a rule, run this command.

$ rpk cluster config set kafka_client_group_byte_rate_quota '[{"group_name": "group1","clients_prefix": "kafka","quota": 10240}]'

To confirm if it is set, run this command.

$ rpk cluster config get kafka_client_group_byte_rate_quota
- clients_prefix: kafka
  group_name: group1
  quota: 10240

To add a second rule, append a new rule to the existing one.

$ rpk cluster config set kafka_client_group_byte_rate_quota '[{"group_name": "group1","clients_prefix": "kafka","quota": 10240},{"group_name": "group2","clients_prefix": "redpanda","quota": 10240}]'

To confirm if it is set, run this command.

$ rpk cluster config get kafka_client_group_byte_rate_quota
- clients_prefix: redpanda
  group_name: group2
  quota: 10240
- clients_prefix: kafka
  group_name: group1
  quota: 10240

 

Known Issue

rpk cluster config edit provides a way to edit configuration values in an interactive fashion. However, current representation of the map object is incorrect. This causes a failure of saving any configuration changes.

Example error message is below.

unable to edit: error updating config: Validation errors:
 * kafka_client_group_byte_rate_quota: YAML::BadSubscript (yaml-cpp: error at line 1, column 2: operator[] call on a scalar (key: "group_name"))

A workaround is to correct the existing map object manually and save it. Example below.

Before

kafka_client_group_byte_rate_quota:
   - map[clients_prefix:redpanda group_name:group2 quota:10240]
   - map[clients_prefix:kafka group_name:group1 quota:10240]

After

kafka_client_group_byte_rate_quota:
   - {"group_name": "group1","clients_prefix": "kafka","quota": 10240}
   - {"group_name": "group2","clients_prefix": "redpanda","quota": 10240}

See more details in https://github.com/redpanda-data/redpanda/issues/10251