Description
Motivation
The original discussion mail :
https://lists.apache.org/thread/m9dkhq1fs6stsdwh78h84fsl5hs5v67f
Introduce the ability to store metadata about topics.
This would be very useful as with metadata you could add labels and other
pieces of information that would allow defining the purpose of a topic,
custom application-level properties.
This feature will allow application-level diagnostic tools and maintenance
tools to not need external databases to store such metadata.
Imagine that we could add a simple key value map (String keys and String
values) to the topic.
These metadata could be set during topic creation and also updated.
API Changes
We need to add a map type field called metadata to the below methods signature:
- PersistentTopics#createPartitionedTopic
PersistentTopics#createPartitionedTopic(AsyncResponse asyncResponse, String tenant, String namespace,String encodedTopic,int numPartitions,boolean createLocalTopicOnly, Map<String, String> metadata); - PersistentTopics#createNonPartitionedTopic
PersistentTopics#createNonPartitionedTopic(String tenant, String namespace, String encodedTopic, boolean authoritative, Map<String, String> metadata);
Implementation
-
For PartitionedTopic
We will put these metadata to the PartitionedTopicMetadata, serialized to store in configurationMetadataStore, like partitioned-topic partitions.public class PartitionedTopicMetadata { /* Number of partitions for the topic */ public int partitions; /* Topic metadata */ public Map<String, String> metadata; -
For NonPartitionedTopic
We will store the metadata to ML(ManagedLedger) which already supports storing custom properties.ManagedLedger#setProperties(Map<String, String> properties)
Compatibility
The proposal will not introduce any compatibility issues.
Tests Plan
Unit tests & integration tests