Search code examples
c#apache-kafkaconfluent-platformconfluent-kafka-dotnet

Is there a way to query the replication factor and retention time for a topic using the Confluent.Kafka .Net client?


The Confluent.Kafka AdminClient allows you to create a topic, specifying the name, number of partitions, replication factor, and retention (and I'm guessing other settings through the configs property). The GetMetadata() call, however, returns a TopicMetadata that only has the name and partition information on it. Is there a way to retrieve the replication factor and retention time using the .Net client?

 await adminClient.CreateTopicsAsync(new[]
                    {
                        new TopicSpecification
                        {
                            Name = topicName,
                            NumPartitions = _connectionSettings.TopicAutoCreatePartitionCount,
                            ReplicationFactor = _connectionSettings.TopicAutoCreatePartitionCount,
                            Configs = new Dictionary<string, string> {{"retention.ms", "9999999999999"}}
                        }
                    });

Solution

  • To get the retention time you can use DescribeConfigsAsync:

    var results = await adminClient.DescribeConfigsAsync(new[] { new ConfigResource { Name = "topic_name", Type = ResourceType.Topic } });
    
    foreach (var result in results)
    {
        var retentionConfig = result.Entries.SingleOrDefault(e => e.Key == "retention.ms");
    }
    

    But I'm not sure what the correct way to get the replication factor is, since it's not retrieved with DescribedConfigsAsync. One way I can think of is to use GetMetadata but it's not a very clean solution:

    var meta = adminClient.GetMetadata(TimeSpan.FromSeconds(5));
    var topic = meta.Topics.SingleOrDefault(t => t.Topic == "topic_name");
    var replicationFactor = topic.Partitions.First().Replicas.Length;