How to Programmatically Create Topics in Kafka Using Python

Photo by Fotis Fotopoulos on unsplash.com

You can programmatically create topics using either kafka-python or confluent_kafka client which is a lightweight wrapper around librdkafka.

Using kafka-python

from kafka.admin import KafkaAdminClient, NewTopic


admin_client = KafkaAdminClient(
    bootstrap_servers="localhost:9092",
    client_id='test'
)

topic_list = []
topic_list.append(NewTopic(name="example_topic", num_partitions=1, replication_factor=1))
admin_client.create_topics(new_topics=topic_list, validate_only=False)

Using confluent_kafka

from confluent_kafka.admin import AdminClient, NewTopic


admin_client = AdminClient({
"bootstrap.servers": "localhost:9092"
})

topic_list = []
topic_list.append(NewTopic("example_topic", 1, 1))
admin_client.create_topics(topic_list)

And that’s it! I hope you have found this useful.