Demos and production are different sports. This checklist covers what changes when Apache Kafka has to be reliable, secure and observable under real load.
Security and reliability defaults
- Enable TLS and SASL auth; set ACLs per topic and group
- Encrypt data at rest and restrict inter-broker access
A production-minded config
Produce and consume with the Python client.
from kafka import KafkaProducer, KafkaConsumer
p = KafkaProducer(bootstrap_servers='localhost:9092')
p.send('orders', b'{"id": 42}'); p.flush()
c = KafkaConsumer('orders', bootstrap_servers='localhost:9092', group_id='billing')
for msg in c:
print(msg.value)Operational best practices
- Size partitions for target consumer parallelism up front
- Monitor consumer lag and set alerts on it
- Use a Schema Registry and compatibility rules
In production, boring and reproducible beats clever every time.
Want senior help putting Apache Kafka to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.