Most Apache Kafka pain is self-inflicted and predictable. Here are the mistakes we see most, and the fix for each.
Mistake 1: Too few partitions, capping consumer parallelism
Fix: Size partitions for target consumer parallelism up front. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
Mistake 2: Ignoring consumer lag until it becomes an outage
Fix: Monitor consumer lag and set alerts on it. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
Mistake 3: No schema management, so producers break consumers
Fix: Use a Schema Registry and compatibility rules. Make it the default in your templates and enforce it in review or CI so the mistake cannot come back.
A safer baseline
Produce and consume with the Python client.
from kafka import KafkaProducer, KafkaConsumer
p = KafkaProducer(bootstrap_servers='localhost:9092')
p.send('orders', b'{"id": 42}'); p.flush()
c = KafkaConsumer('orders', bootstrap_servers='localhost:9092', group_id='billing')
for msg in c:
print(msg.value)Want senior help putting Apache Kafka to work - or a team bootcamp on it? CodeWizard runs Israel's largest DevOps community and embeds named senior engineers with a transparent hours log. Book a free consult or explore the Academy.