Search code examples
pythonapache-kafkafaust

Difference between Faust vs Kafka-python

I could not find any answer to this: what is the difference between Faust and kafka-python?
Is there any pros/cons on preferring any one of them?
As I understand it:

  • Kafka is written in Java, and Kafka-python is a Python client to communicate with "Java stream"
  • Faust is a pure "Python stream"

So, if I plan to use only Python then Faust should be better choice and if I want to have wider compatibility (Go, .NET, C/C#, Java, Python) then use Kafka + Kafka-python?

Note: I am new to using Kafka and I am trying to understand the pros/cons of different solutions.

I would highly appreciate any advice!!

Solution

  • As I understand it you use both with Kafka, and both from Python, but with the difference that:

    • Faust is for stream processing (filtering, joining, aggregating, etc)
    • kafka-python (just like confluent-kafka-python also) is a client library providing Consumer, Producer, and Admin APIs for Kafka.

    So you could easily use both, for different purposes, from Python.