Search code examples
apache-kafkaconfluent-cloud

What are the different ways to get Kafka Cluster Audit log to GCP Logging?

What are the different ways to get Kafka Cluster Audit log to GCP Logging? Can anyone share more information on how can I achieve it?

Thank you!

Solution

  • Assuming you have access to the necessary topic (from what I understand the Audit topic is not stored on your own cluster), to get data out of Kafka, you need a consumer. This could be in any language.

    To get data into Cloud Logging, you need to use its API.

    That being said, you could use any compatible pair of Kafka clients & Cloud logging clients that you would be comfortable with.

    For example, you could write or find a Kafka Connect Sink connector that wraps the Java Cloud Logging client.