Kafka Connect - How to limit permissions to create, update, delete specific connectors

Question

I am faced with a situation in which multiple developers are working on specific kafka source or sink connector configurations. The connectors are created, updated, deleted, and so on via the kafka connect REST API. It must be assured that

1. a developer never deletes or modifies connectors of others.
2. a developer can only manage connectors of a specific connector.class like for example "streams.kafka.connect.sink.Neo4jSinkConnector"

Is there any way to configure Kafka to restrict the use of the Kafka-Connect REST API to specific operations related to connector configuration details? The kafka ACLs seem not to cover this usecase.

Answer 1

Kafka Connect offers Basic HTTP Auth with username/password support, but for "ownership" or "limited management" of Connectors, no, that feature doesn't really exist out of the box.

For this, you'd need to implement your own REST Extension(s).

Resources:

• https://cwiki.apache.org/confluence/display/KAFKA/KIP-285%3A+Connect+Rest+Extension+Plugin
• https://kafka.apache.org/documentation/#connectconfigs_rest.extension.classes
• https://github.com/apache/kafka/blob/3.2/connect/api/src/main/java/org/apache/kafka/connect/rest/ConnectRestExtension.java

Or create your own "proxy server" that will do this and forward requests to the Connect API.

Other option would be to let said developers deploy and maintain their own Connect Clusters, such as via containers in Kubernetes or Terraform some cloud VMs.