How can we use the kafka connect truststore password in an abstract way in the KafkaConnector resource?

Question

We have a connect cluster of 3 nodes. We need couple of certificates in our connect cluster truststore. We have installed those certificates in the following way.

...
spec:
  tls:
      trustedCertificates:
      - certificate: ca.crt
        secretName: my-cluster-cluster-ca-cert
      - secretName: root-cer
        certificate: RootCA.crt
      - certificate: IntermediateCA.crt
        secretName: inter-cer
      - secretName: solace-broker-secret
        certificate: secure-solace-broker.crt
...

As you know, after the three connect clusters spins up, the certificates has been installed into the following truststore /tmp/kafka/cluster.truststore.p12. Also, we can found the randomly truststore password into the following file: /tmp/strimzi-connect.properties.

We direct the truststore path and the truststore password in the KafkaConnector resource file.

apiVersion: kafka.strimzi.io/v1alpha1
kind: KafkaConnector
metadata:
  name: solace-source-connector
  labels:
    strimzi.io/cluster: my-connect-cluster
spec:
  class: com.solace.connector.kafka.connect.source.SolaceSourceConnector
  tasksMax: 1
  config:
    value.converter: org.apache.kafka.connect.converters.ByteArrayConverter
    key.converter: org.apache.kafka.connect.storage.StringConverter
    kafka.topic: solace-test
    sol.host: tcps://msdkjskdjsdfrdfjdffdhxu3n.messaging.solace.cloud:55443
    sol.username: my-solace-cloud-username
    sol.password: password
    sol.vpn_name: solaceservice
    sol.topics: try-me
    sol.message_processor_class: com.solace.connector.kafka.connect.source.msgprocessors.SolSampleSimpleMessageProcessor
    sol.ssl_trust_store: /tmp/kafka/cluster.truststore.p12
    sol.ssl_trust_store_password: HARDCODED_RANDOM_PASSWORD

Right now we are getting inside into one of the connect cluster pod, get the password from the /tmp/strimzi-connect.properties file and then using the password in the sol.ssl_trust_store_password field.

My question:

Is there any way to parametrize the password? Any encapsulated way to use the password (so that we do not need to get inside into the pod to know the password - expectation is, the kafkaconnector resouce would fecth the password from the /tmp/strimzi-connect.properties file, on which pod it is running)

Answer 1

I have got the answer from the Slack channel by Jakub Scholz.

The tls configuration you are using and the truststore are supposed to be used for communication between Connect and Kafka, not for the connectors. I think you have two options how to provide a truststore for the connector

1. You can use the same truststore as you are using now, but load the password using the FileConfigProvider - I think that should load the right password on each connect node
2. You can just create your own secret with the truststore for the connector and load it into connect using this: https://strimzi.io/docs/operators/latest/full/using.html#assembly-kafka-connect-external-configuration-deployment-configuration-kafka-connect

And this is how I have implemented it:

1. Creating a custom keystore along with my certificates:

keytool -import -file RootCA.crt -alias root -keystore myTrustStore

2. Creating a Kubernetes secret with the trust store:

kubectl create secret generic my-trust-store --from-file=myTrustStore

3. loading the secret into the connect resource file:

spec:
  ...
  externalConfiguration:
    volumes:
      - name: my-trust-store
        secret:
          secretName: my-trust-store

4. After the connect cluster pod spins up, the certificate will going to be available at /opt/kafka/external-configuration/my-trust-store/