IIDR CDC Kafka subscription
For an enterprise wide data streaming platform implementation, we are using a confluent kafka and ibm iidr cdc .
Why the IIDR Management console has only two option to use zookeeper & rest proxy while configuring the subscription for kafka ?
From security perspective , it is not recommended at all to expose zookeeper . How can we just provide them broker endpoints to connect to kafka ? Or how can we implement IIDR confluent kafka connector ? https://www.confluent.io/connector/ibm-data-replication/
How SSL can be implemented in IIDR CDC and Kafka ? any document links ?
Thanks!
Sorry for the delay...
The following document describes several scenarios of IDR Kafka configuration. You can provide broker endpoints editing the /conf/kafkaproducer.properties (on the machine you installed IDR Kafka Agent). If you do so, you still have to provide hostname and port number on MC GUI, but it will ignore them (you can provide fake data in those fields). It also explain some considerations on kerberized environments and on SSH connections.
Keep in mind that JRE provided with CDC has restricted security policy files, so, for using keys greater than 56 bytes (i.e. always), you need to download and upgrade them with unrestricted ones (they can be downloaded from IBM FixCentral page).
I hope this helps.
- Will Kafka continue assigning sequential offsets after retention policy deletes old messages?
- Unexpected Kafka request of type METADATA during SASL handshake
- Kafka Getting error No resolvable bootstrap urls given in bootstrap servers
- Kafka Connector is getting time out when opening in connect-distributed mode
- Kafka KRaft mode with docker-compose
- In Apache Kafka why can't there be more consumer instances than partitions?
- Using kafka-schema-registry-maven-plugin with parent and child schemas
- Kafka Best Practices + how to set recommended setting for JVM
- Connect to Kafka on host from Docker (ksqlDB)
- During rolling upgrade/restart, how to detect when a kafka broker is "done"?
- Kafka Connect running out of heap space
- Fault-tolerant queue-worker architecture in Kafka?
- Out of Memory issue when running a beam streaming pipeline using gcp dataflow with Kafka as a Source
- How do I test that an offset has been committed or not to Kafka
- Does it make sense to have a single send inside a kafka transaction using spring kafka?
- NullPointerException in Kafka-Connect service logs on start of Kafka-Connect
- Extracting Key and Appending It to the Message Value in Kafka Connect
- Confluent-Kafka: no broker available for coordinator query: intervaled in state query-coord
- DefaultAfterRollbackProcessor without BackOff for batch listener
- Java Flink NoClassDefFoundError org/apache/flink/shaded/guava30/com/google/common/io/Closer
- Searching messages in large kafka topic
- kafka-acls.sh Command Fails with Operations Containing Underscores in Kafka 2.7.0
- how to send batched data with Spring Kafka producer
- How can i enable kafka metrics in Kafka streams spring boot app
- Count mismatch in AKHQ UI 0.24.0
- Debezium KafkaConnect, kafka_connect_run.sh No such file or directory
- In docker-compose, my init-kafka image is failing to list topics on my kafka image. How do I fix this? (With KRaft, not Zookeeper!)
- Send custom message to Kafka topic
- Error connecting to kafka broker as a k8s pod on local machine
- How to access Kafka Connect created by Strimzi?