In Cloud Foundry I am able to produce message to non ssl url ("kafkaURL:9092"). But it does NOT work for ssl url ("kafkaURL:9093").
Kafka server version 0.10.0.1 and client version 0.10.0.0.
Here are the properties I used:
props.put(org.apache.kafka.clients.producer.ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, config.getString("obs_q_and_a_db.kafka.metadataBrokerList"))
props.put(org.apache.kafka.clients.producer.ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, classOf[StringSerializer])
props.put(org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, classOf[StringSerializer])
props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SSL")
props.put(SslConfigs.SSL_TRUSTSTORE_LOCATION_CONFIG, "mySslFolder/answersapi.kafka.client.keystore.jks")
props.put(SslConfigs.SSL_TRUSTSTORE_PASSWORD_CONFIG, "sslTruststorePassword"))
props.put(SslConfigs.SSL_KEYSTORE_LOCATION_CONFIG, "mySslFolder/answersapi.kafka.client.truststore.jks")
props.put(SslConfigs.SSL_KEYSTORE_PASSWORD_CONFIG, "sslKeystorePassword")
props.put(SslConfigs.SSL_KEY_PASSWORD_CONFIG, "sslKeyPassword")
props.setProperty("metadata.broker.list", "kafkaURL:9093"))
props.setProperty("serializer.class", serializerClass)
props.setProperty("message.send.max.retries", maxRetries.toString)
props.setProperty("request.required.acks", requiredAcks.toString)
props.setProperty("producer.type", producerType)
props.setProperty("batch.num.messages", batchNumMessages.toString)
The same properties and same cert files (truststore,keystore files) when used from Kafka shell on the kafka server work fine using following command in shell:
kafka-console-producer --broker-list kafkaURL:9093 --producer.config config --topic myTopicName
And here is the error:
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT java.io.EOFException
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:99)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.network.BlockingChannel.readCompletely(BlockingChannel.scala:129)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.network.BlockingChannel.receive(BlockingChannel.scala:120)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:77)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.SyncProducer.send(SyncProducer.scala:119)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:68)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:79)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.utils.Logging$class.swallowError(Logging.scala:106)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:68)
2017-01-18T12:03:29.78-0600 [APP/PROC/WEB/0]OUT at kafka.producer.Producer.send(Producer.scala:77)
Our Kafka client code version: “0.9.0.1” stopped working when server moved to 0.10.0.1 When we changed our client code to 0.10.0.0 we were still getting the same EOFileException It got fixed when we replaced deprecated classes kafka.producer.{KeyedMessage, Producer, ProducerConfig} with new 0.10.0 classes org.apache.kafka.clients.producer.{ProducerRecord, KafkaProducer, ProducerConfig} respectively By the way old deprecated classes work fine when pointing to non SSL url, they fail only when pointing to Secured Kafka URL