I'm fairly new to flink/Java/Scala so this might be a non-question but any help is appreciated. I haven't been able to find an example that uses the Flink Kafka connector with Flink 1.13 (and works for me).
My project is here: https://github.com/sysarcher/flink-scala-tests
I want to I'm unable to use FlinkKafkaConsumer
(link) which I want to try out.
I'm using IntelliJ Idea. The project was generated from the tutorial on Flink's website
The following link was used to generate the project: https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/try-flink/datastream/#how-to-follow-along
Error Encountered
$ mvn clean compile
[INFO] ...
[WARNING] Multiple versions of scala libraries detected
[ERROR] /hide/abs/path/github/flink-stuff/frauddetection/src/main/scala/spendreport/FraudDetectionJob.scala:42: error: not found: type FlinkKafkaConsumer
[ERROR] val kafkaConsumer = new FlinkKafkaConsumer[String]("car.create", new SimpleStringSchema(), properties)
[ERROR] ^
[ERROR] one error found
One problem was the missing import
statement:
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
I can build the artifacts using mvn clean compile
but IntelliJ still complains with the following error:
/long/path/flink-stuff/frauddetection/src/main/scala/spendreport/FraudDetectionJob.scala:30:35
object connectors is not a member of package org.apache.flink.streaming
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer
It turns out the problem was with IntelliJ's indexing (or something, I'm not sure). I pressed CTRL+SHIFT+A
and typed Reload All Maven Projects
. Thereafter, the build was successful.
This comment helped: https://lists.apache.org/x/thread.html/rc332d1054886e35af65de4a7c38553e02a2304a2ebdab8197badd94d@%3Cuser.flink.apache.org%3E
It seems sometime IntelliJ does not works well for index, perhaps you could choose mvn -> reimport project from the context menu, if it still not work, perhaps you might try remove the .idea and .iml file and re-open the project again.