I have a Spark streaming app and I have re-setted the build path, in order to make it cleaner.
After I re-imported all the jar files, I get this error I've never had before:
How is it possible? How can I solve this?
static FlatMapFunction<Tuple2<String, String>, String> sentimentFunc = new FlatMapFunction<Tuple2<String, String>, String>(){
private static final long serialVersionUID = 1L;
@Override
public Iterator<String> call(Tuple2<String, String> x) throws Exception {
List<String> output = new ArrayList<String>();
if(x._2==null){
output.add("ERR");
return output.iterator();
}
boolean like = false, sad = false, angry = false, hilarious = false, neutral = false;
boolean [] sentiments = {like, angry, sad, hilarious, neutral};
sentiments = checkEmojis(x, sentiments);
if(checkSentiment(sentiments)){
output.add(setSentiment(sentiments));
return output.iterator();
}
sentiments = checkText(x, sentiments);
output.add(setSentiment(sentiments));
return output.iterator();
}
};
In Spark 1.x, the return type of call
for FlatMapFunction
is Iterable
.
In Spark 2.x, the return type of call
for FlatMapFunction
has been changed to Iterator
.
It seems that when you reset your build path, you changed it to point to spark 1.x instead of 2.x, thus invalidating all your flatmap functions.