Search code examples
javaapache-sparkspark-streamingbuildpath

Iterator<String> not accepted after re-setting the build path


I have a Spark streaming app and I have re-setted the build path, in order to make it cleaner. After I re-imported all the jar files, I get this error I've never had before: enter image description here How is it possible? How can I solve this?

static FlatMapFunction<Tuple2<String, String>, String> sentimentFunc = new FlatMapFunction<Tuple2<String, String>, String>(){
        private static final long serialVersionUID = 1L;


        @Override
        public Iterator<String> call(Tuple2<String, String> x) throws Exception {
            List<String> output = new ArrayList<String>();
            if(x._2==null){
                output.add("ERR");
                return output.iterator();
            }
            boolean like = false, sad = false, angry = false, hilarious = false, neutral = false;
            boolean [] sentiments = {like, angry, sad, hilarious, neutral};
            sentiments = checkEmojis(x, sentiments);
            if(checkSentiment(sentiments)){
                output.add(setSentiment(sentiments));
                return output.iterator();
            }

            sentiments = checkText(x, sentiments);
            output.add(setSentiment(sentiments));
            return output.iterator();
        }

  };

Solution

  • In Spark 1.x, the return type of call for FlatMapFunction is Iterable.

    In Spark 2.x, the return type of call for FlatMapFunction has been changed to Iterator.

    It seems that when you reset your build path, you changed it to point to spark 1.x instead of 2.x, thus invalidating all your flatmap functions.