Search code examples
hadoopmapreduceapache-pigclouderabigdata

Syntax error, unexpected symbol at or near 'FLATTEN'

When I tried this, it worked:

B = FOREACH A {
              X = STRSPLIT(agegroup,'-',0);
}

Output:

((20,30))
((20+))
((20,40))
and so on...

Now, I tried FLATTEN like this:

B = FOREACH A { 
        X = FLATTEN(STRSPLIT(agegroup,'-',0));
}

And I got the below error:

Pig Stack Trace
---------------
ERROR 1200: <file PigScript.pig, line 3, column 5>  Syntax error, unexpected symbol at or near 'FLATTEN'

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error during parsing. <file PigScript.pig, line 3, column 5>  Syntax error, unexpected symbol at or near 'FLATTEN'
            at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1701)
            at org.apache.pig.PigServer$Graph.access$000(PigServer.java:1421)
            at org.apache.pig.PigServer.parseAndBuild(PigServer.java:354)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:379)
            at org.apache.pig.PigServer.executeBatch(PigServer.java:365)
            at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:140)
            at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:769)
            at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
            at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
            at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
            at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
            at org.apache.pig.Main.run(Main.java:613)
            at org.apache.pig.Main.main(Main.java:158)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: Failed to parse: <file PigScript.pig, line 3, column 5>  Syntax error, unexpected symbol at or near 'FLATTEN'
            at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:241)
            at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:179)
            at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1688)
            ... 18 more
================================================================================

Please let me know where I got it wrong.

Solution

  • You cannot use FLATTEN inside a nested FOREACH block.Only CROSS, DISTINCT, FILTER, FOREACH, LIMIT, and ORDER BY are allowed in the nested FOREACH block.

    Change

    B = FOREACH A { 
                  X = FLATTEN(STRSPLIT(agegroup,'-',0));
              }

    To

    B = FOREACH A GENERATE FLATTEN(STRSPLIT(agegroup,'-',0));