Search code examples
javahadoopapache-pigdependency-managementudf

Java dependencies in Pig UDF

I wrote a UDF which uses Joda Time. I included it as a dependency in pom.xml. When I run my pig script I get the error

ERROR 2998: Unhandled internal error. org.joda.time.LocalDate.parse(Ljava/lang/String;)Lorg/joda/time/LocalDate;

I am pretty new to java, maven and hadoop. How can I bundle joda into my jar (Like statical linked in c/c++) which I run on the cluster?

This is NOT a duplicate of how to include external jar file using PIG since I cant use -D option and registering Joda Time in the pig script has no effect.

Solution

  • The problem is that pig comes with Joda 1.6 where LocalDate does not have a parse() method. The easiest and most safe fix would be to use 1.6 in maven and rewrite your code with 1.6 Joda Time.