Search code examples
javahadoopapache-piguser-defined-functionsudf

cannot resolve method in Java UDF from Pig


I am using Pig on Hadoop and DataFu sample here (http://datafu.incubator.apache.org/docs/datafu/guide/set-operations.html), here is my code and error message, anyone have any thoughts what is wrong? Thanks.

register datafu-1.2.0.jar;
define setDifference datafu.pig.sets.SetDifference();

-- ({(3),(4),(1),(2),(7),(5),(6)},{(1),(3),(5),(12)})
input = load 'input.txt' AS (B1:bag{T:tuple(val:int)},B2:bag{T:tuple(val:int)});

differenced = FOREACH input {
  -- input bags must be sorted
  sorted_b1 = ORDER B1 by val;
  sorted_b2 = ORDER B2 by val;
  GENERATE SetDifference(sorted_b1,sorted_b2);
}

-- produces: ({(2),(4),(6),(7)})
DUMP differenced;

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve SetDifference using imports:

thanks in advance, Lin


Solution

  • @LinMa : Looks like you are using wrong case while accessing the UDF.

    Alias defined to access the UDF is :

    define setDifference datafu.pig.sets.SetDifference();

    You have to use the alias name while using/invoking the method.

    GENERATE setDifference(sorted_b1,sorted_b2);