Search code examples
hadoopmapreduceapache-pigcloudera

Count frequency of values in a column in PIG?


I have something like this:

ColA ColB
a    xxx
b    yyy
c    xxx
d    yyy
e    xxx

I need to find out the number of times each value of ColB occurs.

Output:

xxx 3
yyy 2

Here's what I've been trying:

Considering A has my data,

grunt> B = GROUP A by ColB;
grunt> DESCRIBE B;
B: {group: chararray,A: {(ColA: chararray,ColB: chararray)}}

Now I'm confused, do I do something like this?

grunt> C = FOREACH B GENERATE COUNT(B.ColB)

So I need the output to be like this,

xxx 3
yyy 2

Solution

  • I figured it out.

    C = FOREACH B GENERATE GROUP AS ColB, COUNT(A) as count;