Search code examples
timeapache-pigdifferenceepoch

Epoch time difference in Pig

I have 3 columns which contains start_time , end_time and tags. Times are represented in epoch time format as shown in example below. I want to find the the rows which have 1 hour time difference between them.

Example:

Start_time     End_Time    Tags
1235000081    1235000501  "Answered"
1235000081    1235000551  "Answered"

I need to fetch the tags column if the time diff is less than an hour.

I want do it in PIG - can anyone kindly help?

Solution

  • input.txt

    1235000081  1235000501  Answered
1235000081  1235000551  Answered

    pig script

    A =  Load '/home/kishore/input.txt' as (col1:long, col2:long, col3:chararray);
B = Foreach A generate ToDate(col1) as startdate,ToDate(col2) as enddate,col3;    
C = Filter B by GetHour(enddate)-GetHour(startdate) == 1;   
Dump C;

    you can filter the row based on your condition like >,< ,==