Search code examples
hadoopmapreduceapache-pig

Does PIG have NOT IN clause

I'm trying like

select * from A where A.ID NOT IN (select id from B) (in sql)

filtersource= FILTER source BY ID NOT(destination.ID)

How to do NOT IN clause or some other technique to keen out the extra records present in one table using pig

Solution

  • yes you can do this in PIG like:

    filtersource= FILTER source BY NOT ID IN (your condition or joined field);

    example:

    HIVE> select * from table where id NOT IN ('1','2','3');

grunt> A = LOAD 'db.table' USING org.apache.hive.hcatalog.pig.HCatLoader() AS (id:int, value:chararray);
grunt> B = FILTER A BY NOT id IN (1,2,3);