Search code examples
jclsyncsort

Syncsort Sum Fields=None not removing duplicates


I'm trying to run a SYNCSORT job that will remove duplicate entries and when I run it, I'm still getting duplicates. The following is the SYNCSORT code I'm using:

INCLUDE COND=(((61,1,CH,EQ,C'P'),OR, 
              (61,1,CH,EQ,C'V')),AND,
              (8,2,CH,EQ,C'FL'))     
OUTREC FIELDS=(1:12,20,              
               30:36,20,             
               55:61,1)              
SORT FIELDS=(30,20,CH,A,             
             01,20,CH,A)             
SUM FIELDS=NONE                      

The input is as follows:

----+----1----+----2----+----3----+----4----+----5----+----6
      FL                           AMELIA CITY              
32034 FL  NASSAU                  FERNANDINA BEACH         P
32034 FL  NASSAU                  AMELIA CITY              V
32034 FL  NASSAU                  AMELIA ISLAND            S
32034 FL  NASSAU                  FERNANDINA               S

I'm getting most of the expected output, except that I'm still getting duplicates. The output that I have is as follows:

----+----1----+----2----+----3----+----4----+----5----+
MANATEE                      BRADENTON                P
MANATEE                      BRADENTON                P
MANATEE                      BRADENTON                P
MANATEE                      BRADENTON                P
MANATEE                      BRADENTON                P
MANATEE                      BRADINGTON               V
POLK                         BRADLEY                  P
HILLSBOROUGH                 BRANDON                  P
SUWANNEE                     BRANFORD                 P
MIAMI-DADE                   BRICKELL                 V

Any help would be appreciated as I'm not able to find my error.


Solution

  • This is what you are sort summing on:

    < ------------ Sort Field ----------------------->  
    ----+----1----+----2----+----3----+----4----+----5----+----6
          FL                           AMELIA CITY              
    32034 FL  NASSAU                  FERNANDINA BEACH         P
    32034 FL  NASSAU                  AMELIA CITY              V
    32034 FL  NASSAU                  AMELIA ISLAND            S
    32034 FL  NASSAU                  FERNANDINA               S
    

    the Duplicate records will be different in the first 11 bytes which you can not see. Try removing the outrec to check.

    Possible changes -

    • Change the outrec to an inrec
    • re-code the sort with fields associated with the output, see the following:

    The following sort sorts based on the output records:

    INCLUDE COND=(((61,1,CH,EQ,C'P'),OR, 
                   (61,1,CH,EQ,C'V')),AND,
                  (8,2,CH,EQ,C'FL'))     
    OUTREC FIELDS=(1:12,20,              
                  30:36,20,             
                  55:61,1)              
    SORT FIELDS=(42,20,CH,A,             
                 12,20,CH,A)             
    SUM FIELDS=NONE