Search code examples
springspring-batchclassificationitemwritercompositeitemwriter

Error with Spring batch Classifier Composite Item Writer


I have to basically produce multiple xml files for each file_id per currency ( ie. usd,zar ect) these transactions are all in 1 DB table. Do I create a composite writer for each currency and on my Item Processor I filter for each different currency that I read from the DB. or Can I use multiple steps for each currency per file_id ? I have been struggling to find a Springbatch solution around this.

The filename resource will be different for each file and currency. For example I can recieve file_id=1 currency=USD needs to be 1 file 'USD20051701 with 01 the file sequence'. I can also get two files file_id 1 & 2 for Currency ='ZAR' and those need to be two files 'ZAR20051701' & 'ZAR20051702' 01 & 02 file sequences.

I was using this link as a guide from one of the posts. https://stackoverflow.com/a/53388876/13056119

I am getting this Trace Log Log File

Please help, I got stuck trying to debug the Log screenshot. I have written a Classifier to write to 3 different files, per currency but I am getting an Error

@Bean
public ClassifierCompositeItemWriter<Settlement> classifierCompositeItemWriter
        (       ItemWriter<Settlement> ZMWItemWriter,
                ItemWriter<Settlement> USDItemWriter, 
                ItemWriter<Settlement> ZARItemWriter
    ) {
    ClassifierCompositeItemWriter<Settlement> classifierCompositeItemWriter = new ClassifierCompositeItemWriter<>();
    classifierCompositeItemWriter.setClassifier(new Classifier<Settlement, ItemWriter<? super Settlement>>() {
        
        @Override
        public ItemWriter<? super Settlement> classify(Settlement settlement) {
            
            List<SettlementHeader> settlementheader= new ArrayList<SettlementHeader>();
            SettlementHeader header = new SettlementHeader ();
            settlementheader.add(header);
            settlement.setSettlementHeader(settlementheader);
            
        if (header.getCurrency().equalsIgnoreCase("ZMW")) {
            return ZMWItemWriter;
        } 
        else if (header.getCurrency().equalsIgnoreCase("USD")) {
            return USDItemWriter;
        }
        else {
            return ZARItemWriter;}
        }
    });
    return classifierCompositeItemWriter;
}

    @Qualifier ("USDItemWriter")
    @Bean(destroyMethod="")
   public NoRootStaxEventItemWriter<Settlement> USDItemWriter() throws Exception {
    NoRootStaxEventItemWriter<Settlement> ItemWriter = new NoRootStaxEventItemWriter<>();
    FileSystemResource resource = new FileSystemResource("FileUSD1.xml");
    ItemWriter.setName("USDItemWriter");
       ItemWriter.setResource(resource);
       marshaller.setPackagesToScan("com.model");
       ItemWriter.setMarshaller(marshaller);
       ItemWriter.afterPropertiesSet();
       return ItemWriter;
   }

@Primary
@Qualifier("ZARItemWriter") 
@Bean(destroyMethod="")
   public NoRootStaxEventItemWriter<Settlement> ZARItemWriter() throws Exception {
    NoRootStaxEventItemWriter<Settlement> ItemWriter = new NoRootStaxEventItemWriter<>();
    FileSystemResource resource = new FileSystemResource("FileZAR1.xml");
    ItemWriter.setName("ZARItemWriter");
       ItemWriter.setResource(resource);
       marshaller.setPackagesToScan("com.model");
       ItemWriter.setMarshaller(marshaller);
       ItemWriter.afterPropertiesSet();
       return ItemWriter;
   }
    @Qualifier("ZMWItemWriter")
    @Bean(destroyMethod="")
   public NoRootStaxEventItemWriter<Settlement> ZMWItemWriter() throws Exception {
    NoRootStaxEventItemWriter<Settlement> ItemWriter = new NoRootStaxEventItemWriter<>();
    FileSystemResource resource = new FileSystemResource("FileZMW1.xml");
    ItemWriter.setName("ZMWItemWriter");
    ItemWriter.setResource(resource);
    marshaller.setPackagesToScan("com.model");
    ItemWriter.setMarshaller(marshaller);
    ItemWriter.afterPropertiesSet();
      return ItemWriter;
   }

Solution

  • Please check the example below and see if it works for you. You create a Map of Writers to avoid instantiating a new Writer each time.

    A Writer is identified uniquely by (FileId, ProcessDate, Detail Currency):

    String fileNameKey = item.getHeader().getFileId() + item.getHeader().getProcessDate().toString() + settlementDetail.getCurrency();
    

    Here is the code for the writer:

    @Component
    public class SettlementWriter implements ItemStream, ItemWriter<Settlement> {
    
        private final Map<String, StaxEventItemWriter<Settlement>> writers = new HashMap<>();
    
        private ExecutionContext executionContext;
    
        @Autowired
        private Marshaller marshaller;
    
        @Override
        public void open(ExecutionContext executionContext) throws ItemStreamException {
            this.executionContext = executionContext;
        }
    
        @Override
        public void update(ExecutionContext executionContext) throws ItemStreamException {
        }
    
        @Override
        public void close() throws ItemStreamException {
            for (StaxEventItemWriter f : writers.values()) {
                f.close();
            }
        }
    
        @Override
        public void write(List<? extends Settlement> items) throws Exception {
            for (Settlement item : items) {
                for (SettlementDetail det : item.getDetails()) {
                    StaxEventItemWriter<Settlement> detailWriter = getFlatFileItemWriter(item, det);
                    detailWriter.write(Arrays.asList(item));    
                }
            }
        }
    
        public StaxEventItemWriter<Settlement> getFlatFileItemWriter(Settlement item, SettlementDetail settlementDetail) {
            String fileNameKey = item.getHeader().getFileId() + item.getHeader().getFilename() + settlementDetail.getCurrency();
    
            StaxEventItemWriter<Settlement> itemWriter = writers.get(fileNameKey);
    
            if (itemWriter == null) {
    
                itemWriter = new StaxEventItemWriter<>();
    
                try {
    
                    FileSystemResource resource = new FileSystemResource(fileNameKey+".xml");
    
                    itemWriter.setName(fileNameKey);
                    itemWriter.setResource(resource);
                    marshaller.setPackagesToScan("com.model");
                    itemWriter.setMarshaller(marshaller);
                    itemWriter.afterPropertiesSet();
    
                    itemWriter.open(executionContext);
    
                } catch (Exception e) {
                    e.printStackTrace();
                }
                writers.put(fileNameKey, itemWriter);
            }
            return itemWriter;
        }
    }
    

    Hope this helps.