How to create a generic FlatFileItemReader to read CSV files with different headers?

Question

I'm creating a job that will read and process different .csv files based on an input parameter. There are 3 different types of .csv files with different headers. I want to map each line of a file to a POJO using a generic FlatFileItemReader.

Each type of file will have its own POJO implementation, and all "File Specific POJOs" are subclassed from an abstract GenericFilePOJO.

A tasklet will first read the input parameter to decide which file type needs to be read, and construct a LineTokenizer with the appropriate header columns. It places this information in the infoHolder for retrieval at the reader step.

@Bean
public FlatFileItemReader<GenericFilePOJO> reader() {
    FlatFileItemReader<RawFile> reader = new FlatFileItemReader<GenericFilePOJO>();
    reader.setLinesToSkip(1); // header

    reader.setLineMapper(new DefaultLineMapper() {
        {
            // The infoHolder will contain the file-specific LineTokenizer
            setLineTokenizer(infoHolder.getLineTokenizer());
            setFieldSetMapper(new BeanWrapperFieldSetMapper<GenericFilePOJO>() {
                {
                    setTargetType(GenericFilePOJO.class);
                }
            });
        }
    });
    return reader;
}

Can this reader handle the different File Specific POJOs despite returning the GenericFilePOJO?

Answer 1

You wrote:

A tasklet will first read the input parameter to decide which file type needs to be read.

Because the tasklet or infoHolder knows about type of file you can implement the creation of specific FieldSetMapper instance.

This is a demo example how it can be implemented:

public class Solution<T extends GenericFilePOJO> {
    private InfoHolder infoHolder = new InfoHolder();

    @Bean
    public FlatFileItemReader<T> reader()
    {
        FlatFileItemReader<T> reader = new FlatFileItemReader<T>();
        reader.setLinesToSkip(1);

        reader.setLineMapper(new DefaultLineMapper() {
            {
                setLineTokenizer(infoHolder.getLineTokenizer());
                setFieldSetMapper(infoHolder.getFieldSetMapper());
            }
        });
        return reader;
    }

    private class InfoHolder {
        DelimitedLineTokenizer getLineTokenizer() {
            return <some already existent logic>;
        }

        FieldSetMapper<T> getFieldSetMapper() {
            if (some condition for specific file POJO 1){
                return new BeanWrapperFieldSetMapper<T>() {
                    {
                        setTargetType(FileSpecificPOJO_1.class);
                    }
                };
            } else if (some condition for specific file POJO 2){
                return new BeanWrapperFieldSetMapper<T>() {
                    {
                        setTargetType(FileSpecificPOJO_2.class);
                    }
                };
            }
        }
    }
}