How can I split a CSV file into different CSV files by line in Java?

Question

I have a class which reads a CSV file but when size of file is high, the program throws Java heap size error, so I need to split that file into pieces and transfer lines to other files according to line size.

For example; I have a file of 500 000 lines and I'm dividing it into 5 files by 100 000 lines. So I have 5 files consisting of 100 000 lines so that I can read them.

I couldn't find a way to do that so it would be nice if I see example lines of code.

Answer 1

public static void splitLargeFile(final String fileName, 
                                   final String extension, 
                                   final int maxLines,
                                   final boolean deleteOriginalFile) {

    try (Scanner s = new Scanner(new FileReader(String.format("%s.%s", fileName, extension)))) {
        int file = 0;
        int cnt = 0;
        BufferedWriter writer = new BufferedWriter(new FileWriter(String.format("%s_%d.%s", fileName, file, extension)));

        while (s.hasNext()) {
            writer.write(s.next() + System.lineSeparator());
            if (++cnt == maxLines && s.hasNext()) {
                writer.close();
                writer = new BufferedWriter(new  FileWriter(String.format("%s_%d.%s", fileName, ++file, extension)));
                cnt = 0;
            }
        }
        writer.close();
    } catch (Exception e) {
        e.printStackTrace();
    }

    if (deleteOriginalFile) {
        try {
            File f = new File(String.format("%s.%s", fileName, extension));
            f.delete();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}