Search code examples
javacsvfilebufferedreaderbufferedwriter

Time-efficient way to append to existing header in existing CSV file in Java?


I'd like to modify the first line or header, of an existing csv file by adding a string to the end of that line.

I've tried using BufferedWriter to do so, but I can only get it to append at the end of the file.

My working code:

      public static void writeStringtoCsvFile(String filePath, String input) throws IOException {
    PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(filePath, true)));
    out.append(input);
    out.close();
  }

It seems OpenCsv doesnt have an available method to append to an existing file either. The file I need to process is at least 160mb or 1+mil records in size, would the alternative method:

  1. BufferedReader to read all the lines
  2. Append whatever is needed to first line
  3. BufferedWriter to write everything to a new file

be too slow? Is there a more elegant solution to this? Thanks!


Solution

  • File systems do not allow you to insert bytes into or delete bytes from the middle of a file without rewriting all of the bytes after the insertion / deletion point.

    Since the file system doesn't support it, the Java file I/O APIs can't support it.

    Since the Java file I/O APIs don't support it, neither can the OpenCvs APIs. (Or at least not without rewriting the file under the covers.)

    So, the answer is that there isn't a more efficient way than reading writing all of the lines of the file.

    Notes:

    1. While it is technically possible to do an in-place rewrite of a file, it is safer (and possibly more efficient) to create a new file, write to it and rename it when the writing is completed.

    2. Reading and writing a line at a time (using BufferedReader / BufferedWriter) is better than reading all lines into memory at the same time, then writing them all out. Especially if the files are large.