Search code examples
javaapache-commons-io

Util method to get Line by Line#

Is there any Util method to get the line contents by Line# from given file?

Solution

  • The simplest approach is to read all the lines into a list and look up the line by number in this list. You can use 

    List<String> lines = FileUtils.readLines(file);

    My file is 3GB and I don't want to store all the lines in my java memory

    I would make sure you have plenty of memory. You can buy 32 GB for less than $200.

    However, assuming this is not an option you can index the file by reading it once storing the offset of each line in another file. It could be a 32-bit offset, but it would simpler/more scalable if you used a 64-bit offset.

    You can then lookup the offset of each line and the next one to determine where to read each line. I would expect this to take about 10 micro-seconds if implemented efficiently.

    BTW: If you had it loaded in Java memory it would be about 100x faster.