Search code examples
lucenefull-text-searchlucene.netasp.net-core-6.0

Lucene.net corrupted index (segments.gen)


We're using Lucene.NET library (4.8.0-beta000015) for our search engine. Our server was shut down unexpectedly and ever since then we're getting this error:

Format version is not supported (resource: BufferedChecksumIndexInput(MMapIndexInput(path="index\path\to\segments.gen"))): 0 (needs to be between -2 and -3) at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit) at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run()

Is there any solution to fixing this without recreating index? Or how can we prevent this for happening again?

Our system runs on .netcore 6

Couldnt find any solution only related topics but all said to recreate index. We cant afford this since full index creation takes about 1 day.


Solution

  • You could attempt to run the index fix command to repair the broken segments and then re-add any documents that had to be removed to fix the segments.

    IMPORTANT: Always make a backup copy of your index before running this command.

    Install the lucene-cli tool. Make sure you are using the same version you used to create the index.

    dotnet tool install lucene-cli -g --version 4.8.0-beta00015
    

    Next, run the index fix command with the --dry-run option to see what actions it will take. Redirect the standard output to a log file in case there is more output than will fit in the console buffer.

    lucene index fix [<INDEX_DIRECTORY>] --verbose --dry-run
    

    Take note of any documents that the command will delete and prepare a method to recreate them. Also take note of how much time the command takes, as you will need at least this amount of downtime on your server to do the fix (probably more).

    Finally, run the command to fix the index. You must shut down any writes to the index for the duration of the operation. It may also be a good idea to shut down all reads if you can afford to do so.

    lucene index fix [<INDEX_DIRECTORY>] --verbose
    

    References