Search code examples
lucenequery-parser

Lucene Queryparser with multiple fields


I use Lucene 5.3 and try to search over multiple fields using the queryparser-syntax. I found within the Lucene Tutorials a short example and modified it to Version 5.3 and to search over those fields.

package lucenewriterexample;

import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.StringField;
import org.apache.lucene.document.TextField;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.queryparser.classic.ParseException;
import org.apache.lucene.queryparser.classic.QueryParser;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TopScoreDocCollector;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.RAMDirectory;

import java.io.IOException;

public class LuceneWriterExample {


    public static void main(String[] args) throws IOException, ParseException {
        StandardAnalyzer analyzer = new StandardAnalyzer();
        Directory index = new RAMDirectory();
        IndexWriterConfig config = new IndexWriterConfig(analyzer);
        try (IndexWriter writer = new IndexWriter(index, config)) {
            addDoc(writer, "Day first : Lucence Introduction test.", "3436NRX");
            addDoc(writer, "Day second , part one : Lucence Projects.", "3437RJ1");
            addDoc(writer, "Day second , part two: Lucence Uses testing rr.", "3437RJ2");
            addDoc(writer, "Day third : Lucence Demos.", "34338KRX");
        }

        String querystr = "title:(part) AND course_code:(3437RJ1)";
        Query q = new QueryParser("title", analyzer).parse(querystr);

        // 3. searching
        int hitsPerPage = 10;
        IndexReader reader = DirectoryReader.open(index);
        IndexSearcher searcher = new IndexSearcher(reader);
        TopScoreDocCollector collector = TopScoreDocCollector.create(hitsPerPage);
        searcher.search(q, collector);
        ScoreDoc[] hits = collector.topDocs().scoreDocs;

        // 4. display results
        System.out.println("Query string: " + querystr );
        System.out.println("Found " + hits.length + " hits.");        
        for (int i = 0; i < hits.length; ++i) {
            int docId = hits[i].doc;
            Document d = searcher.doc(docId);
            System.out.println((i + 1) + ". " + d.get("course_code") + "\t" + d.get("title"));
        }

        // Finally , close reader
    }

    private static void addDoc(IndexWriter w, String title, String courseCode) throws IOException {
        Document doc = new Document();
        doc.add(new TextField  ("title",       title,      Field.Store.YES));
        doc.add(new StringField("course_code", courseCode, Field.Store.YES));
        w.addDocument(doc);
    }

The queryparser is working for "title:part", then I get all documents containing "part", but if I use "title:(part) AND course_code:(3437RJ1)" or "title:(part) OR course_code:(3436NRX) the result is 0. Where is the mistake within the search?


Solution

  • I ran this and tried title:(part) OR course_code:(3436NRX), and I get 2 results, exactly as I would expect. Perhaps you meant you were expecting a third result matching the course_code, but didn't get it. If you really meant you actually got zero results with that query, I'm not sure what the problem is.

    So, why aren't you able to get a match on course_code?

    As is so often then case with lucene, you have mismatched analyzers. You queryparser is using StandardAnalyzer, but course_code is a StringField, and so it is not being analyzed at all. StandardAnalyzer includes a filter to lowercase everything, so the end result is you have a field that has 3436NRX, and a query for course_code:3436nrx.

    Possible solutions would be:

    • Use a TermQuery instead of the query parser for your StringFields
    • Lowercase your course_code yourself before passing it to lucene
    • Make course_code a TextField

    etc.