Search code examples
javaderbylucene

Lucene and External DB

I am working with the Lucene and Derby databases. Lucene contains the text index, and Derby has information regarding additional user data. For example, each document has a tag. For this purpose the Derby database has two tables

TAGS:

ID

Name

LUCENETAGS:

ID

LUCENEID (docID in Lucene, not a field)

TAGID

I want a user to be able to search something like:

very interesting text AND tag:fun

Changing the structure in a way that tag is a Lucene field is not an option.

Thank you!

Solution

  • I believe you'll have to simply perform your text search in Lucene, and then filter your results based on the result of a query into a Derby.

    If few documents will match a particular tag, you could also query the database for the IDs to be queried, and rewrite the query like:

    (very interesting text) AND id:(1 2 3 etc.)

    Probably not feasible, but in the case that tags are pretty sparse, it might be worth considering.

    I do wonder, though, why a field can't be added to the index, duplicating the stored value in the Derby Database. In any implementation you choose to get what you want from your stated structure, you will see much poorer performance, and more complexity for you to deal with, than if the data were available in the index as well.