Search code examples
oraclelucenebigdatafuzzy-searchstring-search

How to search for multiple strings in very large database


I want to search for multiple strings in a very large database. These strings are part of different attributes of database table. I have tried string search using LIKE in sql query. But it is taking a lot of time to get results. I have used Oracle database.

Should I use indexing of database? I found that Lucene can be used for it. I also got some suggestions of using big data concepts. Which approach should I use?


Solution

  • The easiest way is: 1.) adding an index to the columns you like to search trough 2.) using oracle text as @lalitKumarB wrote

    The most powerful way is: 3.) use an separate search engine (solr, elaticsearch).

    But, probably you have to change you application in order to explicit use the search index for searching trough the data,...

    I had the same situation some years before. Trying to search text in an big database. After a wile I found out, that database based search will never reach the performance of an dedicate search engine. And: you will have much more search features working out of the box, if you use solr (for example), like spelling correction, "More like this", ...

    One option is to hold the data on orcale, searching in solr and return the ID of the document in order to only load the one row form oracle, the is referenced by the ID. 2nd option is to keep oracle as base datapool for your search engine and search in solr (or elasticsearch) in order to return the whole document/row from solr, not only the ID. So you don't need to load the data from the database any more.

    The best option depends on your needs.