Search code examples
c#lucenefull-text-search

What is the best full text search open source project (.NET preferred)?

I've developed an index and search application with Lucene library. but this library has some limitation in custom ranking in my context, aside from its performance, i need scalability and access to all kinds of word frequencies and etc. is there any powerful open source full text library available?

Solution

  • http://www.sphinxsearch.com

    http://www.sphinxconnector.net/

    Key Sphinx features are:

    • high indexing and searching performance;
    • advanced indexing and querying tools (flexible and feature-rich text tokenizer, querying language, several different ranking modes, etc);
    • advanced result set post-processing (SELECT with expressions, WHERE, ORDER BY, GROUP BY etc over text search results);
    • proven scalability up to billions of documents, terabytes of data, and thousands of queries per second;
    • easy integration with SQL and XML data sources, and SphinxAPI, SphinxQL, or SphinxSE search interfaces;
    • easy scaling with distributed searches.

    To expand a bit, Sphinx:

    • has high indexing speed (upto 10-15 MB/sec per core on an internal benchmark);
    • has high search speed (upto 150-250 queries/sec per core against 1,000,000 documents, 1.2 GB of data on an internal benchmark);
    • has high scalability (biggest known cluster indexes over 3,000,000,000 documents, and busiest one peaks over 50,000,000 queries/day);
    • provides good relevance ranking through combination of phrase proximity ranking and statistical (BM25) ranking;
    • provides distributed searching capabilities;
    • provides document excerpts (snippets) generation;
    • provides searching from within application with SphinxAPI or SphinxQL interfaces, and from within MySQL with pluggable SphinxSE storage engine;
    • supports boolean, phrase, word proximity and other types of queries;
    • supports multiple full-text fields per document (upto 32 by default);
    • supports multiple additional attributes per document (ie. groups, timestamps, etc);
    • supports stopwords;
    • supports morphological word forms dictionaries;
    • supports tokenizing exceptions;
    • supports both single-byte encodings and UTF-8;
    • supports stemming (stemmers for English, Russian and Czech are built-in; and stemmers for French, Spanish, Portuguese, Italian, Romanian, German, Dutch, Swedish, Norwegian, Danish, Finnish, Hungarian, are available by building third party libstemmer library);
    • supports MySQL natively (all types of tables, including MyISAM, InnoDB, NDB, Archive, etc are supported);
    • supports PostgreSQL natively;
    • supports ODBC compliant databases (MS SQL, Oracle, etc) natively;
    • ...has 50+ other features not listed here, refer to API and configuration manual!