Search code examples
sphinxthinking-sphinx

Sphinx ranged query

I have a Sphinx instance with two indexes: main and delta. The delta index is selecting documents which were modified after the last merge. this includes documents with new ids and documents with already existing ids.

I followed the common approach to create a ranged query (min and max ids from documents and a step of 1000 records). The problem with this approach is that in a database with 1M documents, there will be loads of queries returning nothing, because the ranged query covers the full range of ids. i.e:

sql_query_range = SELECT MIN(id),MAX(id) FROM documents
sql_range_step = 1000
sql_query = SELECT * FROM documents where modified_date > (select merge_date from SphinxTable)
WHERE id>=$start AND id<=$end

is there any way that i can define my sql range query so that the indexer only iterates through modified records instead of the whole database ? (and execute less queries because of a shortened range ofc)

Solution

  • As noted in my reply to your other question, would do something like

    sql_query_range   = SELECT (SELECT maxID from SphinxTable),MAX(id) FROM documents