Search code examples
sphinx

main/delta split marker with sphinx using xmlpipe2?


we are going to have sphinx running main+delta with nosql source. So we're going to use xmlpipe2. To implement main+delta strategy we have to maintain a marker to distinguish "main rows" from "new rows".

The problem is that, unlike sql sources, with xmlpipe2 we can't tell (at least don't know how) if the indexing was successful or not. If we simply update the marker at the end of the main feed generator, and indexing fails for whatever reason, the setup is going to be in inconsistent state.

For SQL sources we have sql_query_post_index hook, how can we achieve similar thing with xmlpipe2?


Solution

  • You could have a wrapper around indexer. that wrapper runs indexer, captures the output, and if the index worked, then it updates your 'marker'.

    Call this wrapper from cron, rather can calling indexer directly