Search code examples
ruby-on-railsrubyelasticsearchtire

ElasticSearch async delete? 200 just after deleting index in Rails app

We're using ElasticSearch with the Tire gem on a Rails app. For our integration tests, we delete and recreate the index before each example, something along the lines of

Foo.index.delete
Foo.create_elasticsearch_index

Where Foo includes Tire::Model::Persistence. But we started having missing index errors from time to time when running our test suite on CI. I went ahead and enabled debugging and then I found the following in the logs:

# 2013-10-04 09:25:05:839 [DELETE] ("test_index")
#
curl -X DELETE http://some-server:9200/test_index

# 2013-10-04 09:25:05:840 [200]
#
# {
#   "ok": true,
#   "acknowledged": true
# }

# 2013-10-04 09:25:05:852 [HEAD] ("test_index")
#
curl -I "http://some-server:9200/test_index"

# 2013-10-04 09:25:05:852 [200]

As you can see, I get a 200 ok response for the DELETE request, but then when Tire does a HEAD request to see if the index exists before creating it, it still returns a 200 instead of a 404. This happens randomly, most of the times it works ok, but at some point in the test suite it will fail.

I tried waiting for a yellow status between the delete and the create operations, but it didn't work. So my question here would be, is the delete index operation asynchronous in some way? (Couldn't find anything on the ES doc about that). Is there any way to wait for the index to be deleted? (Since the yellow status doesn't work).

Edit: Thanks @PinnyM for the clarification. So all the operations through the HTTP API are indeed asynchronous. So the question that remains is, how can I wait for the index to be deleted so I can create it again?

Solution

  • Before each example, you delete and recreate the index. As part of that process you could then ensure that the delete process has completed (asynchronously) by polling for the exists function to return false... In crappy pseudo-code...

    max_wait = 5
while wait < max_wait and Tire.index("test_index").exists:
    wait some more
    if wait > max_wait:
        throw WaitedTooLongException()

    The Tire docs for exist are here - essentially they are doing the same index polling you found in the debug code: http://rubydoc.info/github/karmi/tire/master/Tire/Index#exists?-instance_method and here's the call: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-exists.html

    Personally, I would adjust your tests that use the ES instance... If you put the not-exists loop at the start of the test and the index tear-down at the end of the test, then you could get two benefits. You could:

    • Assert that the index didn't exist when testing started - nice and clean.
    • Ask for index tear down at the end of a test and run your next test which might not need ES - possible slight speed-up depending on your App and CI implementation of test ordering.