Elasticsearch with django-haystack run update index automatically

Question

I am developing a open blogging website, for its search functionality I am using Elastic search with django-haystack but the problem is after every blog post I need to run the command - python manage.py update_index, how to make update_index automatically ? and could you please tell me if site has millions of posts then, Is this a good idea or it will crash my site because I need to refresh index every time, I am newbie please tell me the correct way for heavy search. What search technologies does instagram and other social websites uses.

Answer 1

You can enable real time updating by adding this to settings.py:

HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'

More details here:

http://django-haystack.readthedocs.io/en/v2.4.1/signal_processors.html#realtime-realtimesignalprocessor

Where the re-indexing is likely to take some time you should use a queue to prevent the request/response cycle being impeded, possible solutions such as celery are suggested here:

http://django-haystack.readthedocs.io/en/v2.4.1/other_apps.html#ref-other-apps