Connecting ElasticSearch to Django using "django-elasticsearch-dsl" results in ConnectionError when attempting to create/rebuild index

Question

I am trying to call a local ES instance running on docker. I used the following instructions to setup my ES instance: https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started.html https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-cli-run-dev-mode

I am able to play around with my instance on Kibana at http://0.0.0.0:5601/app/dev_tools#/console. Everything works upto here.

Now I am trying to define some sample documents using django models and index them through the library; I am following the instructions here: https://django-elasticsearch-dsl.readthedocs.io/en/latest/quickstart.html#install-and-configure

First, I add pip installed and added django_elasticsearch_dsl to INSTALLED_APPS

Next, I added in settings.py:

ELASTICSEARCH_DSL = {
    'default': {
        'hosts': 'localhost:9200'
    },
}

Then I create a sample model and document that looks like this:

# models.py

from django.db import models

class Car(models.Model):
    name = models.CharField(max_length=30)
    color = models.CharField(max_length=30)
    description = models.TextField()
    type = models.IntegerField(choices=[
        (1, "Sedan"),
        (2, "Truck"),
        (4, "SUV"),
    ])

# documents.py

from django_elasticsearch_dsl import Document
from django_elasticsearch_dsl.registries import registry
from .models import Car

@registry.register_document
class CarDocument(Document):
    class Index:
        # Name of the Elasticsearch index
        name = 'cars'
        # See Elasticsearch Indices API reference for available settings
        settings = {'number_of_shards': 1,
                    'number_of_replicas': 0}

    class Django:
        model = Car # The model associated with this Document

        # The fields of the model you want to be indexed in Elasticsearch
        fields = [
            'name',
            'color',
            'description',
            'type',
        ]

Finally, running python3 manage.py search_index --rebuild results in the following connection error:

    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))

I suspect that there might be an issue with my ELASTICSEARCH_DSL setup as I have not specified any config for https but the documentation does not make this clear.

How do I resolve this issue?

Django version: 
Django==4.0.1
django-elasticsearch-dsl==7.2.2

Python version: Python 3.9.10

Thanks!

Answer 1

I figured that it was an issue with my certificate.

I needed to add some additional config param to the ELASTICSEARCH_DSL variable. Adding this solves the issue:

from elasticsearch import RequestsHttpConnection

# Elasticsearch configuration in settings.py
ELASTICSEARCH_DSL = {
    'default': {
        'hosts': 'localhost:9200',
        'use_ssl': True,
        'http_auth': ('user', 'password'),
        'ca_certs': '/path/to/cert.crt'
        'connection_class': RequestsHttpConnection
    }
}

See this section of the elastic docs on verifying a certificate.

If you haven't setup a certificate to authenticate the connection with and just need to get something up and running quickly, you can pass 'verify_certs': False and set 'ca_certs' to None.