Search code examples
elasticsearchelasticsearch-py

Trying to load json docs in elasticsearch

I am trying to follow this answer, but I am getting error in the actions parameter of bulk method. I am able to use next to generate array of json objects but when I pass it to helpers.bulk I get error.

This is my code:

from elasticsearch import Elasticsearch, helpers
import sys, json
import os

es = Elasticsearch("localhost:9200")
filename = "path_to_file.json"
def load_json(filename):
#     " Use a generator, no need to load all in memory"
print(filename)
with open(filename,'r') as open_file:
    yield json.load(open_file)`

helpers.bulk(es, load_json(filename), index='cllimaster_test', doc_type='clli_data_test')

Error:

enter image description here

Solution

  • This python (not ES) error would occur if .pop() had been called on a list instead of a dict. I haven't seen your json file but here's your linted code which works perfectly fine:

    from elasticsearch import Elasticsearch, helpers
import sys
import json
import os

es = Elasticsearch("localhost:9200")
filename = "path_to_file.json"


def load_json(filename):
    with open(filename, 'r') as open_file:
        yield json.load(open_file)


helpers.bulk(es, load_json(filename), index='cllimaster_test',
             doc_type='clli_data_test')

print(es.count(index='cllimaster_test'))