Elasticsearch _bulk returns empty dict

Question

I have a strange behavior on my ES7.8 cluster, when inserting data using elasticsearch.helpers.streaming_bulk it will say this strange error:

  File "./lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 193, in streaming_bulk
    raise_on_error, *args, **kwargs)
  File "./lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 124, in _process_bulk_chunk
    for data, (op_type, item) in zip(bulk_data, map(methodcaller('popitem'), resp['items'])):
KeyError: u'items'

To debug this, I added a line to perform_request before return like this:

def perform_request(...):
    print(response.status, response.getheaders(), raw_data)
    return response.status, response.getheaders(), raw_data

in logs it shows:

200 HTTPHeaderDict({'content-length': '2', 'content-type': 'application/json; charset=UTF-8'}) {}

This looks really messed up. When carefully compare the input and actual ES docs, I find that there's 50% chance the data was successfully written into the index and the other 50% fails.

Why does the ES7.8 cluster return an empty response and breaks everything downwards? How I can further debug this?

Answer 1

Turns out it's my own stupidity. I specified filter_path=['hits.hits._id']) parameter all the way down during bulk request.

Thanks for the tip @Val.