Bulk indexing in elasticsearch 8.x : 'Action/metadata line [1] contains an unknown parameter [_type]'

Question

I have been trying to do a bulk index in the elasticsearch 8.x using this:

from elasticsearch.helpers import bulk as bulk_indexer

success, failed = bulk_indexer(self.es_client, actions, stats_only=True, chunk_size=900)

Apparently i am seeing :

RuntimeError: elasticsearch.BadRequestError: BadRequestError(400, 'illegal_argument_exception', 'Action/metadata line [1] contains an unknown parameter [_type]') [while running 'bulk index']

I came to know about https://www.elastic.co/guide/en/elasticsearch/reference/7.17/removal-of-types.html

and i think this is the reason for it , but can someone help me to know why i am seeing this or what could be the issue?

I tried to migrate from es 5 to es 8.x

Answer 1

There must be a _type in the action command line that shouldn't be there anymore. If those records have been stored in Kafka in earlier versions of ES and you're replaying them, or if the client application still produces those action records with the same logic, then you either need to change that producing logic and/or post-process those records in your bulk indexer to remove the _type field.