Search code examples
amazon-web-servicesamazon-elasticsearch

Error inserting json file in aws elasticsearch

I have a small json file am using t2 instance in elasticsearch, but I am unable to insert the data in aws elasticsearch

my file looks like following

[{"Company_Name": "11 plc (NGSE:MOBIL)", "Business_Description": "11 plc markets petroleum products in Nigeria."}, {"Company_Name": "3Power Energy Group, Inc. (OTCPK:PSPW)", "Business_Description": "3Power Energy Group, Inc. focuses on developing, building, and operating power plants."}, {"Company_Name": "4Sight Holdings Limited (JSE:4SI)", "Business_Description": "4Sight Holdings Limited, through its subsidiaries, provides technology support solutions across various industries in Mauritius."}, {"Company_Name": "A'ayan Leasing and Investment Company K.S.C.P. (KWSE:AAYAN)", "Business_Description": "A'ayan Leasing and Investment Company K.S.C.P. engages in financial investments, trading and investing in properties"}]

I have tried using the following command

curl -s -XPOST https://elasticsearchdomain/_bulk?pretty -H "Content-Type: application/x-ndjson" --data-binary '@data.json'

I get status 400 error

{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "The bulk request must be terminated by a newline [\\n]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "The bulk request must be terminated by a newline [\\n]"
  },
  "status" : 400
}

what am I doing wrong?

Solution

  • The Bulk API in Elastic Search does not do what you expect it does. The bulk API is used to perform multiple operations (e.g. Index, Update, Delete) in one API call. So you have to specify the specific type of operation you want to perform. In this case you want to index multiple documents:

    curl -X POST "localhost:9200/_bulk?pretty" -H 'Content-Type: application/json' -d'
{ "index" : { "_index" : "myindex", "_id" : "1" } }
{"Company_Name": "11 plc (NGSE:MOBIL)", "Business_Description": "11 plc markets petroleum products in Nigeria."}
{ "index" : { "_index" : "myindex", "_id" : "2" } }
{"Company_Name": "3Power Energy Group, Inc. (OTCPK:PSPW)", "Business_Description": "3Power Energy Group, Inc. focuses on developing, building, and operating power plants."}
'

    So for each document you specify the operation (index in this case), terminate the operation by newline and then specify the document (terminated by newline) again.