Search code examples
powershellcurlelasticsearch

Problems bulk insert elasticsearch curl powershell

I'm new to Elasticsearch and I'm following the tutorial loading sample data: https://www.elastic.co/guide/en/kibana/current/tutorial-load-dataset.html When I try to bulk insert the data in powershell 'curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary "@shakespeare.json"'

I get the following error

PS C:\Development\elasticsearch-5.4.1\importdata> curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary "@shakespeare.json" Invoke-WebRequest : Cannot bind parameter 'Headers'. Cannot convert the "Content-Type: application/x-ndjson" value of type "System.St ring" to type "System.Collections.IDictionary". At line:1 char:9 + curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/s ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (:) [Invoke-WebRequest], ParameterBindingException + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

I've tried several things, read the bulk api documentation but with no result

Here're some examples of the content of the json file:

{"index":{"_index":"shakespeare","_type":"act","_id":0}} {"line_id":1,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"} {"index":{"_index":"shakespeare","_type":"scene","_id":1}} {"line_id":2,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"SCENE I. London. The palace."} {"index":{"_index":"shakespeare","_type":"line","_id":2}} {"line_id":3,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"Enter KING HENRY, LORD JOHN OF LANCASTER, the EARL of WESTMORELAND, SIR WALTER BLUNT, and others"}

Solution

  • I found the solution. Because I used curl on windows and with powershell, I had to translate 

    curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/shakespeare/_bulk?pretty' --data-binary @shakespeare.json

    to

    PS C:\Development\elasticsearch-5.4.1\importdata> $postParams = @"
>> {"index":{"_index":"shakespeare","_type":"act","_id":0}}
>> {"line_id":1,"play_name":"Henry IV","speech_number":"","line_number":"","speaker":"","text_entry":"ACT I"}
>>
>> "@

PS C:\Development\elasticsearch-5.4.1\importdata> curl -H @{"Content-Type" = "application/x-ndjson"} -Method POST 'http://localhost:92
00/shakespeare/_bulk?pretty' -body $postParams

    When I use the demo file, the bulk insert didn't succeeded but that's another problem

    PS C:\Development\elasticsearch-5.4.1\importdata> $postParams = Get-Content "shakespeare.json"