Search code examples
elasticsearchbulk

Elastic Search bulk request does not import all data, but shows no error


I use GuzzleHttp to send data via "_bulk" to an Elastic Search index. It is only a small dataset of 850 records. When I transfer the data record by record, I get an error message for 17 records. That's fine for me, so I can fix the errors.

But when I use _bulk, I do not get any error message at all. The 17 incorrect records are just ignored and are missing inside the index. How can I get an error message here? Are there some kind of options that I can use? Any ideas?

The endpoint is:

Here are my main code parts:

$jsonData = "xxxxx"; // the payload for the request
$elasticUrl = "https://xxxx.xx/xxxxx/_doc/_bulk";

$client = new Client([
        "verify" => false, // disable ssl certificate verification
        "timeout" => 600, // maximum timeout for requests
        "http_errors" => false // disable exceptions
]);

$header = ["Content-Type" => "application/json"];

$result = $client->post($elasticUrl,
          [
            "headers" => $header,
            "body" => $jsonData
          ]
);
        
if ($result->getStatusCode() != 200) {
    $ret = "Error ".$result->getStatusCode()." with message: ".$result->getReasonPhrase();
}

Solution

  • As @Val pointed out the use of $response->getBody() gives the needed information:

    $body      = (string) $result->getBody();
    $bodyArray = json_decode($body, true);
    
    if ($bodyArray["errors"]) {
        $retArray = [];
        foreach ($bodyArray["items"] as $key => $item) {
            if (isset($item["create"]["error"])) {
                $retArray[] = $item["create"]["error"]["reason"].": ".json_encode($data[$key]);
            }
        }
        $ret = implode(", ", $retArray);
    }
    

    As side note: in $data I keep the data as php array before sending it to Elastic Search.