Search code examples
javajsonelasticsearchmappingbulk

I am working on ES for the first time. I am trying to upload a bulk json file into a new Elasticsearch index in java but I am getting errors


I am creating a new index and then mapping the json data

        CreateIndexRequest request = new CreateIndexRequest("demoreport");
        request.mapping(
            "{\n" +
                    "      \"properties\": {\n" +
                    "        \"Identifier code\": {\n" +
                    "          \"type\": \"char(3)\"\n" +
                    "        },\n" +
                    "        \"User ID\": {\n" +
                    "          \"type\": \"char(38)\"\n" +
                    "        },\n" +
                    "      }\n" +
                    "}",XContentType.JSON);
  HttpPost post = new HttpPost("http://localhost:9200/demoreport/_doc/5");

My JSON File:

[
{
    "IDENTIFIER_CD": "PT ",
    "USER_ID": "123458 
}
{
    "IDENTIFIER_CD": "SR ",
    "USER_ID": "12345678  
}
]

I am getting error as :

java.lang.IllegalArgumentException: mapping source must be pairs of fieldnames and properties definition.

here is the complete code : Here I am trying to post the Json file generated from the database to elasticsearch server.

import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.FileEntity;
import org.apache.http.impl.client.DefaultHttpClient;
import org.elasticsearch.action.admin.indices.create.CreateIndexRequest;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.common.xcontent.XContentType;
public class MyScheduler {
public static void send() {
    CreateIndexRequest request = new CreateIndexRequest("kibanareport");
    request.settings(Settings.builder()
            .put("index.number_of_shards", 20)
            .put("index.number_of_replicas", 10)
    );
    request.mapping(
            "{\n" +
                    "      \"properties\": {\n" +
                    "        \"user_ID\": {\n" +
                    "          \"type\": \"text\"\n" +
                    "        },\n" +
                    "        \"doc_ID\": {\n" +
                    "          \"type\": \"text\"\n" +
                    "        },\n" +
                    "      }\n" +
                    "}", XContentType.JSON);
    Map<String, Object> message = new HashMap<>();
    message.put("type", "text");
    Map<String, Object> properties = new HashMap<>();
    properties.put("message", message);
    Map<String, Object> mapping = new HashMap<>();
    mapping.put("properties", properties);
    XContentBuilder builder = XContentFactory.jsonBuilder();
    builder.startObject();
    {
        builder.startObject("properties");
        {
            builder.startObject("message");
            {
                builder.field("type", "text");
            }
            builder.endObject();
        }
        builder.endObject();
    }
    builder.endObject();
    String fileName = "downloads/JSONFile.json";
    File jsonFile = new File(fileName);
    HttpEntity entity = new FileEntity(jsonFile);
    HttpPost post = new
            HttpPost("http://localhost:9200/kibanareport/_doc");
    post.setEntity(entity);
    HttpClient client = new DefaultHttpClient();
    post.addHeader("content-type", "application/json");
    post.addHeader("Accept", "application/json");
    HttpResponse response = client.execute(post);
    System.out.println("Response: " + response);
  }
}

Solution

  • Datatype char(3) that you have used in your CreateIndexRequest is wrong. Please refer to this Elasticsearch docs for field datatype supported by Elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html

    Currently for your mapping I have used text datatype for your Identifier code since it is string and Integer for User ID field. You can change as per your requirements.

    Here , is the updated CreateIndexRequest :

    CreateIndexRequest request = new CreateIndexRequest("demoreport");
        request.mapping(
                    "{\n" +
                    "      \"properties\": {\n" +
                    "        \"Identifier code\": {\n" +
                    "          \"type\": \"text\"\n" +
                    "        },\n" +
                    "        \"User ID\": {\n" +
                    "          \"type\": \"integer\"\n" +
                    "        }\n" +
                    "      }\n" +
                    "}",XContentType.JSON);