Reading from elastic search using Spring boot, where ES record without _class attribute

Question

I am trying to read/search the existing records in Elastic Search via Spring Data + Elastic Search.The existing records in the elastic search does not have the _class attribute.

 {  "_index": "status-000001",  
    "_type": "_doc",
    "_id": "demo",
    "_version": 1,
    "_score": null,
    "_source": {
    "Header.Type": "1",
    "Header.Version": "1.0.0",
    "Data.sample_rate": "10",
    "Data.upload_rate": "60",
    "Data.error_code": "0x00000000",
  },
}

When I went through the documentation for Mapping rules, the documentation calls out for the need of _class. https://docs.spring.io/spring-data/elasticsearch/docs/current-SNAPSHOT/reference/html/#elasticsearch.mapping.meta-model.rules

But at this point, we won't be able to update the format in which data is ingested into ES.

The mapping for the index

{
  "status-000001": {
    "mappings": {
      "properties": {
        "Data": {
          "properties": {
                "error_code": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "sample_rate": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                },
                "upload_rate": {
                  "type": "text",
                  "fields": {
                    "keyword": {
                      "type": "keyword",
                      "ignore_above": 256
                    }
                  }
                }          
            }
          }
        },
        "Header": {
          "properties": {
            "Type": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "Version": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            }
          }
        }
      }
    }
  }
}

Are there any recommendations, as to how to handle such scenario?

Answer 1

The problem here is not the missing _class entry. Spring Data Elasticsearch can read entities without this (it is really needed when dealing with inherited classes and collections).

Normally an entity like this would be all you'd need (I leave out the getter and setter for brevity):

@Document(indexName = "status")
public class Status {
    @Id
    private String id;

    @Field(name = "Header", type = FieldType.Object)
    private Header header;

    @Field(name = "Data", type = FieldType.Object)
    private Data data;

    static class Header {
        @Field(name = "Version", type = FieldType.Text)
        private String version;

        @Field(name = "Type", type = FieldType.Text)
        private String type;

    }

    static class Data {
        @Field(name = "error_code", type = FieldType.Text)
        private String errorCode;
        @Field(name = "sample_rate", type = FieldType.Text)
        private String sampleRate;
        @Field(name = "upload_rate", type = FieldType.Text)
        private String uploadRate;
    }

}

The problem is that your elasticsearch documents have the object structure in dot-notation keys:

{
    "Header.Type": "1",
    "Header.Version": "1.0.0",
    "Data.sample_rate": "10",
    "Data.upload_rate": "60",
    "Data.error_code": "0x00000000"
}

It would be no problem for Spring Data Elasticsearch if the _source would be:

{
    "Header": {
        "Type": "1",
        "Version": "1.0.0"
    },
    "Data": {
        "sample_rate": "10",
        "upload_rate": "60",
        "error_code": "0x00000000"
    }
}

What you can do is write a custom converter like this (no error checks here):

import org.springframework.core.convert.converter.Converter;
import org.springframework.data.convert.ReadingConverter;
import org.springframework.data.elasticsearch.core.document.Document;

import java.util.Map;

@ReadingConverter
public enum MapToStatusConverter implements Converter<Map<String, Object>, Status> {

    INSTANCE;

    @Override
    public Status convert(Map<String, Object> source) {

        if (source instanceof Document) {
            Document document = (Document) source;

            Status status = new Status();
            status.setId(document.getId());

            Status.Header header = new Status.Header();
            header.setType(document.getString("Header.Type"));
            header.setVersion(document.getString("Header.Version"));
            status.setHeader(header);

            Status.Data data = new Status.Data();
            data.setSampleRate(document.getString("Data.sample_rate"));
            data.setUploadRate(document.getString("Data.upload_rate"));
            data.setErrorCode(document.getString("Data.error_code"));
            status.setData(data);

            return status;
        }

        return null;
    }
}

This converter needs to be registered when configuring Spring Data Elasticsearch:

@Configuration
public class RestClientConfig extends AbstractElasticsearchConfiguration {

    @Override
    @Bean
    public RestHighLevelClient elasticsearchClient() {

        ClientConfiguration clientConfiguration = ClientConfiguration.builder() //
            .connectedTo("localhost:9200") //
            .build();

        return RestClients.create(clientConfiguration).rest();
    }

    @Override
    public ElasticsearchCustomConversions elasticsearchCustomConversions() {
        Collection<Converter<?, ?>> converters = new ArrayList<>();
        converters.add(MapToStatusConverter.INSTANCE);
        return new ElasticsearchCustomConversions(converters);
    }
}

With this you should be able to read the data from Elasticsearch.