Spring Data Elasticsearch query by JSON structure

Question

I'm using spring data elasticsearch, and it is a lot easier to associate the code to the actual JSON elasticsearch query when I use the @Query annotation, as in the examples in this linked reference:

https://www.programcreek.com/java-api-examples/index.php?api=org.springframework.data.elasticsearch.annotations.Query

I was wondering if there is a way to make a query by the full JSON body via the elasticsearch java library without an annotation. I.E. within a method implementation or something. This will help me parse highlighting in the response, etc..

Thanks for any information.

Clarification from comments: I'm using spring-data-elasticsearch 3.0.10.RELEASE with Elasticsearch 6. As spring-data-elasticsearch does not seem to support the RestHighLevelClient yet, I'm using the TransportClient client = new PreBuiltTransportClient(elasticsearchSettings); approach when creating the ElasticsearchTemplate: return new ElasticsearchTemplate(client());

Answer 1

I figured out one way to do it but it requires you to make a script that lives on the Elastic node. See File-based scripts. It's not extremely flexible but give it a shot. Here's what to do.

Create a file named template_doctype.mustache and copy it to $ELASTIC_HOME/config/scripts. This is the script you could tailor as needed. Restart Elastic or wait 60 seconds for it to reload.

{
    "query" : {
        "match" : {
            "type" : "{{param_type}}"
        }
    }
}

My pom.xml dependencies:

<dependencies>
    <dependency>
        <groupId>org.springframework.data</groupId>
        <artifactId>spring-data-elasticsearch</artifactId>
        <version>3.0.10.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>transport</artifactId>
        <version>5.5.0</version>
    </dependency>
    <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
        <version>2.8.2</version>
        <scope>compile</scope>
    </dependency>
</dependencies>

(FYI, I discovered using mvn dependency:tree that your version of spring-data-elasticsearch implicitly uses the 5.5 version of the ElasticSearch library, even though you're using ElasticSearch 6.)

Create a dummy index:

curl -X PUT http://localhost:9200/myindex

Create a couple of documents that can be used to match to ensure the code works:

curl -X POST http://localhost:9200/myindex/mydoc -d '{"title":"foobar", "type":"book"}'
curl -X POST http://localhost:9200/myindex/mydoc -d '{"title":"fun", "type":"magazine"}'

Try running a query. This code should return a single document:

String clusterName = "my-application";
Settings elasticsearchSettings = Settings.builder().put("cluster.name", clusterName).build();
TransportClient client = new PreBuiltTransportClient(elasticsearchSettings)
        .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"),9300));
Map<String, Object> template_params = new HashMap<>();

// Here is where you put parameters to your script.
template_params.put("param_type", "book");
SearchResponse sr = new SearchTemplateRequestBuilder(client)
        .setScript("template_doctype")  // this is where you specify what template to use
        .setScriptType(ScriptType.FILE)
        .setScriptParams(template_params)
        .setRequest(new SearchRequest())
        .get()
        .getResponse();

SearchHit[] results = sr.getHits().getHits();
for(SearchHit hit : results){

    String sourceAsString = hit.getSourceAsString();
    if (sourceAsString != null) {
        Gson gson = new GsonBuilder().setPrettyPrinting()
                .create();
        Map map = gson.fromJson(sourceAsString, Map.class);
        System.out.println( gson.toJson(map));
    }
}

Output:

{
  "title": "foobar",
  "type": "book"
}