I have a custom @Query in one of my elasticsearch repositories because the autoGenerated method didn't use match (instead query_string with analyze_wildcard) and so didn't work for example with spaces. This query looks pretty simple to me so I thought it wouldn't be a problem to write it myself.
@Query("\"bool\": { " +
" \"filter\": [ " +
" { " +
" \"term\": { " +
" \"userId.keyword\": \"?0\" " +
" } " +
" }, " +
" {" +
" \"match\": { " +
" \"content\": \"?1\" " +
" }" +
" } " +
" ] " +
" }")
Page<SearchablePageHistory> findAllByUserIdAndContentLike(String userId, String content, Pageable pageable);
But when I try to execute that function I get the following error:
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=x_content_parse_exception, reason=Failed to derive xcontent]
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) ~[elasticsearch-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1793) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1770) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1527) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1484) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1454) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:970) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.lambda$search$10(ElasticsearchRestTemplate.java:265) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.execute(ElasticsearchRestTemplate.java:351) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.elasticsearch.core.ElasticsearchRestTemplate.search(ElasticsearchRestTemplate.java:265) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.elasticsearch.repository.query.ElasticsearchStringQuery.execute(ElasticsearchStringQuery.java:89) ~[spring-data-elasticsearch-4.0.0.RELEASE.jar:4.0.0.RELEASE]
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor$QueryMethodInvoker.invoke(QueryExecutorMethodInterceptor.java:195) ~[spring-data-commons-2.3.0.RELEASE.jar:2.3.0.RELEASE]
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:152) ~[spring-data-commons-2.3.0.RELEASE.jar:2.3.0.RELEASE]
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:130) ~[spring-data-commons-2.3.0.RELEASE.jar:2.3.0.RELEASE]
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://localhost:9200], URI [/history/_search?pre_filter_shard_size=128&typed_keys=true&max_concurrent_shard_requests=5&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=dfs_query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"Failed to derive xcontent"}],"type":"x_content_parse_exception","reason":"Failed to derive xcontent"},"status":400}
at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:283) ~[elasticsearch-rest-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:261) ~[elasticsearch-rest-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235) ~[elasticsearch-rest-client-7.6.2.jar:7.6.2]
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1514) ~[elasticsearch-rest-high-level-client-7.6.2.jar:7.6.2]
... 124 common frames omitted
With debuggin I tracked down the raw Rest-Request that is sent to elasticsearch in org.elasticsearch.client.RestClient.java:244 and found that this is the payload sent to the server:
{"from":0,"size":10,"query":{"wrapper":{"query":"ImJvb2wiOiB7ICAgICJmaWx0ZXIiOiBbICAgICB7ICAgICAgICJ0ZXJtIjogeyAgICAgICAidXNlcklkLmtleXdvcmQiOiAiMzFjMjA5NTktNjg5Zi00YjI4LWExNzctNmQ3ZTQ2YTBhYzMwIiAgICAgIH0gICAgIH0sICAgeyJtYXRjaCI6IHsgICAgImNvbnRlbnQiOiAidGVzdHNzIiAgIH19ICAgIF0gICB9"}},"version":true,"sort":[{"id":{"order":"desc"}}]}
With that payload an error is not suprising however I have no idea why there is this weird mumble of characters. I suspect that that is supposed to be my custom query which is not used correctly. I got this payload by debugging into this line:
httpResponse = client.execute(context.requestProducer, context.asyncResponseConsumer, context.context, null).get();
and then executing:
StandardCharsets.UTF_8.decode(((NByteArrayEntity) ((HttpPost) ((HttpAsyncMethods.RequestProducerImpl) context.requestProducer).request).entity).buf).toString()
These are my imports and the classname that I use in the Repository-Class:
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.elasticsearch.annotations.Query;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import java.util.stream.Stream;
public interface SearchablePageHistoryRepository extends ElasticsearchRepository<SearchablePageHistory, Integer> {
Page<SearchablePageHistory> findAllByUserId(String userId, Pageable pageable);
@Query("\"bool\": { " +
" \"filter\": [ " +
" { " +
" \"term\": { " +
" \"userId.keyword\": \"?0\" " +
" } " +
" }, " +
" {" +
" \"match\": { " +
" \"content\": \"?1\" " +
" }" +
" } " +
" ] " +
" }")
Page<SearchablePageHistory> findAllByUserIdAndContentLike(String userId, String content, Pageable pageable);
}
All other queries where I don't use @Query work fine without a problem. I have no idea what I am doing wrong since my example seems very similar to the one given in the documentation: https://docs.spring.io/spring-data/elasticsearch/docs/current/reference/html/#elasticsearch.query-methods.at-query
Hard facepalm, I found my error -> still gonna leave this post up in case someone else stumbles across the same problem since the error message is not very helpful in my opinion.
I simply forgot the surrounding brackets around the outside of the query:
changing this:
@Query("\"bool\": { " +
" \"filter\": [ " +
" { " +
" \"term\": { " +
" \"userId.keyword\": \"?0\" " +
" } " +
" }, " +
" {" +
" \"match\": { " +
" \"content\": \"?1\" " +
" }" +
" } " +
" ] " +
" }")
Page<SearchablePageHistory> findAllByUserIdAndContentLike(String userId, String content, Pageable pageable);
to this:
@Query("{\"bool\": { " +
" \"filter\": [ " +
" { " +
" \"term\": { " +
" \"userId.keyword\": \"?0\" " +
" } " +
" }, " +
" {" +
" \"match\": { " +
" \"content\": \"?1\" " +
" }" +
" } " +
" ] " +
" }}")
Page<SearchablePageHistory> findAllByUserIdAndContentLike(String userId, String content, Pageable pageable);
solved the problem.
Addition:
"ImJvb2wiOiB7ICAgICJmaWx0ZXIiOiBbICAgICB7ICAgICAgICJ0ZXJtIjogeyAgICAgICAidXNlcklkLmtleXdvcmQiOiAiMzFjMjA5NTktNjg5Zi00YjI4LWExNzctNmQ3ZTQ2YTBhYzMwIiAgICAgIH0gICAgIH0sICAgeyJtYXRjaCI6IHsgICAgImNvbnRlbnQiOiAidGVzdHNzIiAgIH19ICAgIF0gICB9"
is a wrapper query, that's a base64 encoded string conataining
""bool": { "filter": [ { "term": { "userId.keyword": "31c20959-689f-4b28-a177-6d7e46a0ac30" } }, {"match": { "content": "testss" }} ] }"