How to sort the composite aggregation in elasticsearch/opensearch?

Question

I am using opensearch-java library in my spring boot application. I want to perform the composite aggregation and need to sort the final result based on one of the aggregation field in descending order. Below is my code

private SearchRequest buildSearchRequest(SelectedFilters filters) {
  BoolQuery.Builder boolQueryBuilder = QueryBuilders.bool();
 
 Aggregation aggregation1 = new Aggregation.Builder()
          .terms(aggField -> aggField
                  .field("fieldA.keyword")
                  .order(Map.of("_key", SortOrder.Desc))            // Sorting here      
          )
          .build();
  Aggregation aggregation2 = new Aggregation.Builder()
          .terms(aggField -> aggField
                  .field("fieldB"))
          .build();

  CompositeAggregationSource compositeAggregationSource1 = new CompositeAggregationSource.Builder()
          .terms(aggregation1.terms())
          .build();

  CompositeAggregationSource compositeAggregationSource2 = new CompositeAggregationSource.Builder()
          .terms(aggregation2.terms())
          .build();

  Map<String, CompositeAggregationSource> compositeAggregation1 = new HashMap<>();
  compositeAggregation1.put("fieldA_agg", compositeAggregationSource1);

  Map<String, CompositeAggregationSource> compositeAggregation2 = new HashMap<>();
  compositeAggregation2.put("fieldB_agg", compositeAggregationSource2);

  List<Map<String, CompositeAggregationSource>> list = new ArrayList<>();
  list.add(compositeAggregation1);
  list.add(compositeAggregation2);

  Aggregation compositeAggregation = new CompositeAggregation.Builder()
          .sources(list)
          .size(1000)
          .build()
          ._toAggregation();

  return new SearchRequest.Builder()
          .index("my-index")
          .query(boolQueryBuilder.build().toQuery())
          .aggregations("my_bucket", compositeAggregation)
          .size(0)
          .build();
}

For this code, i am getting the below result:

[x_content_parse_exception] [1:125] [composite] failed to parse field [sources]

However, if i just remove the order(Map.of("_key", SortOrder.Desc)), i am getting the results but not in sorted way. What changes are required in my code to make this working??

PS: If i directly query the opensearch, its also not working there. (Their autocompletion also suggests this format but it does not work simply!)

"terms": {
  "field": "fieldA.keyword",
  "order": {
    "_key": "asc"
  }
}

but if I run this way, then it gets working

"terms": {
  "field": "fieldA.keyword",
  "order": "desc"
}

The java client classes however are only expecting the Map class (Map<String, SortOrder> value), so just can't do there in application code somehow.

Answer 1

The issue might stem from the fact that the CompositeAggregationSource class expects a TermsAggregation when terms is specified in a composite source, i.e. the same TermsAggregation structure that is used when specifying standalone terms aggregations. TermsAggregation has a different way of supplying the sort order, i.e using List<Map<String, SortOrder>>

On the server-side, the TermsValuesSourceBuilder class expects a simple SortOrder value

To sum up, it's a bug in the OpenSearch Java client that needs to be fixed.

PS: I commented your ticket