Hibernate Search AWS OpenSearch Serverless connection problem

Question

I am using Hibernate Search v. 6.1.8.Final within my Spring Boot application. My implementation works for OpenSearch v. 2.5 which runs locally and in Kubernetes.

Besides the basis libraries I include org.hibernate.search:hibernate-search-backend-elasticsearch-aws AWS adapter.

When I install it in AWS EKS I configure my Spring properties like

spring:
  jpa:
    properties:
      hibernate:
        search:
          backend:
            type: elasticsearch
            hosts: xxxxxxxx.<region>.aoss.amazonaws.com
            version: opensearch:2.5
            version_check:
              enabled: false
            aws:
              region: <region>
              signing:
                enabled: true

I do not put my AWS credentials in this configuration since I hope I can authenticate with Service Account my pod is running with.

The service account eks.amazonaws.com/role-arn: arn:aws:iam::xxxxxxxxx:role/<role-name> defines the role name which has following policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "aoss:*"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:aoss:<region>:xxxxxxxxx:collection/<collection-id>"
        }
    ]
}

I have connection to the OpenSearch Serverless Collection (with curl). Also in the IAM Policy Simulator I can verify that my role has APIAccessAll for aoss.

Now when I start my application I get following error:

2023-06-05T08:51:22,164+0000  WARN [,] --- [main] dialect.impl.ElasticsearchDialectFactory : HSEARCH400085: Unknown Elasticsearch version running on the cluster: 'opensearch:2.5'. Hibernate Search may not work correctly. Consider updating to a newer version of Hibernate Search, if any.
2023-06-05T08:51:22,367+0000  WARN [,] --- [main] als.internal.WebIdentityCredentialsUtils : To use web identity tokens, the 'sts' service module must be on the class path.
2023-06-05T08:51:22,863+0000 ERROR [,] --- [Hibernate Search - default backend - Transport thread - 2] ngine.reporting.spi.RootFailureCollector : HSEARCH000521: Hibernate Search encountered a failure during bootstrap; continuing for now to list all problems, but the process will ultimately be aborted.
Context: Hibernate ORM mapping, type 'com.example.MyEntity'
Failure:
org.hibernate.search.util.common.SearchException: HSEARCH400034: Unable to retrieve index metadata from Elasticsearch: HSEARCH400007: Elasticsearch request failed: HSEARCH400090: Elasticsearch response indicates a failure.
Request: GET /myentity-write,myentity-read with parameters {ignore_unavailable=true, allow_no_indices=true}
Response: 403 'Forbidden' from 'https://xxxxxx.<region>.aoss.amazonaws.com' with body
{
  "status": 403,
  "request-id": "<request-id>",
  "error": {
    "reason": "Credential should be scoped to correct service: \u0027aoss\u0027, not \u0027es\u0027.",
    "type": "AccessDenied"
  }
}

Does anyone know this issue? For me it seems that the policy is correct, also without Resource restriction.

Answer 1

Hibernate Search AWS integration is not tested against Amazon OpenSearch Serverless. From the message you get back from AWS:

"Credential should be scoped to correct service: \u0027aoss\u0027, not \u0027es\u0027."

I would assume that the problem comes from the fact that hibernate-search-backend-elasticsearch-aws is signing the request for the es service rather than the aoss that you need...

I'd suggest starting a discussion here https://discourse.hibernate.org/ or open a request at https://hibernate.atlassian.net/