Search code examples
javaamazon-sqstestcontainerslocalstack

Can't get attributes of AWS SQS queue using Java & TestContainers


The code below reproduces a problem I'm seeing in some of our tests. We

  • start up a test container
  • create an SQS client
  • use that client to create a queue
  • attempt to look up the queue ARN

The queue creation appears to be successful, and returns this URL:

http://sqs.us-east-1.localhost:4566/000000000000/my-queue

But, attempting to query the queue attributes results in:

java.net.UnknownHostException: sqs.us-east-1.localhost

public class Main {

    private static final LocalStackContainer localStack =
            new LocalStackContainer(DockerImageName.parse(
                    "localstack/localstack:3.1")).withServices(SQS);

    public static void main(String[] args) {

        localStack.start();

        // create SQS client
        AmazonSQSAsync sqs = AmazonSQSAsyncClientBuilder
                .standard()
                .withEndpointConfiguration(new EndpointConfiguration(
                        localStack.getEndpointOverride(SQS).toString(),
                        localStack.getRegion()))
                .withCredentials(new AWSStaticCredentialsProvider(
                        new BasicAWSCredentials(
                                localStack.getAccessKey(),
                                localStack.getSecretKey())))
                .build();

        // create a new queue
        String url = sqs.createQueue("my-queue").getQueueUrl();

        // get ARN of new queue 
        String arn = sqs.getQueueAttributes(url, List.of("QueueArn"))
                .getAttributes().get("QueueArn");
    }
}

The code above is part of maven project with these (and only these) dependencies:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-sqs</artifactId>
    <version>1.12.646</version>
</dependency>
<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>localstack</artifactId>
    <version>1.19.4</version>
</dependency>

I can't figure out why createQueue() works, but getQueueAttributes() doesn't. What am I doing wrong here?

[In the 'real' code I can't change the aws-java-sdk-sqs version, but I'm happy to change the TestContainers and LocalStack versions if that would help]

Update: fixed by Thrau's suggestion (see below). Need to use the latest LocalStack and set SQS_ENDPOINT_STRATEGY to dynamic:

LocalStackContainer localStack =
  new LocalStackContainer(DockerImageName.parse("localstack/localstack:latest"))
    .withEnv("SQS_ENDPOINT_STRATEGY", "dynamic")
    .withServices(SQS);

Solution

  • A simple fix would be to use the path strategy for generating SQS Queue URLs, by adding

        private static final LocalStackContainer localStack =
                new LocalStackContainer(DockerImageName.parse("localstack/localstack:3.1"))
            .withServices(SQS)
            .withEnv("SQS_ENDPOINT_STRATEGY", "path");
    

    Explanation: In LocalStack, there are different ways how resource URLs like SQS Queue URLs are generated. For SQS, they are documented here.

    It seems your setup uses the standard strategy, where the Queue URL pattern is sqs.<region>.localhost.localstack.cloud:4566/<account_id>/<queue_name>. Just that it seems instead of localhost.localstack.cloud, you're getting only localhost, which leads to invalid URLs. This can be because you're also somehow setting LOCALSTACK_HOST to localhost (perhaps this is an issue with Testcontainers?). With the path strategy, you will get a valid URL, even if you use localhost as the default localstack hostname.