localstack with docker and dynamodb

Question

after successfully using standalone dynamodb local container with my node.js containerized application I also needed S3 which is not provided anywhere except in localstack. I decided to fully switch to localstack with intention to emulate AWS environment.

This is my compose.yaml:

services:
  node:
    build: .
    image: htmlcsstopdfimage:latest
    ports: ["${PROD_PORT}:${PROD_PORT}"]
    depends_on: [localstack]
    environment:
      HOST_NAME: $HOST_NAME
      PROD_PORT: $PROD_PORT
      AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
      AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
      AWS_SESSION_TOKEN: $AWS_SESSION_TOKEN
      AWS_REGION: $AWS_REGION
      AWS_PORT: $AWS_PORT

  localstack:
    image: localstack/localstack:latest
    ports: [4566:4566]
      # - '4563-4599:4563-4599'
      # - '8080:8080'            # Port for LocalStack web dashboard
    environment:
      - EDGE_PORT=4566
      - BROWSING-TOPICS=1
      - SERVICES=dynamodb
      - DEBUG=1
      - DATA_DIR=/tmp/localstack/data
      - AWS_ACCESS_KEY_ID=dummy
      - AWS_SECRET_ACCESS_KEY=dummy 
    volumes:
      # - './.localstack:/tmp/localstack'
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
      - '/var/run/docker.sock:/var/run/docker.sock'

I have read many articles and posts + official documentation but I do not understand the usage and networking of localstack.

For example if I go to: localhost:4566 I get blank page inside the browser with response of 200 with (headers: and content-length: 0)

http://localhost:4566/_localstack/health returns json response with all services as available

Before this last (docker compose) draft I tried accessing many different ports in hopes of some meaningful response, but I never got any response, even the dashboard was nowhere to be found when I exposed 8080.

While writing this question I stumbled upon something that I have not seen in the documentation: https://dynamodb.localhost.localstack.cloud:4566/ which returns "exception while calling dynamodb with unknown operation" and some kind of parse error "An unknown error occurred when trying to parse the request."

But the official data mentions that all services are on port 4566 and are accessed via url parameter as domain:port/service where service could be dynamodb. When I try it: localhost:4566/dynamodb response is:

<Error>
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist</Message>
<RequestId>8c3bca60-146f-4ae1-9761-b7abe88247bd</RequestId>
<BucketName>dynamodb</BucketName>
</Error>

My questions are:

1. What is appropriate compose.yaml file?
2. Why does the newer versions use: "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" instead of './.localstack:/tmp/localstack', how is LOCALSTACK_VOLUME_DIR variable supplied (by localstack I suppose? Or if it is my job to supply it via .env file, what should the value be?
3. What is example of generic query something like localhost:4566/dynamodb/health just so I know that networking is healthy and that service is accessible or some alternative like reading all tables or something generic that always works (even before dynamo has any data or entities) how can I verify a working service?
4. How can I understand localstack reference: localstack dynamodb documentation, for example, if I go to the documentation linked above and search ListTables, on the right side it has Show Tests option, when I open it there are countless options regarding that action, I do not know if that is relevant, but how can I understand the documentation and use the endpoints that I need like that one, ListTables, where does it say what endpoint to query and what data to send and what is the expected response? (I am interested in the HTTP requests not a CLI solution, I want to be able to do the request with Node.js evenutally).
5. Please explain domains, ports and url parameters in localstack, the following from documentation is confusing me: language SDKs example, why does it use http://s3.localhost.localstack.cloud:4566 is that deprecated or up to date and what is the meaning of such networking?
6. I came across some other issues like: CORS, "no service set in context", 400, 403, 500, so I hope that provided solution will explain common beginner pitfalls and take care of such problems too.

I am forever grateful to anyone that even attempts to respond to this question.

Answer 1

Hi — These are a lot of questions, and I would try to answer them in detailed manner. With that said, I would recommend you to use the official & latest LocalStack documentation, in conjunction with the AWS documentation to make things simple.

What is appropriate docker-compose.yaml file?

The appropriate Docker Compose configuration has been documented here — https://docs.localstack.cloud/getting-started/installation/#starting-localstack-with-docker-compose

Why does the newer versions use: "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"

This is the location on the host of the LocalStack volume directory mount which is then automatically mounted into /var/lib/localstack.

The defaults are:

• Mac: ~/Library/Caches/localstack/volume
• Linux: ~/.cache/localstack/volume
• Windows: %LOCALAPPDATA%/localstack/cache/volume

You don't need to supply them manually. LocalStack takes care of that. The ${LOCALSTACK_VOLUME_DIR} could be an arbitrary location on the host, e.g., ./volume.

What is example of generic query something like localhost:4566/dynamodb/health?

We don't provide service-specific health endpoints. If you need to see if a service is available or not, you can run the following query:

curl localhost:4566/_localstack/health | jq

It will list all LocalStack services, and let you know if they are available. With all the recent changes made to LocalStack, the services are up in just a few milliseconds and you won't even need to check if an individual service is available or not.

To verify a working service, just run some sample commands like awslocal s3 ls or awslocal dynamodb list-tables.

How can I understand localstack reference?

Based on what I read and understood, I think you are using the reference page wrong. The reference page purely serves the purpose of communicating what LocalStack APIs are available, and what is the test coverage. It is necessary for us to communicate what APIs are available for a particular service, and how stable are they.

We have individual service documentation for AWS services, like for DynamoDB — https://docs.localstack.cloud/user-guide/aws/dynamodb/

Coming to your next part of the question around sending requests with Node.js, you should be looking into Language SDKs and configure them to use LocalStack. For JavaScript, we have — https://docs.localstack.cloud/user-guide/integrations/sdks/javascript/

For rest of your understanding, we highly recommend you to see and understand AWS documentation because LocalStack is a drop-in replacement for local dev & test, and does not provide a complete replica of AWS docs. We want the users to understand how to start and use LocalStack, as well as integrations, language SDKs, or toolings and continue using AWS docs to accomplish their tasks.

Please explain domains, ports and url parameters in LocalStack

This comprehensive doc can surely help you in this regard — https://docs.localstack.cloud/references/network-troubleshooting/

Why does it use http://s3.localhost.localstack.cloud:4566?

In case you have missed our S3 docs, LocalStack categorizes requests as either Path-Style or Virtual Hosted-Style based on the Host header of the request (similar to AWS) The following example illustrates this distinction:

• http://<bucket-name>.s3.<region>.localhost.localstack.cloud:4566/<key-name> — Host-style request
• http://s3.<region>.localhost.localstack.cloud:4566/<bucket-name>/<key-name> — Path-style request

As a special case in LocalStack, leaving out <region> also works for the s3.localhost.localstack.cloud domain. <bucket-name>.s3.localhost.localstack.cloud is also a host-style request.

All other requests are treated as path-style requests. Using the s3.localhost.localstack.cloud endpoint URL is recommended for all requests aimed at S3.

I came across some other issues

I would need reproducible samples to actually get to see these issues. Given the fact that LocalStack emulates many AWS services, the beginner pitfalls can be endless depending on how skilled you are with AWS.

I hope this answers your query! :)