typescript amazon-web-services amazon-s3 localstack pulumi

Deploying Localstack with Pulumi: Missing credentials in config

I am trying to deploy localstack with Pulumi. In particular I am trying to follow this example.

The code is the following.

import * as aws from "@pulumi/aws";
import * as pulumi from "@pulumi/pulumi";


// Create a bucket each for TPS reports and their archived zips.
const tpsReports = new aws.s3.Bucket("tpsReports");
const tpsZips = new aws.s3.Bucket("tpsZips");

// Anytime a new TPS Report is uploaded, archive it in a zipfile.
tpsReports.onObjectCreated("zipTpsReports", async (e) => {
    const admZip = require("adm-zip");
    const s3 = new aws.sdk.S3();
    for (const rec of e.Records || []) {
        const zip = new admZip();
        const [ buck, key ] = [ rec.s3.bucket.name, rec.s3.object.key ];
        console.log(`Zipping ${buck}/${key} into ${tpsZips.bucket.get()}/${key}.zip`);
        const data = await s3.getObject({ Bucket: buck, Key: key }).promise();
        zip.addFile(key, data.Body);
        await s3.putObject({
            Bucket: tpsZips.bucket.get(),
            Key: `${key}.zip`,
            Body: zip.toBuffer(),
        }).promise();
    }
});

// Finally, export the zipfile bucket name, for ease of access.
export const tpsReportsBucket = tpsReports.bucket;
export const tpsZipsBucket = tpsZips.bucket;

The Pulumi.local.yaml file is the following.

encryptionsalt: v1:+vnO5hzuWuw=:v1:TmVzyjbKEBaC2kfZ:PdUUCqpjq+3EUrycO+vTCtcmTKVgtg==
config:
  aws:accessKey: test
  aws:endpoints:
  - cloudwatch: http://localhost:4566
    cloudwatchlogs: http://localhost:4566
    dynamodb: http://localhost:4566
    iam: http://localhost:4566
    lambda: http://localhost:4566
    s3: http://localhost:4566
    secretsmanager: http://localhost:4566
    sns: http://localhost:4566
    sqs: http://localhost:4566
  aws:region: eu-south-1
  aws:s3ForcePathStyle: "true"
  aws:secretKey: test
  aws:skipCredentialsValidation: "true"
  aws:skipRequestingAccountId: "true"

Then, after pulumi up, I upload a simple txt file in tpsReportsBucket:

awslocal s3 cp ./tps004.txt s3://$(pulumi stack output tpsReportsBucket)

and by looking at localstack terminal I see the following:

2021-03-16T15:53:54:INFO:localstack.services.awslambda.lambda_executors: Running lambda cmd: CONTAINER_ID="$(docker create -i   -e AWS_REGION="$AWS_REGION" -e DOCKER_LAMBDA_USE_STDIN="$DOCKER_LAMBDA_USE_STDIN" -e LOCALSTACK_HOSTNAME="$LOCALSTACK_HOSTNAME" -e EDGE_PORT="$EDGE_PORT" -e _HANDLER="$_HANDLER" -e AWS_LAMBDA_FUNCTION_TIMEOUT="$AWS_LAMBDA_FUNCTION_TIMEOUT" -e AWS_LAMBDA_FUNCTION_NAME="$AWS_LAMBDA_FUNCTION_NAME" -e AWS_LAMBDA_FUNCTION_VERSION="$AWS_LAMBDA_FUNCTION_VERSION" -e AWS_LAMBDA_FUNCTION_INVOKED_ARN="$AWS_LAMBDA_FUNCTION_INVOKED_ARN" -e AWS_LAMBDA_COGNITO_IDENTITY="$AWS_LAMBDA_COGNITO_IDENTITY" -e NODE_TLS_REJECT_UNAUTHORIZED="$NODE_TLS_REJECT_UNAUTHORIZED"   --rm "lambci/lambda:nodejs12.x" "__index.handler")";docker cp "/tmp/localstack/zipfile.50a4d689/." "$CONTAINER_ID:/var/task"; docker start -ai "$CONTAINER_ID";
2021-03-16T15:54:01:WARNING:bootstrap.py: Thread run method <function LambdaExecutor.execute.<locals>.do_execute at 0x7f8ea0bec040>(None) failed: Lambda process returned error status code: 1. Result: {"errorType":"CredentialsError","errorMessage":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1"}. Output:
START RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254 Version: $LATEST
2021-03-16T15:53:55.872Z        801df08b-8f2c-1e6b-796b-7afd00772254    INFO    Zipping tpsreports-6f6b3d2/tps004.txt into tpszips-d30a9e7/tps004.txt.zip
2021-03-16T15:54:01.131Z        801df08b-8f2c-1e6b-796b-7afd00772254    ERROR   Invoke Error    {"errorType":"CredentialsError","errorMessage":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1","code":"CredentialsError","message":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1","errno":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.126Z","originalError":{"message":"Could not load credentials from any providers","errno":"ECONNREFUSED","code":"CredentialsError","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.125Z","originalError":{"message":"EC2 Metadata roleName request returned error","errno":"ECONNREFUSED","code":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.125Z","originalError":{"errno":"ECONNREFUSED","code":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"message":"connect ECONNREFUSED 169.254.169.254:80"}}},"stack":["Error: connect ECONNREFUSED 169.254.169.254:80","    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16)"]}
END RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254
REPORT RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254  Init Duration: 221.93 ms        Duration: 5673.56 ms    Billed Duration: 5674 ms     Memory Size: 1536 MB    Max Memory Used: 56 MB   Traceback (most recent call last):
  File "/opt/code/localstack/localstack/utils/bootstrap.py", line 653, in run
    result = self.func(self.params)
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 168, in do_execute
    return _run(func_arn=func_arn)
  File "/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py", line 149, in wrapped
    raise e
  File "/opt/code/localstack/localstack/utils/cloudwatch/cloudwatch_util.py", line 145, in wrapped
    result = func(*args, **kwargs)
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 159, in _run
    raise e
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 147, in _run
    result = self._execute(func_arn, func_details, event, context, version)
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 325, in _execute
    result = self.run_lambda_executor(cmd, stdin, env_vars=environment, func_details=func_details)
  File "/opt/code/localstack/localstack/services/awslambda/lambda_executors.py", line 231, in run_lambda_executor
    raise InvocationException('Lambda process returned error status code: %s. Result: %s. Output:\n%s' %
localstack.services.awslambda.lambda_executors.InvocationException: Lambda process returned error status code: 1. Result: {"errorType":"CredentialsError","errorMessage":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1"}. Output:
START RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254 Version: $LATEST
2021-03-16T15:53:55.872Z        801df08b-8f2c-1e6b-796b-7afd00772254    INFO    Zipping tpsreports-6f6b3d2/tps004.txt into tpszips-d30a9e7/tps004.txt.zip
2021-03-16T15:54:01.131Z        801df08b-8f2c-1e6b-796b-7afd00772254    ERROR   Invoke Error    {"errorType":"CredentialsError","errorMessage":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1","code":"CredentialsError","message":"Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1","errno":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.126Z","originalError":{"message":"Could not load credentials from any providers","errno":"ECONNREFUSED","code":"CredentialsError","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.125Z","originalError":{"message":"EC2 Metadata roleName request returned error","errno":"ECONNREFUSED","code":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"time":"2021-03-16T15:54:01.125Z","originalError":{"errno":"ECONNREFUSED","code":"ECONNREFUSED","syscall":"connect","address":"169.254.169.254","port":80,"message":"connect ECONNREFUSED 169.254.169.254:80"}}},"stack":["Error: connect ECONNREFUSED 169.254.169.254:80","    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1144:16)"]}
END RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254
REPORT RequestId: 801df08b-8f2c-1e6b-796b-7afd00772254  Init Duration: 221.93 ms        Duration: 5673.56 ms    Billed Duration: 5674 ms     Memory Size: 1536 MB    Max Memory Used: 56 MB

To sum up, the error message is "Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1". I tried every suggestion I found here in other questions but none of these works. Maybe this happens because I'm using Pulumi.

Thanks for your help.

Solution

The solution was to add network_mode: bridge to my docker compose file, which I show here.

version: '3'
services:
localstack:
  image: localstack/localstack
  container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
  network_mode: bridge
ports:
  - "4566:4566" 
environment:
  - SERVICES=serverless
  - LAMBDA_REMOVE_CONTAINERS=false
  - DOCKER_HOST=unix:///var/run/docker.sock
  - AWS_DEAFULT_REGION=us-east-1
  - DEFAULT_REGION=us-east-1
  - AWS_ACCESS_KEY_ID=test
  - AWS_SECRET_ACCESS_KEY=test
  - LAMBDA_REMOTE_DOCKER=true
  - LAMBDA_EXECUTOR=docker
volumes:
  - "${TMPDIR}:/tmp/localstack"
  - "/var/run/docker.sock:/var/run/docker.sock"