Search code examples
amazon-web-servicesamazon-ecsamazon-vpcaws-fargateaws-secrets-manager

Fargate calls to AWS Secrets Manager timing out


I am trying to get my VPC-connected Fargate instances to call the AWS Secrets Manager API, but in doing so the call is timing out:

Connect to secretsmanager.us-east-2.amazonaws.com:443 [secretsmanager.us-east-2.amazonaws.com/172.31.65.102, secretsmanager.us-east-2.amazonaws.com/172.31.66.72, secretsmanager.us-east-2.amazonaws.com/172.31.64.251] failed: connect timed out

I am aware that as of earlier this year in Fargate 1.3.0 you can get the secrets injected in as environment variables as documented here. In fact, I have that type of integration working great!

My issue is that I am unable to fetch the exact same secret programmatically using the Secret Manager SDK. When I do, I get the above timeout. In addition to the appropriate policy on the IAM ecsTaskExecutionRole role (which is what enabled me to get the secret via env variable), I also added a VPC Endpoint (because my Fargate instances are on VPC) as documented here. My Fargate instances are regularly talking to the outside internet as well.

Any ideas on what else could cause the timeout?


Solution

  • Update: the problem was ultimately some unwise route entries. Thanks to the comments for reminding me that the error was a timeout and thus upstream from any IAM configuration issues. That helped me focus exclusively on network-related solutions.