Search code examples
amazon-web-servicesamazon-ec2amazon-elastic-beanstalkaws-sdkamazon-iam

NodeJS AWS SDK unable to make API calls from EC2 Instance to SQS with Instance Profile Attached


Issue

I have a NodeJS REST API running on Elastic Beanstalk. Node API accesses services from AWS such as SQS, SES etc. To make API calls to these services, I used the recommended practice from AWS. i.e. to attach an instance profile to EC2 so that we don't have to manually handle and put AWS credentials in our EC2.

It was working as expected for the last few months, for some reason for the last couple of days all of the requests to AWS Services has started throwing this error below :

code: 'IncompleteSignature'
Error: "Authorization header requires 'Credential' parameter. Authorization header requires 'Signature' parameter. Authorization header requires 'SignedHeaders' parameter. Authorization=AWS xxxxx:xxxxxx"

What I did

Since I couldn't find a solution to this one, I SSH'ed into the instance and restarted the node process. API started working fine after that. All the requests to SQS, SES,S3 etc. started working again. Without changing anything in code, elastic beanstalk or permissions for role.

Dilemma

I thought this was a one off issue, but, after the load balancer started a new instance and terminated the previous one (in which I restarted the node process) this issue appeared again, I SSH'ed into the new instance again and restarted the node process, to no surprise it started working again.

Question

Why isn't AWS SDK able to properly sign the requests to SQS or other services ? Why does it start working after I kill and restart the node process in EC2. Considering the fact that AWS states in the docs that we don't need to worry about managing credentials if instance profile is attached to EC2, aws-sdk handles it out of the box

EC2 Instance and environment info :

OS: Linux

aws-sdk: v2.964.0

NodeJS : v14.20.1

Permissions : EC2 Instance Profile has all the appropriate permissions

EC2 Role : Attached to Instance

Any help with this one is much appreciated !


Solution

  • I also had the same issue, fixed after running npm update. Issue may be due to some updates by aws security updates

    Note: Just update aws-sdk package only.