Lambda Function cannot connect to S3 "Request send failed"

Question

I wanted to deploy my project I shared on GitHub on Lambda Function. For this, I added AWS CDK to the project. Everything works very well, program executes by API Gateway. But, the application cannot connect to S3. Here is the error details:

START RequestId: 800fb738-cd7f-4368-8757-46563c802cfd Version: $LATEST
2024/03/12 21:03:25 Processing request data for user Berkan with test_match.dem file.
2024/03/12 21:04:57 Error downloading file: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , HostID: , request send failed, Get "https://csense-demos.s3.eu-central-1.amazonaws.com/test_match.dem?x-id=GetObject": dial tcp 52.219.47.148:443: i/o timeout
2024/03/12 21:04:57 unexpected EOF
2024/03/12 21:04:57 unexpected EOF
RequestId: 800fb738-cd7f-4368-8757-46563c802cfd Error: Runtime exited with error: exit status 1
Runtime.ExitError
END RequestId: 800fb738-cd7f-4368-8757-46563c802cfd
REPORT RequestId: 800fb738-cd7f-4368-8757-46563c802cfd    Duration: 91709.49 ms    Billed Duration: 91710 ms    Memory Size: 128 MB    Max Memory Used: 48 MB    Init Duration: 176.00 ms    

At first, I thought it had something to do with the VPC where the Lambda Function is located and its public subnet (I attached a screenshot), but I think everything is as it should be. When I tried Lambda Function on my local machine with AWS Toolkit, everything worked fine; it connects to the same S3 and performs operations as needed.

You can directly see the Stack on GitHub.

And also, you can reach the executed code by Lambda Function in here.

Answer 1

When deploying a lambda inside VPC, we need to figure out routing aspects like below,

• If internet access is needed, you must deploy it in a private subnet where you have a default route to NAT-GW or some sort of firewalls/proxies if you are are not using AWS provided NAT-GW
• Since a lambda inside VPC uses ENI, make sure you have required outbound rules on SG's attached and NACL(inbound/outbound) on subnet(Timeouts usually happens due to this).
• If lambda needs to access S3, use either S3 Gateway Endpoint or S3 Interface Endpoint. If Gateway type is used, make sure your Lambda subnet has a route entry and this works even if your current subnet is public. On the other side if you use interface endpoint no routes needed but you can't use standard s3 endpoint rather you have to explicitly pass S3 endpoint like this. One more important thing make sure your VPC endpoints policy doesn't block.
• Last but not least make sure lambda execution role has necessary permissions. In your case it may not be a problem as error could have been more of AccessDenied rather Timeouts

If your lambda doesn't access any resource inside VPC then you don't have to deploy with VPC config or any resource inside VPC need to invoke Lambda then you can setup VPC Interface Endpoint for Lambda.

Hope this helps, if not add your SG-Outbound/NACL-Inbound-Outbound/Route Table entries also you can inspect VPC flow logs on lambda ENI for any traffic that is getting rejected or simply use Reachability Analyzer(GUI) where source will be lambda ENI and destination will be IP address of S3 CIDR Blocks.