Having a Python Flask API Module deployed using ElasticBeanstalk running in 2 Private subnets connected to a Route Table with NAT Gateway and having Outbound inbound access as required.
Having One Lambda Function deployed in 2 other private subnets of Same VPC as of Python Module with The same NAT Gateway attached.
RULES: Python Security Grooup OUTBOUND: 0.0.0.0 -- All Inbound: 80, 443 from Security group of Lambda
Lambda Security Group OUTBOUND: 0.0.0.0 -- All Inbound: 80, 443 from Security group of Python Module (Not needed but added for trail purpose) Inbound: SG of VPC Endpoint to access Secrets manager
QUE: Can I call the API module directly using the Autogenerated HTTP API link of Elastic Beanstalk directly using Request Module in Lambda without having Any load balancer ?
Error: TTPConnectionPool(host='MY-URL.elasticbeanstalk.com', port=80): Max retries exceeded with url: /APIModuleNameData (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7f6c2e09eba0>
I have outbound connection properly.
Tried: Giving Complete network access to inbound and outbound SG of Python Module. NAT Gateway Access check by directly login to Ec2
Your Elastic Beanstalk security group has the following inbound rule:
All Inbound: 80, 443 from Security group of Lambda
That rule will only allow the Lambda function to connect to the Elastic Beanstalk EC2 server, if the Lambda function uses the EC2 server's private IP address to keep the network connection inside the VPC.
The Elastic Beanstalk DNS name you are using is almost certainly resolving to the public IP address of the EC2 server. That means the Lambda function is having to send the network request out of the VPC, via the NAT Gateway, and the request then comes back into the VPC via the public IP that is mapped to the EC2 instance. At that point (the network connection leaving the VPC and re-entering the VPC) the association with the Lambda function's security group is lost, so the EC2 instance's security group would deny the request.
Can I call the API module directly using the Autogenerated HTTP API link of Elastic Beanstalk
No, I don't think that will be able to work. You will need to change the Lambda function to use the private IP address of the EC2 instance, instead of the DNS name of the Elastic Beanstalk environment.
Alternatively, change the EC2 instance's security group to allow the public (elastic) IP of the NAT Gateway in the inbound rules, instead of the Lambda function's security group ID. The network traffic would still be leaving the VPC and coming back in, but it would then be allowed by the security group because it sees the traffic as coming from the NAT Gateway's IP address.