c#.netamazon-web-servicesaws-lambdaamazon-sqs
.net C# API GW triggered AWS Lambda returns "Connection refused" when sending to AWS SQS
My trivial .net C# AWS Lambda function, which is triggered by a HTTP GET to API GW runs but returns "Connection refused" when sending to AWS SQS
responseSendMsg = await sqsClient.SendMessageAsync(sendMessageRequest);
It seems the call to SQS is the issue.
Here is the full code, it is 90% generated by Visual Studio 2019, AWS Toolkit for Visual Studio, new AWS project template "AWS Serverless Application (.NET Core - C#)", I have only:
- Updated the lambda function handler method signature to be async with a Task, I followed the pattern documented at https://docs.aws.amazon.com/lambda/latest/dg/csharp-handler.html#csharp-handler-async
public async Task<APIGatewayProxyResponse> Get(APIGatewayProxyRequest request, ILambdaContext context)
- Added in the code to write to SQS
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.Lambda.APIGatewayEvents;
using Amazon.SQS;
using TEST.SQS;
using Amazon.SQS.Model;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace TestAsyncFromLambda
{
public class Functions
{
public Functions()
{
}
public async Task<APIGatewayProxyResponse> Get(APIGatewayProxyRequest request, ILambdaContext context)
{
Console.WriteLine("Get Request\n");
// Call a local async method to prove that works, leaving SQS out of the situation for the moment
await Method1();
// Create the Amazon SQS client
var clientConfig = new AmazonSQSConfig
{
ServiceURL = SQSConstants.AWS_SERVICE_URL,
};
var sqsClient = new AmazonSQSClient(clientConfig);
// Create and initialize a SendMessageRequest instance
SendMessageRequest sendMessageRequest = new SendMessageRequest();
sendMessageRequest.QueueUrl = SQSConstants.MyQueueUrl;
sendMessageRequest.MessageBody = "this is a test message";
Console.WriteLine($"About to send message to queue: {sendMessageRequest.QueueUrl}");
// Send the SQS message using "await"
SendMessageResponse responseSendMsg = null;
try
{
responseSendMsg =
await sqsClient.SendMessageAsync(sendMessageRequest);
//responseSendMsg =
//sqsClient.SendMessageAsync(sendMessageRequest).GetAwaiter().GetResult(); ;
}
catch (Exception e)
{
Console.WriteLine(e);
throw e;
}
Console.WriteLine("SendMessage() complete");
var response = new APIGatewayProxyResponse
{
StatusCode = (int)HttpStatusCode.OK,
Body = "Hello AWS Serverless",
Headers = new Dictionary<string, string> { { "Content-Type", "text/plain" } }
};
return response;
}
public static async Task Method1()
{
await Task.Run(() =>
{
int iterations = 3;
for (int i = 1; i <= iterations; i++)
{
Console.WriteLine(" Method 1, i=" + i + " of " + iterations);
// Do something
Task.Delay(100).Wait();
}
});
}
}
}
Obviously this is a simplified example, I plan to do a lot of SQS processing in the function and expect that the use of async could be beneficial.
The cloud watch logs show the function has been triggered, gets as far as the send to SQS call, then throws an exception:
System.Net.Http.HttpRequestException: Connection refused
2021-06-24T16:13:54.751+10:00 START RequestId: MyID-91a3-f1e61c3c3f7a Version: $LATEST
2021-06-24T16:13:55.240+10:00 Get Request
2021-06-24T16:13:55.261+10:00 Method 1, i=1 of 3
2021-06-24T16:13:55.380+10:00 Method 1, i=2 of 3
2021-06-24T16:13:55.481+10:00 Method 1, i=3 of 3
2021-06-24T16:13:56.160+10:00 About to send message to queue: https://sqs.ap-southeast-2.amazonaws.com/MyID/MyQueue.fifo
2021-06-24T16:14:32.503+10:00 System.Net.Http.HttpRequestException: Connection refused
2021-06-24T16:14:32.503+10:00 ---> System.Net.Sockets.SocketException (111): Connection refused
2021-06-24T16:14:32.503+10:00 at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 --- End of inner exception stack trace ---
2021-06-24T16:14:32.503+10:00 at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.SQS.Internal.ValidationResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)
2021-06-24T16:14:32.503+10:00 at TestAsyncFromLambda.Functions.Get(APIGatewayProxyRequest request, ILambdaContext context) in
I believe everything I am using is the latest version:
I also tried a few other permutations of C# async await syntax with the same result.
responseSendMsg = sqsClient.SendMessageAsync(sendMessageRequest).GetAwaiter().GetResult(); ;
There is no VPC involved for the API GW, Lambda or SQS, everything is in the same AWS account, just a basic simple setup.
I also did a test where I changed the queue URL in SQS that the Lambda is sending to into an invalid queue name that does not exist, and I got identical behaviour, so this implies that possibly the SQS send request never even gets to AWS.
For the AWS .net SDK for SQS, there is no choice between sync and async methods for working with SQS, there is ONLY async: https://docs.aws.amazon.com/sdk-for-net/latest/developer-guide/SendMessage.html.
The Lambda function is attached to a role with these permissions, which I think should be sufficient:
I have even added the "AdministratorAccess" role, so I assume as long as the Lambda and SQS are on the same AWS account, the Lambda will have access to send to SQS.
The file aws-lambda-tools-defaults.json has:
{
"Information" : [
"This file provides default values for the deployment wizard inside Visual Studio and the AWS Lambda commands added to the .NET Core CLI.",
"To learn more about the Lambda commands with the .NET Core CLI execute the following command at the command line in the project root directory.",
"dotnet lambda help",
"All the command line options for the Lambda command can be specified in this file."
],
"profile" : "myProfile",
"region" : "ap-southeast-2",
"configuration" : "Release",
"framework" : "netcoreapp3.1",
"s3-prefix" : "TestAsyncFromLambda/",
"template" : "serverless.template",
"template-parameters" : "",
"s3-bucket" : "awsserverless2stack-bucket-myID",
"stack-name" : "TestAsyncFromLambda"
}
The file serverless.template is:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "An AWS Serverless Application by matt.",
"Resources": {
"Get": {
"Type": "AWS::Serverless::Function",
"Properties": {
"Handler": "TestAsyncFromLambda::TestAsyncFromLambda.Functions::Get",
"Runtime": "dotnetcore3.1",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 187,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AmazonSQSFullAccess",
"AdministratorAccess"
],
"Events": {
"RootGet": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
}
}
}
},
"Outputs": {
"ApiURL": {
"Description": "API endpoint URL for Prod environment",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}
I have tried both with a manually created queue via AWS Console Web UI and via Cloud Formation script, example taken from here:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Resources": {
"MyQueue": {
"Properties": {
"QueueName": "MyQueue.fifo",
"FifoQueue": true,
"ContentBasedDeduplication": true
},
"Type": "AWS::SQS::Queue"
}
},
"Outputs": {
"QueueName": {
"Description": "The name of the queue",
"Value": {
"Fn::GetAtt": [
"MyQueue",
"QueueName"
]
}
},
"QueueURL": {
"Description": "The URL of the queue",
"Value": {
"Ref": "MyQueue"
}
},
"QueueARN": {
"Description": "The ARN of the queue",
"Value": {
"Fn::GetAtt": [
"MyQueue",
"Arn"
]
}
}
}
}
I increased the lambda timeout to a long custom value to:
- differentiate timeouts at that duration from other timeouts
- give more time for SQS send to retry and/or timeout
I increased the timeout by changing serverless.template with:
"Timeout": 187,
Before increasing the timeout, when it was the default 30 seconds, I got not exception in the cloud watch logs, just as Lambda timeout.
Solution
As I mentioned in the comment, looks like the URL values being used for AWS Service URL and Queue URL are not correct. That's why the connection was failing while sending the messages.
So you need to make sure that SQSConstants.AWS_SERVICE_URL and SQSConstants.MyQueueUrl has correct values.
- Why does 1.0/100.0 == 0.1/10.0 give True?
- How to organize the receive msg and user current input in C network such that it's clean
- Using inclusive scan syntax in OpenMP in the C language
- Unable to understand context in book "OOP in C" by Axel Schreiner
- How to dereference member from struct in which the definition is hid from user?
- variable-length array in struct with TI compiler in C (socket programming)
- A faster way to test 32bpp DDBs for a valid Alpha channel
- How can I compile the zephyr example-application as a freestanding application?
- Why does my forked process sometimes overwrite data in a file?
- Why in C can I initialize more values than the size of an array?
- _Generic in C needs typecasting?
- Declaring/defining an unused variable changes the output from an unrelated variable
- How to use gdb to explore the stack/heap?
- How can I read an input string of unknown length?
- Confused by difference between expression inside if and expression outside if
- Fast Arc Cos algorithm?
- Discrepancy of `unsigned long` size between llvm and gcc in riscv32
- GCC (C) - error: 'x' redeclared as different kind of symbol
- Why does GCC’s static analyser falsely warn that a pointer to an allocated memory block itself stored in an allocated memory block may leak?
- c language gcc compiler *.i file #3 "" 2 what is this?
- How to make clang compile to llvm IR
- Hashtable implementation in C
- EFI variables or config file on the EFI partition for EFI application?
- Pack high bit of every byte in ARM, for 64 bytes like AVX512 vpmovb2m?
- Calling an external C function (in a shared lib) from Perl with Inline::C does not work
- Can a C program detect if its own source code has been modified?
- Is this declaration UB?
- Eclipse C/C++ how to find variable belongs to which struct quickly
- Is this truly best way to delete last element in C?
- flint/arb.h: No such file or directory