How do I configure an AWS Lambda in a VPC to be able to log to another Cloudwatch log group?

Question

How do you configure an AWS Lambda that is in a VPC to be able to log to a Cloudwatch log group other than the one that is associated with the lambda itself?

This is required so that I can avoid the cost of the summary logs that are created with every request, eg:

2022-02-10T10:16:19.133+00:00   START RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3 Version: $LATEST
2022-02-10T10:16:24.250+00:00   END RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3
2022-02-10T10:16:24.250+00:00   REPORT RequestId: 52cbfd7e-71ec-4499-a292-7c81c94535f3 Duration: 5100.10 ms Billed Duration: 7412 ms Memory Size: 256 MB Max Memory Used: 201 MB Init Duration: 2311.15 ms

So I wish to log my own logs to a separate Cloudwatch log group and add a Deny policy to prevent logging to the default Cloudwatch group that's associated with the lambda by default.

I'm using NLog, so I've done the following so far (following Nlog instructions from here):

• Installed Nuget package AWS.Logger.Nlog
• Added nlog.config and set it to copy if newer

The contents of nlog.config:

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      throwConfigExceptions="true">
  <extensions>
    <add assembly="AWS.Logger.NLog" />
  </extensions>
  <targets>
    <target name="logconsole" xsi:type="Console" />
    <target name="aws" type="AWSTarget" logGroup="my-lambda-logs" region="us-east-1"/>
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="logconsole,aws" />
  </rules>
</nlog>

• The name of my cloudwatch log group is my-lambda-logs and the region is set to us-east-1

When I run my lambda at the moment I'm getting a timeout because it is not able to log to Cloudwatch from within a VPC.

Answer 1

1. If you haven't already created your lambda, create it within your VPC, specify your private subnet ids and attach a security group that allows outbound (egress) communication on at least port 443 TCP aka HTTPS.

2. Within VPC on the AWS console, create a VPC Endpoint for Cloudwatch Logs in the region that you wish to log to in the same subnets as your lambda VPC configuration. This will allow your lambda to use the normal cloudwatch endpoint from within your VPC without any log traffic leaving the AWS network.

3. Ensure that you set Private Dns Enabled to true (you can do this by modifying after creation if you can't find this during creation)

4. Specify a security group for your new VPC endpoint that includes an inbound (ingress) rule to allow the security group of your lambda to communicate over port 443.

5. Now that you can communicate with Cloudwatch, you just need to create your log group and give your lamda a policy that allows it to log to your cloudwatch log group.

   {
    "Version": "2012-10-17",
    "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups"
      ],
      "Resource": [
         "arn:aws:logs:us-east-1:369074612400:log-group:my-lambda-logs:*", 
         "arn:aws:logs:us-east-1:369074612400:log-group:my-lambda-logs:*:*"
      ],
      "Effect": "Allow"
    }
   ]
   

   }

For the sake of this example the ARN of the cloudwatch log group is:

arn:aws:logs:us-east-1:000000000001:log-group:my-lambda-logs

... but you can get yours from the AWS console (it will contain your account id and the correct region of the Cloudwatch group.

Voilá your lambda should now be able to use NLog (or another logging framework) to log to Cloudwatch logs.