AWS Cloudwatch log query times out in lambda function

Question

I'm trying to query a log group using a lambda with javascript sdk v3:

export const realTimeQueryString = (route_key: string, account: string) => `fields @timestamp, @message | 
    filter route_key = "ANY /${route_key}/{proxy+}" and user != "-"  and account = "${account}" | 
    stats count(*) by account, user, path`;

export const getFirstDayOfMonthTimestamp = (): number => {
  const date = new Date();
  const firstDayOfMonth = new Date(date.getFullYear(), date.getMonth(), 1);
  return Math.floor(firstDayOfMonth.getTime() / 1000);
};


const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));

export const startQuery = async (queryString: string) => {
  logger.info(`query initiated for ${queryString}`);
  const params: StartQueryCommandInput = {
    logGroupName: 'api_gw_v2',
    queryString: queryString,
    startTime: getFirstDayOfMonthTimestamp(),
    endTime: Math.floor(new Date().getTime() / 1000),
    limit: 10000,
  };
  logger.info(`init query: ${JSON.stringify(params)}`);
  const command = new StartQueryCommand(params);
  const response = await client.send(command);
  return response.queryId;
}


export const stopQuery = async (queryId: string) => {
  logger.info(`stopping query: ${JSON.stringify(queryId)}`);
  const command = new StopQueryCommand({ queryId: queryId });
  await client.send(command);
}

export const getQueryResults = async (queryId: string) => {
  logger.info(`Processing results for query: ${queryId}`);
  const command = new GetQueryResultsCommand({ queryId: queryId });
  const result = await client.send(command);
  
  while (result['status'] === 'Running') {
    logger.info(`current result: ${JSON.stringify(result)}`);
    logger.info(`waiting for query results to complete...`);
    await sleep(5000);
  }
  return result;
}

This code is called in the handler function like this:

const apiQueryId = await startQuery(realTimeQueryString('api',  event["account"]));
const apiQueryResults = await getQueryResults(apiQueryId);

From the log-group of this function I get the following:

Query with params(logging in startQuery function):

{
    "logGroupName": "*****",
    "queryString": "fields @timestamp, @message | \n    filter route_key = \"ANY /mgt/{proxy+}\" and user != \"-\"  and account = \"SmartLaw\" | \n    stats count(*) by account, user, path",
    "startTime": 1690848000,
    "endTime": 1691741668,
    "limit": 10000
}

Checking the current results of the query I get this(getQueryResults function):

{
    "$metadata": {
        "httpStatusCode": 200,
        "requestId": "******",
        "attempts": 1,
        "totalRetryDelay": 0
    },
    "results": [],
    "statistics": {
        "bytesScanned": 0,
        "recordsMatched": 0,
        "recordsScanned": 0
    },
    "status": "Running"
}

I guess it's fair to assume that the query is valid based on statusCode, status, etc. I can also view the history in Logs Insights and run them from there.

I've currently set the timeout of the lambda to 5 minutes, and when testing it from the console the task times out after 5 minutes.

The log is the same for every iteration of the while loop, no results and statistics. StopQueryCommand has also been attempted, but this gives the same results and I'm not sure how to determine when to stop the query.

Answer 1

Your getQueryResults only retrieves the results once and then reuses the old result over and over again. The

const command = new GetQueryResultsCommand({ queryId: queryId });
const result = await client.send(command);

parts needs to be inside the while loop.