Invoke an SSM document from Lambda - NodeJS - Invalid Instance ID error

Question

I am trying to invoke an SSM document from AWS Lambda. Here is the lambda function for the same:

const AWS = require('aws-sdk');
const ssm = new AWS.SSM();
const SSM_DOCUMENT_NAME = process.env.SSM_DOCUMENT_NAME;
AWS.config.update({region: 'ap-south-1'});

const sendCommand = (instance_id) => {
 
    var params = {
        DocumentName: SSM_DOCUMENT_NAME,
        InstanceIds: [instance_id],
        TimeoutSeconds: 300
    };
    console.log("before call:"+params.InstanceIds);
    return ssm.sendCommand(params).promise();
 
}
exports.handler = async (event) => {
    console.log('event ', JSON.stringify(event));
    try {
        
            const instance_id = event.detail.EC2InstanceId;
            console.log("processing autoscaling event for "+instance_id);
        
            await sendCommand(instance_id);
            console.log('sent command');
        
    } catch (error) {
        throw error;
    }
}

This is throwing the following exception after logging the line "before call: ". The instance already exists for a long time and is not recently created, so no consistency issues.

Invoke Error    
{
    "errorType": "InvalidInstanceId",
    "errorMessage": null,
    "code": "InvalidInstanceId",
    "message": null,
    "time": "2020-06-19T09:43:10.348Z",
    "requestId": "150decf1-be1d-407f-9a86-a4f986c04807",
    "statusCode": 400,
    "retryable": false,
    "retryDelay": 20.23844834854607,
    "stack": [
        "InvalidInstanceId: null",
        "    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:51:27)",
        "    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",
        "    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",
        "    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)",
        "    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)",
        "    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)",
        "    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10",
        "    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)",
        "    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)",
        "    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:116:18)"
    ]
}

The function works fine while testing from lambda. The error occurs when I run the complete setup. Here is the flow of what I am trying to achieve:

Instance termination > Lifecycle hook > Cloudwatch event > Lambda > SSM run Command. Based on my understanding the only difference between testing from lambda and the real time setup is that while testing the instance is in running state and the final run it is in termination:wait state. I have also increased the wait time to 600 seconds but no luck.

Additional observation: In auto scaling groups > activity - I can a message saying waiting for lifecycle event. However as soon as i terminate the instance it vanishes from the list of manages instances in Systems Manager tab. Hence by the time lambda is triggered, the instanceid is not available to Systems Manager.

Observation/solution 2: I was testing it wrong. The life-cycle hook works only if the instance is deemed unhealthy by the scaling policy. terminating the instance manually does not work. Now that I write it down it seems obvious.

Answer 1

Based on the comments.

The issue was that the instances were not registered with SSM service.

The solution was to register them by adding AWS managed policy AmazonSSMManagedInstanceCore to the instance role.