Datapipeline task stuck in WAITING_FOR_RUNNER state

Question

I have created a simple ShellCommandActivity which echos some text. It runs on a plain ec2 (vpc) instance. I see that the host has spinned up but it never executes the tasks and the task remains in WAITING_FOR_RUNNER status. After all the retries I get this error

Resource is stalled. Associated tasks not able to make progress. 

I followed this troubleshoot-link but it didn't resolve my problem.

Here is the json description of the pipeline:

{
  "objects": [
    {
      "resourceRole": "DataPipelineDefaultResourceRole",
      "role": "DataPipelineDefaultRole",
      "name": "ec2-compute",
      "id": "ResourceId_viWO9",
      "type": "Ec2Resource"
    },
    {
      "failureAndRerunMode": "CASCADE",
      "resourceRole": "DataPipelineDefaultResourceRole",
      "role": "DataPipelineDefaultRole",
      "pipelineLogUri": "s3://xyz-logs/",
      "scheduleType": "ONDEMAND",
      "name": "Default",
      "id": "Default"
    },
    {
      "name": "EchoActivity",
      "id": "ShellCommandActivityId_kc8xz",
      "runsOn": {
        "ref": "ResourceId_viWO9"
      },
      "type": "ShellCommandActivity",
      "command": "echo HelloWorld"
    }
  ],
  "parameters": []
}

What could be the problem here?

Thanks in advance.

Answer 1

I figured this out. The routing table in the VPC subnets was not properly configured.

To be specific, in my case the routing table didn't have 0.0.0.0/0 mapped to an internet-gateway. When I added this mapping, everything started working.