Search code examples
aws-lambdaamazon-athenacube.js

Cube.js timing out in serverless environment

I've been following the guide on https://cube.dev/docs/deployment#express-with-basic-passport-authentication to deploy Cube.js to Lambda. I got it working against an Athena db such that the /meta endpoint works successfully and returns schemas.

When trying to query Athena data in Lambda however, all requests are resulting in 504 Gateway Timeouts. Checking the CloudWatch logs I see one consistent error:

/bin/sh: hostname: command not found

Any idea what this could be?

Here's my server.yml:

service: tw-cubejs

provider:
  name: aws
  runtime: nodejs12.x
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "sns:*"
        # Athena permissions
        - "athena:*"
        - "s3:*"
        - "glue:*"
      Resource:
        - "*"
  # When you uncomment vpc please make sure lambda has access to internet: https://medium.com/@philippholly/aws-lambda-enable-outgoing-internet-access-within-vpc-8dd250e11e12
  vpc:
    securityGroupIds:
      # Your DB and Redis security groups here
      - ########
    subnetIds:
      # Put here subnet with access to your DB, Redis and internet. For internet access 0.0.0.0/0 should be routed through NAT only for this subnet!
      - ########
      - ########
      - ########
      - ########
  environment:
    CUBEJS_AWS_KEY: ########
    CUBEJS_AWS_SECRET: ########
    CUBEJS_AWS_REGION: ########
    CUBEJS_DB_TYPE: athena
    CUBEJS_AWS_S3_OUTPUT_LOCATION: ########
    CUBEJS_JDBC_DRIVER: athena
    REDIS_URL: ########
    CUBEJS_API_SECRET: ########
    CUBEJS_APP: "${self:service.name}-${self:provider.stage}"
    NODE_ENV: production
    AWS_ACCOUNT_ID:
      Fn::Join:
        - ""
        - - Ref: "AWS::AccountId"

functions:
  cubejs:
    handler: cube.api
    timeout: 30
    events:
      - http:
          path: /
          method: GET
      - http:
          path: /{proxy+}
          method: ANY
  cubejsProcess:
    handler: cube.process
    timeout: 630
    events:
      - sns: "${self:service.name}-${self:provider.stage}-process"

plugins:
  - serverless-express
Solution

  • Even this hostname error message is in logs however it isn't an issue cause. Most probably you experiencing issue described here.

    @cubejs-backend/serverless uses internet connection to access messaging API as well as Redis inside VPC for managing queue and cache. One of those doesn't work in your environment.

    Such timeouts usually mean that there's a problem with internet connection or with Redis connection. If it's Redis you'll usually see timeouts after 5 minutes or so in both cubejs and cubejsProcess functions. If it's internet connection you will never see any logs of query processing in cubejsProcess function.