Search code examples
amazon-web-servicesapiaws-glueaws-appsyncaws-glue-data-catalog

AppSync with Glue integration


I wanted to check if anyone has worked or seen any scenario where AWS AppSync is integrated with AWS Glue Data Catalog?

We are trying to create graphql API with the help of AWS AppSync and out data source is the data catalog in AWS Glue. What we had planned is to use lambda as the source from the AppSync side and then use this lambda to retrieve data from Glue Data Catalog.

If you have worked on or know of this scenario please help me!

Thanks!


Solution

  • I've made that integration thougth stepfunctions and it works very well

    1. Appsync call stepfunctions with http call to https://states.${AWS::Region}.amazonaws.com/
    {
        "version": "2018-05-29",
        "method": "POST",
        "resourcePath": "/",
        "params": {
            "headers": {
                "content-type": "application/x-amz-json-1.0",
                "x-amz-target":"AWSStepFunctions.StartExecution"
             },
             "body": {
                  "stateMachineArn": "${StateMachineARN}",
                  "input": "$util.escapeJavaScript($util.toJson({"payload": $ctx.args.input}))"
              }
         }
    }
    
    1. Call glue job from from stepfunctions workflow
    "GlueJob": {
         "Type": "Task",
         "Resource": "arn:aws:states:::glue:startJobRun.sync",
         "InputPath": "$.payload",
         "ResultPath": "$.result",
         "Parameters": {
              "JobName": "${EtlJobName}",
              "Arguments": {
                   "--bucket.$":"$.bucket",
                   "--key.$":"$.key"
              }
         },
         "End": true
    } 
    

    Make sure your IAM has states:StartExecution permission