Search code examples
aws-glue

Using AWS Glue data connection without Glue Data Catalog


it is very handy to create a shared data connection and store login/password etc. but I do not want to use glue catalog or crawler. Is it possible to use data connection just to get connection info/credentials for glue spark job and then use spark methods, bypassing Glue catalog and dynamic frames and such?


Solution

  • found much better way that does not require using boto3 and hardcoding region name:

    
        glueContext.extract_jdbc_conf('my_database_connection_name')
    
    

    returns something like that:

    {'enforceSSL': 'true', 'skipCustomJDBCCertValidation': 'false', 'url': 'jdbc:sqlserver://xxxxxx', 'customJDBCCertString': '', 'user': 'xxxx', 'customJDBCCert': '', 'password': 'xxxxxxxxxx', 'vendor': 'sqlserver'}