I want to take a join on tables in two different S3 sources using Apache Drill. Is there a way to add AWS access keys and secret keys in conf/core-site.xml file? Or is there any other way around?
In the storage plugin configuration that you have for S3, the connection URL can be written like “s3n://accessKeyId:secretKeyId@bucket”. This way you can have as many S3 sources configured with your Drillbit cluster as you want.