How to access local files on EMR using java jar?

Question

Is it possible to copy files from s3 to EMR local storage and access it via java jar? If yes, how do we know which EMR node it copied the files? (This is required because, java jar needs to be on the same box as files?)

Answer 1

Have you tried SCP client? You can do something like below to copy the file from s3 to EMR master node :

aws s3 cp s3://yourBucket/folder/object ./
scp -i yourKey.pem ./object [email protected]:/pathToWhereYouWantToStoreFileOnEMRLocal/

Alternatively, you can SSH into the master node of the EMR cluster and then use AWS CLI command to copy the file from S3 to local filesystem.