Search code examples
amazon-web-servicesemr

Re-use Amazon Elastic MapReduce instance using aws java sdk

This link (Re-use Amazon Elastic MapReduce instance) provides a way to re-use an instance like this elastic-mapreduce --jobflow job-id \ --jar s3n://some-path/x.jar \ --step-name "New step name" \ --args ...

but how can I do the same in a Driver program to run the job i.e. by using aws java sdk to run another job . I thought there must be some method in JobFlowInstancesConfig class for setting the id but I couldn't find any thing. http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/elasticmapreduce/model/JobFlowInstancesConfig.html

Any help is appreciated.

Solution

  • Actually I wasn't going to answer because of the downvotes I got initially . I did something like this (I have not added the entire code here but the link following it may help)

    static AddJobFlowStepsRequest AJFSrequest;
List <StepConfig>steps = new LinkedList();
AJFSrequest = new AddJobFlowStepsRequest();
AJFSrequest.setJobFlowId("j-VKJMSIU34SFQ");

HadoopJarStepConfig jarConfig = new HadoopJarStepConfig(S3N_HADOOP_JAR);
jarConfig.setArgs(ARGS_AS_LIST);
StepConfig stepConfig =new StepConfig(S3N_HADOOP_JAR.substring(S3N_HADOOP_JAR.indexOf('/') + 1),jarConfig);

String lastState = "";
steps.add(stepConfig);
AJFSrequest.setSteps(steps);
emr.addJobFlowSteps(AJFSrequest);

    http://chathurah.blogspot.in/2010/03/programmatically-launch-elastic-map.html