We are planning use the jBPM Engine jar which is freely available to download and use it as the Runtime BPM Engine in our java code. We will send over our flat file (in BPEL or whichever format is expected) which describes our processes (Orchestrator) is it possible to to so and if yes that what kind flat file can be used to execute the above task.
It is possible to use jBPM engine in an embedded mode exactly the way you are describing. To provide business assets (processes, rules,..) which jBPM engine will execute, you first need to model them - jBPM provides tooling for this - Web Designer and also Eclipse Modeler - both can produce valid BPMN files (which holds your process definitions). You then package these resources in a kjar (jbpm deployment unit, very similar to a simple jar, but with few extra metadata) and pass the GAV(maven groupid/artifactid/version) to the jBPM engine - which loads this KJAR and thus making the resources found in it available for further execution. These are the jBPM basics which should be thoroughly documented http://docs.jboss.org/jbpm/v6.3/userguide/