I am carrying out static aeroelastic analysis using an OpenMDAO-based script. I am trying to adapt my script to run it on a supercomputer.
The MDA loop includes several components, including a CFD solver usually run in terminal with mpirun -np 24 python runScript.py and a structural solver usually run in terminal with nast20212 nastran_static_alpha_c.bdf.
In order to run CFD in parallel mode within my main script, I need to run it with mpirun -np 24 python aeroelastic_mda.py.
The issue is that when I do that, the script tries to run all components in parallel mode. However I would only need to run my CFD solver to run on multiple nodes while other components run in serial mode.
I have looked up the OpenMDAO documentation and other posts but I did not find information regarding my issue.
Below is a summarized view of my OpenMDAO script which I run with
mpirun -np 24 python aeroelastic_mda.py.
#Importing necessary modules
...
if __name__ == "__main__":
#Problem parameters
Sw = 383.689555/2 #Wing reference surface
V = 295.0 #Airspeed
rho_a = 1.17 #Air density
... #Some other parameters
root = Problem()
#Add independent variables
root.model.add_subsystem('wing_area', IndepVarComp('Sw', Sw), promotes=['*'])
root.model.add_subsystem('airspeed', IndepVarComp('V', V), promotes=['*'])
root.model.add_subsystem('air_density', IndepVarComp('rho_a', rho_a), promotes=['*'])
... #Some other independent variables
mda = Group()
#Add disciplines to the group
# Displacement transfer:
# Inputs -> Displacements at structural nodes
# Outputs -> Displacements to be applied to aerodynamic nodes
# Simple matrix product
mda.add_subsystem('displacement_transfer', DisplacementTransfer(), promotes=['*'])
# Aerodynamic solver
mda.add_subsystem('aerodynamics', Dafoam(), promotes=['*'])
# Load transfer
# Inputs -> Forces at aerodynamic nodes
# Outputs -> Forces to be applied to structural nodes
# Simple matrix product
mda.add_subsystem('load_transfer', LoadTransfer(), promotes=['*'])
# Structural solver
mda.add_subsystem('structures', NastranStatic(), promotes=['*'])
root.model.add_subsystem('mda_group', mda, promotes=['*'])
mda.nonlinear_solver = NonlinearBlockGS()
mda.nonlinear_solver.options['rtol'] = 1.e-3
mda.nonlinear_solver.options['use_aitken'] = True
mda.linear_solver = ScipyKrylov()
root.linear_solver = ScipyKrylov()
root.setup()
root.run_model()
Also here is a simplified view of my aerodynamic component:
#Importing necessary modules
...
class Dafoam(ExplicitComponent):
... #Some constants
def initialize(self):
... #Declaring options
def setup(self):
... #Declaring inputs
... #Declaring outputs
def compute(self, inputs, outputs):
#Generate current deformed shape
self.create_current_geom(inputs, outputs)
#Generate the input file for Dafoam from current geometry and flow conditions
self.create_input_file(inputs, outputs)
# Run DAFoam
p = Popen(['python', 'runScript.py'])
p.wait()
# Parse the output file from the external code
...
def create_current_geom(self, inputs, outputs):
...
#Method that creates the Dafoam input file runScript.py
def create_input_file(self, inputs, outputs):
...
def get_forces(self, inputs, outputs):
...
def get_aero_coeff(self, inputs, outputs):
...
Also here is a simplified view of my structural component:
#Importing necessary modules
...
class NastranStatic(ExplicitComponent):
... #Some constants
def initialize(self):
... #Declaring options
def setup(self):
... #Declaring inputs
... #Declaring outputs
def compute(self, inputs, outputs):
# Generate the input file for Nastran
self.create_input_file(inputs, outputs)
# Run MSC Nastran
p = Popen(['nast20212', 'nastran_static_alpha_c.bdf'])
p.wait()
# Parse the output file from the external code and set output values
...
def create_input_file(self, inputs, outputs):
...
def get_output_data(self, inputs, outputs):
...
Thank you very much in advance for your help!
I have eventually solved my issue by replacing Popen by ShellProc (openmdao.utils.shell_proc) function. For some reason, Popen did not work properly when used with mpirun command.