Search code examples
pythonpandasdataframepyomo

How can I export results from a Pyomo variable to a Pandas dataframe and excel?


I have a few Pyomo variables in a model, some with three indices. For the purpose of explaining, one of my variables is

model.E_grid = Var(model.i, model.m, model.p, within = NonNegativeReals)

and the three indices are i, m and p. After running the model, I would like to see the final values of E_grid with respect to each i, m and p in an pandas dataframe (which will then allow me to export it to excel). So for example, something like [i1, m1, p1, 21.00], [i1, m1, p2, 22.00], and so on. I have already seen the "block" method which tries to export all the variables in one go (e.g. https://or.stackexchange.com/questions/2708/pyomo-looping-over-a-variable-method) but it doesn't work for me because some of the variables have just 1 index. Any help with this would be much appreciated!

Edit: this is specifically what I have tried

results_df = pd.DataFrame()
for v in model.component_objects(model.E_grid, active=True):
    for i, m, p in v:
        results_df.at[i,m,p, v.name] = value(v[i,m,p])

print(results_df) 

But I get the error ValueError: Not enough indexers for scalar access (setting)!


Solution

  • The first argument to component_objects() should be a component type (eg. Var), not a component (ie model.E_grid). You can use .items() to iterate over an IndexedComponent.

    Secondly, the error you're seeing is due to the way of indexing the dataframe and setting the value on the fly. If you're just wanting to build a dataframe from the variable values it may be much simpler to just use dict comprehension and then one of pandas' dataframe constructors:

    import pandas as pd
    from pyomo.core import ConcreteModel, Set, NonNegativeReals, Var, value
    
    model = ConcreteModel()
    model.i = Set(initialize=[1, 2, 3])
    model.m = Set(initialize=[4, 5, 6])
    model.p = Set(initialize=[7, 8, 9])
    
    model.E_grid = Var(model.i, model.m, model.p, within=NonNegativeReals, initialize=1)
    
    E_grid_data = {(i, m, p, v.name): value(v) for (i, m, p), v in model.E_grid.items()}
    df = pd.DataFrame.from_dict(E_grid_data, orient="index", columns=["variable value"])
    print(df)
    #                           variable value
    # (1, 4, 7, E_grid[1,4,7])               1
    # (1, 4, 8, E_grid[1,4,8])               1
    # (1, 4, 9, E_grid[1,4,9])               1
    # (1, 5, 7, E_grid[1,5,7])               1
    # (1, 5, 8, E_grid[1,5,8])               1
    # (1, 5, 9, E_grid[1,5,9])               1
    

    You can now post-process the dataframe if you want to split out the index into a multiindex.