Iterating over multiple model/data pairs in AMPL

Is there a good way to iterate over model/dataset pairs in AMPL in a .run file?

Say you have two different models for the same optimization problems, and four datasets. What I would do up to now was to make a .run file for each model/dataset pair, and run each individually, or keep one script for each model and manually solve for each dataset by modifying the data (file); command. But this is obviously tedious and inviable for larger projects.

So is there any good way to do this? What I've tried is something like the following (just a skeleton for clarity):

# Some global options


for {model_ in {'1.mod', '2.mod'}} {
    reset;
    model (model_);

    for {dat in {'1.dat', '2.dat', '3.dat', '4.dat'}} {
        update data;
        data (dat);
        solve;

        # Do some post-processing
    }
}

But AMPL whines about using the for loop's dummy variable in the model and data command. I've tried declaring symbolic parameters to store the name of the model and data file, but no good either.

Of course this would only be sensible if the models were similar enough, insofar as they could be post-processed in the same way. But there should at least be a way to iterate through the data files within one model without having to go like

update data;
data 1.dat;
solve;

update data;
data 2.dat;
solve;

update data;
data 3.dat;
solve;

[...]

update data;
data 427598.dat;
solve;

right?

Solution

Your example code has a reset inside a loop. This will reset the loop parameter, which is going to cause problems. For instance, if you run this:

for {modelname in {"itertest1.mod","itertest2.mod"}}{
    display (modelname);
    for {dataname in {"itertest_a.dat","itertest_b.dat"}}{
        display (dataname);
    }
}

it will print out the file names as we might hope:

modelname = itertest1.mod
dataname = itertest_a.dat
dataname = itertest_b.dat
modelname = itertest2.mod
dataname = itertest_a.dat
dataname = itertest_b.dat

But if we add a reset statement:

for {modelname in {"itertest1.mod","itertest2.mod"}}{
    reset; ### this is the only line I changed ###
    display (modelname);
    for {dataname in {"itertest_a.dat","itertest_b.dat"}}{
        display (dataname);
    }
}

then we get an error: modelname is not defined (because we just reset it).

However, even without that issue, we'll still get a complaint about using the loop variable.

Solution 1: commands

Depending on exactly what you ran, you may have seen an error message advising use of the commands statement, like so:

reset;
for {scriptname in {"script1.run", "script2.run"}}{
    commands (scriptname);
}

This will then run whatever commands are in the listed .run files, and you should be able to nest that to call separate scripts to define the models and the data. Since you can't use a blanket reset without killing your loop parameter, you will need to use other options for updating the model files. AMPL offers the option to define multiple "problems" and switch between them, which may or may not be helpful here; I haven't explored it.

The commands statement is documented here, although TBH I find the documentation a bit difficult to follow.

Solution 2: code generation

If all else fails, you can write an iterated AMPL script that generates (and optionally runs) a second script containing all the model/data combinations you want to run, like so:

param outfile symbolic := "runme_temp.run";

for{i in 1..3}{
    for{j in 1..4}{
        printf "\nreset;" > (outfile);
        printf "\nmodel model%s;",i > (outfile);
        printf "\ndata data%s;",j > (outfile);
        printf "\nsolve;" > (outfile);
        # add post-processing here as desired
    }
}
close (outfile);
include runme_temp.run;

This will create and then call "runme_temp.run" which looks like this:

reset;
model model1;
data data1;
solve;
reset;
model model1;
data data2;
solve;

etc. etc. Since this does allow you to use a blanket reset; it might simplify the process of cleaning up previous models. Not the most dignified solution, but it works and can be adapted to a wide range of things.

This could be improved by de-nesting the loops:

param outfile symbolic := "runme_temp.run";

for{i in 1..3, j in 1..4}{
    printf "\nreset;" > (outfile);
    printf "\nmodel model%s;",i > (outfile);
    printf "\ndata data%s;",j > (outfile);
    printf "\nsolve;" > (outfile);
    # add post-processing here as desired
}
close (outfile);
include runme_temp.run;

In this particular example it doesn't make much difference, but multiply-nested loops can run slow in AMPL; using a single multi-index for can make a big difference to performance, so it might be better to get into that habit.