I want to be able to get all my statistics out of the logbook so I can use it for graphical representation. As it stands my logbook only contains the generation numbers and number of evaluations. The algorithm is calculating and outputting avg, std, min, and max, but they do not get returned so I cannot use them. Is there a way I can get these values out of my algorithm?
I have tried looking at the documentation for creating records, but what is there either doesn't make sense to me, or pertain to my situation.
def main():
pop = toolbox.population(n=300)
hof = tools.HallOfFame(1)
stats_fit = tools.Statistics(lambda ind: ind.fitness.values)
stats_size = tools.Statistics(len)
mstats = tools.MultiStatistics(fitness=stats_fit, size=stats_size)
# my hope is that the values calculated by these functions show up in my logbook
mstats.register("avg", numpy.mean)
mstats.register("std", numpy.std)
mstats.register("min", numpy.min)
mstats.register("max", numpy.max)
pop, log = algorithms.eaSimple(pop, toolbox, 0.5, 0.1, 200, stats=mstats,
halloffame=hof, verbose=True)
print(str(log[0])) # only outputs generations, number of evaluations
my output from this looks like this (note I excluded the output from the algorithm that had to do with the tree's size as I didn't think it was necessary and cluttered the output, but it does output that data)
gen nevals avg gen max min
0 300 1125.92 0 45318.7 83.1079
1 173 1031.65 1 33883.4 83.1079
2 163 779.317 2 1888.68 83.1079
3 149 901.061 3 33606.2 82.4655
4 165 686.407 4 33883.4 81.8855
5 177 962.785 5 33757 81.8855
6 184 1632.86 6 33885.7 81.8855
7 171 1509.72 7 33856.9 81.8855
8 182 984.048 8 33732.6 81.6701
9 177 1534.63 9 34009.9 81.3255
10 159 1277.39 10 33885.7 80.9722
{'gen': 0, 'nevals': 300}
I expect that the final line should include everything else in the log
EDIT:
digging deeper I have found that this may be a bug. The documentation says that it is supposed to be logged when stats are included here https://deap.readthedocs.io/en/master/api/algo.html
it reads "It returns the optimized population and a Logbook with the statistics of the evolution. The logbook will contain the generation number, the number of evaluations for each generation and the statistics if a Statistics is given as argument."
I have included statistics but it doesn't seem to be working.
When you use multiple statistics, you have to specify the chapter of the logbook you want to access. In your example, the statistics have two chapters: fitness and size. Therefore, the last line in your example should be
print(log.chapters['fitness'][0])
print(log.chapters['size'][0])
which will output
{'avg': 0.5061951303359752,
'std': 0.12547520913281693,
'min': 0.15187521062437034,
'max': 0.9576681760851814,
'gen': 0,
'nevals': 300}
{'avg': 5.0, 'std': 0.0, 'min': 5, 'max': 5, 'gen': 0, 'nevals': 300}