rust Criterion won't create HTML report

Question

I'm following along the rust Criterion book on how to benchmark a function and plot the results as the size of the input grows. Both the book and documentation however are quite sparse in giving actual examples on how to plot the benchmark result. There is a section in the book named Benchmarking with Inputs that walks through code for the exact scenario I need with an accompanying graph. Here is that graph and accompanying code:

use std::iter;

use criterion::{BenchmarkId, criterion_group, criterion_main};
use criterion::Criterion;
use criterion::Throughput;

fn from_elem(c: &mut Criterion) {
    static KB: usize = 1024;

    let mut group = c.benchmark_group("from_elem");
    for size in [KB, 2 * KB, 4 * KB, 8 * KB, 16 * KB].iter() {
        group.throughput(Throughput::Bytes(*size as u64));
        group.bench_with_input(BenchmarkId::from_parameter(size), size, |b, &size| {
            b.iter(|| iter::repeat(0u8).take(size).collect::<Vec<_>>());
        });
    }
    group.finish();
}

criterion_group!(benches, from_elem);
criterion_main!(benches);

I couldn't find any examples on how to actually graph these benchmarks! I tried modifying the code based on what I read from the PlotConfiguration docs page to make the following changes but nothing changed:

use std::iter;

use criterion::{BenchmarkId, criterion_group, criterion_main, PlotConfiguration};
use criterion::Criterion;
use criterion::Throughput;

fn from_elem(c: &mut Criterion) {
    static KB: usize = 1024;

    let plot_config = PlotConfiguration::default(); // Add plotter

    let mut group = c.benchmark_group("from_elem");
    group.plot_config(plot_config); // Plot stuff?

    for size in [KB, 2 * KB, 4 * KB, 8 * KB, 16 * KB].iter() {
        group.throughput(Throughput::Bytes(*size as u64));
        group.bench_with_input(BenchmarkId::from_parameter(size), size, |b, &size| {
            b.iter(|| iter::repeat(0u8).take(size).collect::<Vec<_>>());
        });
    }
    group.finish();
}

criterion_group!(benches, from_elem);
criterion_main!(benches);

Lastly, I combed through the expected file structure output for a benchmark. For each benchmark the output should be placed under target/criterion/$BENCHMARK_NAME/report, but there is no report section for my benchmark. I also installed gnuplot and again no report section was created. Rust newbie here so any help on figuring out how to get the plotter working would be greatly appreciated!

TLDR: All I need to do is plot a graph of my benchmarks but can't get the simple example code to do create an HTML report.

Answer 1

The most recent release of Criterion (0.4.0) changed things so that, by default, the capability to produce HTML reports is no longer compiled into each benchmark. You can do either of these two things to get reports:

1. Activate the html_reports feature when depending on criterion:

   [dependencies]
   criterion = { version = "0.4.0", features = ["html_reports"] }
   

2. Install and use the cargo criterion tool instead of using cargo bench. (Note that this is missing the feature to save and use named baselines, which I find necessary, so I don't use this option.)