OKhttp h2c protocol is slower than HTTP1.1 protocol

Question

I'm investigating whether to upgrade http 2 for communication between microservices. So I tested the performance of the h2c protocol using okhhtp. But the results were strange. H2c doesn't even perform as well as http1.1. Is there something wrong with my test method or scenario?

H2c performs worse when there are more concurrent threads.

# 20 thread

http1.1
Benchmark        Mode  Cnt   Score   Error  Units
JMHTest.request  avgt   10  30.997 ± 0.185  ms/op

h2c
Benchmark        Mode  Cnt   Score   Error  Units
JMHTest.request  avgt   10  32.816 ± 1.022  ms/op

# 50 thread

http1.1
Benchmark        Mode  Cnt   Score   Error  Units
JMHTest.request  avgt   10  31.286 ± 0.555  ms/op

h2c
Benchmark        Mode  Cnt   Score   Error  Units
JMHTest.request  avgt   10  72.564 ± 0.678  ms/op

I use JMH for testing and my server is a spring-web server with the HTTP2 protocol enabled. I expect higher performance when using okhttp with the h2c protocol.

Here is my test code.

Env

• Jdk: openjdk version "17.0.6" 2023-01-17 LTS
• okhttp3: 5.0.0-alpha.12

Client

@State(Scope.Benchmark)
public class JMHTest {
    private static final OkHttpClient CLIENT = HttpClientFactory.buildH2PriorClient();
    private static final String URL = "http://localhost:8080/hello";
    private static final Request REQUEST = new Request.Builder()
            .url(URL)
            .post(RequestBody.create(MediaType.get("text/plain"), BodyConstant.BODY_100B))
            .build();

    public static void main(String[] args) throws RunnerException {
        Options opt = new OptionsBuilder()
                .include(JMHTest.class.getSimpleName())
                .build();

        new Runner(opt).run();
    }

    @BenchmarkMode({Mode.AverageTime})
    @OutputTimeUnit(TimeUnit.MILLISECONDS)
    @Fork(value = 2, warmups = 1)
    @Warmup(iterations = 2)
    @Measurement(iterations = 5)
    @Threads(20)
    @Benchmark
    public void request() {
        post();
    }

    @SneakyThrows
    private void post() {
        Call call = CLIENT.newCall(REQUEST);
        try (Response response = call.execute()) {
            try (ResponseBody body = response.body()) {
                if (!response.isSuccessful()) {
                    throw new RuntimeException();
                }
            }
        }
    }
}

public class HttpClientFactory {

    public static OkHttpClient buildClient() {
        return getClientBuilder().build();
    }

    public static OkHttpClient buildH2PriorClient() {
        OkHttpClient.Builder builder = getClientBuilder();
        builder.setProtocols$okhttp(List.of(Protocol.H2_PRIOR_KNOWLEDGE));
        return builder.build();
    }

    @SneakyThrows
    private static OkHttpClient.Builder getClientBuilder() {
        OkHttpClient.Builder builder = new OkHttpClient.Builder();
        builder.setReadTimeout$okhttp(3000);
        builder.setConnectTimeout$okhttp(3000);
        builder.setReadTimeout$okhttp(3000);
        builder.setWriteTimeout$okhttp(3000);
        return builder;
    }
}

Answer 1

HTTP/2 trades a small amount of additional CPU work (generally not the bottleneck for HTTP calls) to reduce the number of TCP and TLS connections created, and to also reduce the number of packets transmitted. For example, it employs HPACK to compress headers.

I suspect your benchmark would show HTTP/2 to be a better choice if your client and server had additional latency on their connections, and if you limited how many TCP connections the client used concurrently.

🎄