org.eclipse.jetty.http.BadMessageException: 400: Invalid SNI

Question

I am trying to solve an issue with my Jetty servlet running over HTTPS.

It is showing this error page in the browser:

What I did:

1. I created my Keystore and Truststore as is described here: How to generate keystore and truststore

2. Both files were placed in the directory of my project and code was written up to load these files.

package sk.cood.metahost.server;

import jakarta.servlet.ServletException;
import jakarta.servlet.annotation.WebServlet;
import jakarta.servlet.http.HttpServlet;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.eclipse.jetty.server.*;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.eclipse.jetty.util.ssl.SslContextFactory;

import java.io.*;

@WebServlet(displayName = "MetaHostServlet", urlPatterns = { "/*" })
public class MetaHostServlet extends HttpServlet {
    private static File keyStoreFile;
    private static File trustStoreFile;

    public static void main(String[] args) throws Exception {
        loadKeyStores();

        Server server = new Server(8080);
        ServerConnector connector = createSSLConnector(server, "password", "password", false);
        server.addConnector(connector);

        ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
        context.addServlet(new ServletHolder(new MetaHostServlet()),"/*");
        context.setContextPath("/");
        server.setHandler(context);

        server.start();
        server.join();
    }

    private static void loadKeyStores() {
        keyStoreFile = new File("keystore.jks");
        trustStoreFile = new File("truststore.jks");
        if (!keyStoreFile.exists()) {
            throw new RuntimeException("Key store file does not exist on path '"+keyStoreFile.getAbsolutePath()+"'");
        }
        if (!trustStoreFile.exists()) {
            throw new RuntimeException("Trust store file does not exist on path '"+trustStoreFile.getAbsolutePath()+"'");
        }
    }

    private static ServerConnector createSSLConnector(Server server, String keyStorePassword, String trustStorePassword, boolean isClientAuthNeeded) {
        SslContextFactory.Server sslContextFactory = new SslContextFactory.Server();
        sslContextFactory.setKeyStorePath(keyStoreFile.getAbsolutePath());
        sslContextFactory.setKeyStorePassword(keyStorePassword);
        sslContextFactory.setTrustStorePath(trustStoreFile.getAbsolutePath());
        sslContextFactory.setTrustStorePassword(trustStorePassword);
        sslContextFactory.setNeedClientAuth(isClientAuthNeeded);

        HttpConfiguration https_config = new HttpConfiguration();
        https_config.setSendServerVersion(false);
        https_config.setRequestHeaderSize(512 * 1024);
        https_config.setResponseHeaderSize(512 * 1024);

        SecureRequestCustomizer src = new SecureRequestCustomizer();
        https_config.addCustomizer(src);

        return new ServerConnector(server, sslContextFactory, new HttpConnectionFactory(https_config));
    }

    @Override
    public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
        res.setContentType("text/html");
        res.setStatus(HttpServletResponse.SC_OK);
        res.getWriter().println("<h1>Hello World!</h1>");
        res.getWriter().println("session=" + req.getSession(true).getId());
    }
}

3. I started my servlet with jetty and tried to connect to localhost:8080 and mentioned error appears.

Then I tried to create other Keystores and Truststores by other guides, for example, I added DNS alternative names to my new Keystore as is described here: https://serverfault.com/questions/488003/keytool-subjectalternativename but nothing helped me.

I don't know what is wrong in my case, maybe someone more experienced with jetty and certificates will help.

Thank you so much!

Answer 1

This message means that the SNI provided by your HTTP Client's TLS layer doesn't match one of the SNI hosts in your keystore.

Know that SNI itself has restrictions:

• localhost is not allowed
• IP Address Literals are not allowed (eg: 192.168.1.215 or fe80::3831:400c:dc07:7c40)
• All SNI HostNames must contain at least 1 . in it's name (so don't use shorthand host/domain names, use fully qualified canonical host/domain names)

These restrictions are in the spec and are often the cause of most of the issues like what you are experiencing.

Depending on your version of Java these SNI restrictions are checked sooner in the Java networking stack then other versions of Java. (Java 11 is less restrictive with the . in hostname restriction vs Java 17+ which enforces it)