kafka kRaft with schema registry

we installed new kafka cluster with Kafka version - 3.5.x on linux 7.x machines

Kafka KRaft cluster is actually without zookeeper servers and we installed 3 Kafka servers

from Kafka cluster point of view every thing is fine and services are up and running

the next step is to install the schema registry services , and we are using the schema registry version - 5.4.3 ( we download the rpm from confluent site - confluent-schema-registry-5.4.3 rpm )

after installing the schema registry rpm's , we start the schema registry services , but services not started due problem about - Timed out trying to create or validate schema topic configuration

but we noticed about the following exception from schema registry logs

ERROR Error starting the schema registry (io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication)
io.confluent.kafka.schemaregistry.exceptions.SchemaRegistryInitializationException: Error initializing kafka store while initializing schema registry
        at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:250)
        at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.initSchemaRegistry(SchemaRegistryRestApplication.java:75)
        at io.confluent.kafka.schemaregistry.rest.SchemaRegistryRestApplication.configureBaseApplication(SchemaRegistryRestApplication.java:90)
        at io.confluent.rest.Application.configureHandler(Application.java:217)
        at io.confluent.rest.ApplicationServer.doStart(ApplicationServer.java:185)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
        at io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain.main(SchemaRegistryMain.java:43)
Caused by: io.confluent.kafka.schemaregistry.storage.exceptions.StoreInitializationException: Timed out trying to create or validate schema topic configuration
        at io.confluent.kafka.schemaregistry.storage.KafkaStore.createOrVerifySchemaTopic(KafkaStore.java:177)
        at io.confluent.kafka.schemaregistry.storage.KafkaStore.init(KafkaStore.java:119)
        at io.confluent.kafka.schemaregistry.storage.KafkaSchemaRegistry.init(KafkaSchemaRegistry.java:248)
        ... 6 more
Caused by: java.util.concurrent.TimeoutException
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:108)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:272)
        at io.confluent.kafka.schemaregistry.storage.KafkaStore.createOrVerifySchemaTopic(KafkaStore.java:170)
        ... 8 more

just to comparing the scenario with non Kraft Kafka cluster , in case for example we used non Kraft Kafka version as Kafka 2.8.X with the same schema registry version then no issue from schema registry services

so seems exception as described above is relevant when installing Kafka Kraft cluster

so what is the right approach in order to install the schema registry services on Kafka KRaft Linux machines?

here is example of how to start the schema registry service ( from systemctl service )

/usr/bin/schema-registry-start /etc/schema-registry/schema-registry.properties

and how looks the schema registry properties file:

more /etc/schema-registry/schema-registry.properties
#
# Copyright 2018 Confluent Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

# The address the socket server listens on.
#   FORMAT:
#     listeners = listener_name://host_name:port
#   EXAMPLE:
#     listeners = PLAINTEXT://your.host.name:9092
listeners=http://0.0.0.0:8081

# Zookeeper connection string for the Zookeeper cluster used by your Kafka cluster
# (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
#kafkastore.connection.url=

# Alternatively, Schema Registry can now operate without Zookeeper, handling all coordination via
# Kafka brokers. Use this setting to specify the bootstrap servers for your Kafka cluster and it
# will be used both for selecting the master schema registry instance and for storing the data for
# registered schemas.
# (Note that you cannot mix the two modes; use this mode only on new deployments or by shutting down
# all instances, switching to the new configuration, and then starting the schema registry
# instances again.)
kafkastore.bootstrap.servers=PLAINTEXT://kafka1:9092,kafka2:9092,kafka3:9092

# The name of the topic to store schemas in
kafkastore.topic=_schemas

# If true, API requests that fail will include extra debugging information, including stack traces
debug=true

Solution

Your RPM packages are outdated. You should at least have a 7.4 version to use latest Confluent Platform release with Kraft.

Your config is fine, assuming you have 3 reachable brokers available at each of those hostnames that are running plaintext listeners...

Ultimately, Schema Registry doesn't care about Kraft mode. Your issue exists elsewhere

You could also create your _schemas topic manually (1 partition, cleanup.policy=compact, as many replicas as you want)