On the project, it was necessary to update the JDK to version 17 from 11 and gradle to gradle-7.3-all from gradle-7.0.2-all. After that, when I tried to launch the application, an error began to appear that the InternalKafkaListenerEndpointRegistry bean failed to start. Here is the log:
2024-03-12 18:08:44,822 [main] - Application run failed
org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:181)
at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155)
at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:782)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:774)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:439)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:339)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1340)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329)
at some.SomeApplication.main(SomeApplication.java:12)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
Caused by: org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:823)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:632)
at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createRawConsumer(DefaultKafkaConsumerFactory.java:358)
at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:326)
at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumerWithAdjustedProperties(DefaultKafkaConsumerFactory.java:302)
at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createKafkaConsumer(DefaultKafkaConsumerFactory.java:269)
at org.springframework.kafka.core.DefaultKafkaConsumerFactory.createConsumer(DefaultKafkaConsumerFactory.java:243)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.<init>(KafkaMessageListenerContainer.java:639)
at org.springframework.kafka.listener.KafkaMessageListenerContainer.doStart(KafkaMessageListenerContainer.java:305)
at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:338)
at org.springframework.kafka.listener.ConcurrentMessageListenerContainer.doStart(ConcurrentMessageListenerContainer.java:204)
at org.springframework.kafka.listener.AbstractMessageListenerContainer.start(AbstractMessageListenerContainer.java:338)
at org.springframework.kafka.config.KafkaListenerEndpointRegistry.startIfNecessary(KafkaListenerEndpointRegistry.java:312)
at org.springframework.kafka.config.KafkaListenerEndpointRegistry.start(KafkaListenerEndpointRegistry.java:257)
at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178)
... 23 common frames omitted
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:172)
at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:157)
at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:73)
at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:105)
at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:740)
... 37 common frames omitted
Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner authentication information from the user
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:925)
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:745)
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:597)
at java.base/javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:679)
at java.base/javax.security.auth.login.LoginContext$4.run(LoginContext.java:677)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:677)
at java.base/javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:60)
at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:103)
at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:62)
at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:105)
at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:158)
... 41 common frames omitted
Here is my jaas config:
@Component
@Setter
@Getter
@ConditionalOnProperty(value = "kafka.etl.enabled", havingValue = "true", matchIfMissing = true)
class KafkaEtlProperties {
private static final String SECURITY_PROTOCOL = "security.protocol";
private static final String SASL_MECHANISM = "sasl.mechanism";
private static final String SASL_KERBEROS_SERVICE_NAME = "sasl.kerberos.service.name";
private static final String SASL_JAAS_CONFIG = "sasl.jaas.config";
private final Map<String, String> props;
public KafkaEtlProperties(@Value("${kafka.etl.principal:${spring.datasource.username}}") String principal) {
props = new HashMap<>();
props.put(SECURITY_PROTOCOL, "SASL_PLAINTEXT");
props.put(SASL_MECHANISM, "GSSAPI");
props.put(SASL_KERBEROS_SERVICE_NAME, "kafka");
props.put(SASL_JAAS_CONFIG, """
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/kerberos/keytab"
principal="%s";
""".formatted(principal));
}
}
Keytab and other did not change. When I went back to jdk 11 and gradle-7.0.2-all everything worked again.
I don't understand why this happened and I want to figure out how to fix it.
UPD: Here is my build.gradle:
buildscript {
ext {
springBootVersion = '2.4.5'
}
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
sourceCompatibility = JavaVersion.VERSION_17
jar {
enabled = true
}
dependencies {
implementation group: 'org.springframework.kafka', name: 'spring-kafka'
implementation project(":some-core")
implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
implementation 'org.flywaydb:flyway-core'
implementation 'org.apache.commons:commons-lang3'
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310'
compileOnly 'org.projectlombok:lombok'
implementation 'org.reflections:reflections:0.9.10'
implementation 'org.postgresql:postgresql'
testImplementation("org.springframework.boot:spring-boot-starter-test") {
exclude group: "org.junit.vintage", module: "junit-vintage-engine"
}
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.3.1'
testImplementation 'org.mockito:mockito-junit-jupiter:2.23.4'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.3.1'
testImplementation("org.junit.jupiter:junit-jupiter-params:5.4.2")
annotationProcessor 'org.projectlombok:lombok'
}
Here "some-core" project build.gradle:
buildscript {
ext {
springBootVersion = '2.4.5'
}
repositories {
mavenLocal()
mavenCentral()
}
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
}
}
apply plugin: 'java'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'
sourceCompatibility = JavaVersion.VERSION_17
jar {
enabled = true
}
bootJar {
enabled = false
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-mail'
implementation 'org.springframework.boot:spring-boot-starter-cache'
implementation 'org.springframework:spring-context-support'
implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
implementation 'org.springframework.kafka:spring-kafka'
implementation 'com.github.ben-manes.caffeine:caffeine'
implementation 'org.hibernate.validator:hibernate-validator-annotation-processor'
implementation 'org.hibernate.validator:hibernate-validator'
implementation 'org.hibernate:hibernate-entitymanager'
implementation 'org.flywaydb:flyway-core'
compileOnly 'javax.validation:validation-api'
implementation 'org.reflections:reflections:0.9.10'
implementation 'org.apache.commons:commons-lang3'
implementation 'com.fasterxml.jackson.core:jackson-core'
implementation 'com.fasterxml.jackson.core:jackson-annotations'
implementation 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310'
implementation group: 'org.apache.poi', name: 'poi-ooxml', version: '3.9'
implementation 'com.fasterxml.jackson.core:jackson-databind'
implementation 'org.postgresql:postgresql'
compileOnly 'org.projectlombok:lombok'
annotationProcessor 'org.projectlombok:lombok'
implementation group: 'com.google.guava', name: 'guava', version: '27.0.1-jre'
implementation group: 'org.apache.commons', name: 'commons-csv', version: '1.6'
annotationProcessor 'org.hibernate:hibernate-jpamodelgen:5.4.9.Final'
implementation 'com.vladmihalcea:hibernate-types-52:2.9.8'
testImplementation 'org.junit.jupiter:junit-jupiter-api'
testImplementation 'org.mockito:mockito-junit-jupiter'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine'
testImplementation 'org.junit.jupiter:junit-jupiter-params'
}
UPD 2:
I solved the problem, but not on the Java side. It looks like keytab encryption via -crypto RC4-HMAC-NT is already deprecated in JDK 17. I encrypted via -crypto ALL and now the pods are up. But how can this be solved from the side of the java-service, and not from the keytab? Is there really no backward compatibility for RC4-HMAC-NT?
I solved the problem, but not on the Java side. It seems that keytab encryption via -crypto RC4-HMAC-NT is considered insecure in JDK 17. I encrypted via -crypto ALL and now the pods are up, the error is gone.
The following also helped me a lot in answering my question: JDK 17 (Java 17) +Kerberos authentication fail