KafkaTemplate is sending acknowledgment header but when we send the message from StreamBridge acknowledgment is being sent. Application.properties:
server.port=8082
spring.cloud.stream.function.definition=sink1;sink2
spring.cloud.stream.function.bindings.sink1-in-0=inbound-events
spring.cloud.stream.bindings.inbound-events.group=ama1-channel-group
spring.cloud.stream.bindings.inbound-events.destination=squaredNumbers-test4
spring.cloud.stream.bindings.inbound-events.consumer.header-mode=headers
spring.cloud.stream.bindings.inbound-events.content-type=application/json
spring.cloud.stream.kafka.bindings.inbound-events.consumer.ack-mode=manual
spring.cloud.stream.function.bindings.sink2-in-0=inbound-stream
spring.cloud.stream.bindings.inbound-stream.group=ama2-channel-group
spring.cloud.stream.bindings.inbound-stream.destination=squaredNumbers-test5
spring.cloud.stream.bindings.inbound-stream.consumer.header-mode=headers
spring.cloud.stream.bindings.inbound-stream.content-type=application/json
spring.cloud.stream.kafka.bindings.inbound-stream.consumer.ack-mode=manual
Service class:
@Service
public class KafkaConsumer {
BindingServiceProperties bindingProperties;
StreamBridge streamBridge;
@Autowired
public KafkaConsumer(final BindingServiceProperties bindingServiceProperties, StreamBridge streamBridge) {
this.bindingProperties = bindingServiceProperties;
this.streamBridge = streamBridge;
}
@Bean
public Consumer<Message> sink1() {
return (message) -> {
System.out.println("******************");
System.out.println("At Sink1");
System.out.println("******************");
System.out.println("Received message " + message);
streamBridge.send("inbound-stream",MessageBuilder.fromMessage(message));
};
}
@Bean
public Consumer<Message> sink2() {
return (message) -> {
System.out.println("******************");
System.out.println("At Sink2");
System.out.println("******************");
System.out.println("Received message " + message);
};
}
}
Main application:
@SpringBootApplication(scanBasePackages = "demo")
public class MultipleFunctionsApplication {
public static void main(String[] args) {
SpringApplication.run(MultipleFunctionsApplication.class, args);
}
}
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>multi-functions-kafka</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>multi-functions-kafka</name>
<description>Spring Cloud Stream Sample Multiple functions Kafka</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.5</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<spring-cloud.version>2021.0.5</spring-cloud.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream</artifactId>
<scope>test</scope>
<classifier>test-binder</classifier>
<type>test-jar</type>
</dependency>
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro</artifactId>
<version>1.10.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/libs-snapshot-local</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</repository>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/libs-milestone-local</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/libs-snapshot-local</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
<releases>
<enabled>false</enabled>
</releases>
</pluginRepository>
<pluginRepository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/libs-milestone-local</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
<pluginRepository>
<id>spring-releases</id>
<name>Spring Releases</name>
<url>https://repo.spring.io/libs-release-local</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</project>
My use case: We receiving events from DB and we are trying to send them to channel using StreamBridge. We want manually ack the events.
Please suggest if I am missing something. spring-cloud-dependencies : 2021.0.5
It is because you are doing this
streamBridge.send("inbound-stream",MessageBuilder.fromMessage(message));
You are simply sending the message directly to the other binding's input channel. This bypasses all the Kafka logic; you need to send the record to Kafka, not directly to the input binding's channel.
When using a KafkaTemplate instead it is, of course, going via Kafka; that's why you are seeing different behavior.
You need an output binding with inbound-stream as its destination.