Start Apache Kafka in Kraft mode as a servcice on Ubuntu

Question

I am using the instructions from this answer to set-up Kafka as a service on Ubuntu-22.04 on AWS EC2. I am installing Kafka using instructions from here. Here is my /etc/init.d/kafka file. Please note that, I am able to launch Kafka successfully from the command line; but, the launch fails as a service.

KAFKA_HOME=/opt/kafka
# See how we were called.
case "$1" in
  start)
        # Start daemon.
        echo "Generate cluster UUID"
        KAFKA_CLUSTER_ID="$(${KAFKA_HOME}/bin/kafka-storage.sh random-uuid)"
        echo "Format log directories"
        ${KAFKA_HOME}/bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c ${KAFKA_HOME}/config/kraft/server.properties
        echo "Start Kafka server"
        ${KAFKA_HOME}/bin/kafka-server-start.sh -daemon ${KAFKA_HOME}/config/kraft/server.properties
        ;;
  stop)
        # Stop daemons.
        echo "Shutting down Kafka";
        pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
        if [ -n "$pid" ]
          then
          kill -9 $pid
        else
          echo "Kafka was not Running"
        fi
        ;;
  restart)
        $0 stop
        sleep 2
        $0 start
        ;;
  status)
        pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
        if [ -n "$pid" ]
          then
          echo "Kafka is Running as PID: $pid"
        else
          echo "Kafka is not Running"
        fi
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
esac

exit 0

Upon running sudo systemctl kafka start I get the following error.

Job for kafka.service failed because the control process exited with error code.
See "systemctl status kafka.service" and "journalctl -xeu kafka.service" for details.

And, sudo systemctl kafka status shows this.

× kafka.service
     Loaded: loaded (/etc/init.d/kafka; generated)
     Active: failed (Result: exit-code) since Fri 2023-10-27 15:42:14 UTC; 2s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 8492 ExecStart=/etc/init.d/kafka start (code=exited, status=203/EXEC)
        CPU: 946us

Oct 27 15:42:14 ip-172-31-15-126 systemd[8492]: kafka.service: Failed to execute /etc/init.d/kafka: Exec format error
Oct 27 15:42:14 ip-172-31-15-126 systemd[8492]: kafka.service: Failed at step EXEC spawning /etc/init.d/kafka: Exec format error
Oct 27 15:42:14 ip-172-31-15-126 systemd[1]: Starting kafka.service...
Oct 27 15:42:14 ip-172-31-15-126 systemd[1]: kafka.service: Control process exited, code=exited, status=203/EXEC
Oct 27 15:42:14 ip-172-31-15-126 systemd[1]: kafka.service: Failed with result 'exit-code'.
Oct 27 15:42:14 ip-172-31-15-126 systemd[1]: Failed to start kafka.service.

Here is my Java version - if it helps.

java -version
openjdk version "19.0.2" 2023-01-17
OpenJDK Runtime Environment (build 19.0.2+7-Ubuntu-0ubuntu322.04)
OpenJDK 64-Bit Server VM (build 19.0.2+7-Ubuntu-0ubuntu322.04, mixed mode, sharing)

Please let me know how to launch Kafka as a service.

Answer 1

Putting the shebang on top of the starting shell script probably will fix this error. If it does not fix the error you can check journalctl as explained here also this maybe related to systemd configuration in that case I would suggest to create a service file something like this

why we use shebang, in short it is to telling linux system to use bash to run the code "below" in the file.

working code below

#!/bin/bash
KAFKA_HOME=/opt/kafka
# See how we were called.
case "$1" in
  start)
        # Start daemon.
        echo "Generate cluster UUID"
        KAFKA_CLUSTER_ID="$(${KAFKA_HOME}/bin/kafka-storage.sh random-uuid)"
        echo "Format log directories"
        ${KAFKA_HOME}/bin/kafka-storage.sh format -t $KAFKA_CLUSTER_ID -c ${KAFKA_HOME}/config/kraft/server.properties
        echo "Start Kafka server"
        ${KAFKA_HOME}/bin/kafka-server-start.sh -daemon ${KAFKA_HOME}/config/kraft/server.properties
        ;;
  stop)
        # Stop daemons.
        echo "Shutting down Kafka";
        pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
        if [ -n "$pid" ]
          then
          kill -9 $pid
        else
          echo "Kafka was not Running"
        fi
        ;;
  restart)
        $0 stop
        sleep 2
        $0 start
        ;;
  status)
        pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
        if [ -n "$pid" ]
          then
          echo "Kafka is Running as PID: $pid"
        else
          echo "Kafka is not Running"
        fi
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|status}"
        exit 1
esac

exit 0