Installing Kafka on DCOS without any custom configuration gives following logs
Cluster Configuration is:
8 Private Nodes, 2 Public Nodes
24 Cores Total, 320 GB Ram Total
All resources are free.
StdErr Output:
log4j:WARN No appenders could be found for logger (org.jboss.logging).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NullPointerException
at com.mesosphere.dcos.kafka.plan.KafkaUpdateBlock.initializeStatus(KafkaUpdateBlock.java:196)
at com.mesosphere.dcos.kafka.plan.KafkaUpdateBlock.<init>(KafkaUpdateBlock.java:50)
at com.mesosphere.dcos.kafka.plan.KafkaUpdatePhase.createBlocks(KafkaUpdatePhase.java:83)
at com.mesosphere.dcos.kafka.plan.KafkaUpdatePhase.<init>(KafkaUpdatePhase.java:27)
at com.mesosphere.dcos.kafka.scheduler.KafkaScheduler.<init>(KafkaScheduler.java:125)
at com.mesosphere.dcos.kafka.scheduler.Main.run(Main.java:70)
at com.mesosphere.dcos.kafka.scheduler.Main.run(Main.java:21)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:40)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:77)
at io.dropwizard.cli.Cli.run(Cli.java:70)
at io.dropwizard.Application.run(Application.java:80)
at com.mesosphere.dcos.kafka.scheduler.Main.main(Main.java:27)
Stdout Output:
ERROR com.mesosphere.dcos.kafka.scheduler.KafkaScheduler: Got 1 errors from new config. Falling back to last valid config.
INFO com.mesosphere.dcos.kafka.plan.KafkaUpdateBlock: Setting initial status for: broker-0
INFO com.mesosphere.dcos.kafka.plan.KafkaUpdateBlock: TargetConfigName: 67004a73-e9a0-4bf6-8e01-9eb52a674c1a currentConfigName: null
Command exited with status 1 (pid: 10171)
Fixed the problem by following instructions here: https://docs.mesosphere.com/1.8/usage/service-guides/kafka/uninstall
To fresh install Kafka package we first need to clear up its previous state from the Zookeeper. Just uninstalling it from the GUI or CLI doesn't work, after uninstall clear the previous state from zookeeper and then fresh install.