Search code examples
hadoophadoop-yarnspring-cloud-dataflow

Unsuccessful deployment on Spring Cloud Dataflow with Apache YARN

I have installed a single-node Apache YARN with Kafka-Zookeeper and Spring Cloud Dataflow 1.0.3.

All is working fine, but when I made some deployment examples, like:

stream create --name "ticktock" --definition "time | hdfs --rollover=100" --deploy

http --port=8000 | log --name=logtest --level=INFO

The stream's status never stays on "deployed". It keeps changing on "undeployed"->"parcial"->"deployed" in a constant loop.

However, the YARN application deploys successfully, but it's like the comunication between Spring Cloud Dataflow server instance and Apache Hadoop it's constantly failing.

What can be the possible issue for this?

Thanks in advance!

Solution

  • Thanks to all for your answers! Been busy with other high-priority projects, but I was able to build all using Ambari (including the plugin mentioned by Sabby Adannan). Now all is working great!.