Search code examples
mesosmesosphere

How to run HDFS on Mesos in production environment?

I see two options:

  1. Use the the HDFS Mesos repository which is currently deprecated and is suggested not to be used for production workloads.

  2. Use the dc/os HDFS which is fully supported by Mesosphere.

I wonder if there is a way to use the Mesosphere/community supported HDFS without DC/OS.

Solution

  • Running Mesosphere supported HDFS without DC/OS is not easily possible, due to the Service Package approach. I would suggest deploy your own HDFS from Cloudera stack. The Datanode should run on Mesos slaves, the Namenode master should run on Master server, follow https://github.com/mesos/hadoop. Job Tracker should be running on Master node, Mesos should be able to start the task with HDFS underlying.