Search code examples
c#azurehadoopmapreducehue

How can I execute a C# MapReduce job on Cloudera Hue

I am new to Hadoop technology stack and have a little background in .Net, we have a Cloudera cluster running on top of Azure services and I was wondering if I can execute a C# (.Net Standard/Framework/Core) MapReduce job on my cluster to perform any task.

P.S. I know this can be done for HDInsight, but I am unable to find any way to make it work on Cloudera/Hue.

Thanks in Advance.

Solution

  • Per my experience, if you want to write a MapReduce job in C# or other languages not work on JVM, the only way to work with Hadoop/Cloudera is via Hadoop Streaming. So you need to create two console executers in Mono C#(.Net Standard) or dotNet Core C# as the mapper and/or the reducer for Cloudera on Linux.

    Hope it helps.