Search code examples
hadoopclouderacloudera-manager

what is the different between configuration files under /etc/hadoop/conf and /etc/hadoop/conf.cloudera.HDFS and /etc/hadoop/conf.cloudera.YARN

I have cloudera 5.7, I have Cloudera Manager too.

under the directory /etc/hadoop, I saw three sub-directories. /etc/hadoop/conf /etc/hadoop/conf.cloudera.HDFS/ /etc/hadoop/conf.cloudera.YARN/

the hadoop-env.sh in ../conf/ is different from ../conf.cloudera.HDFS/.. the core-site.xml in ../conf/ is different from ../conf.cloudera.HDFS/.. as well.

the hadoop-env.sh in ../conf/ has settings on YARN, while the one under../conf.cloudera.HDFS doesn't has it.

and the one in ../conf.cloudera.HDFS/.. has the settings for Namenode, datanodes, etc.

I have CM installed, I am wondering if these configuration files are really in use?

If yes, and I need to change some environment variables, should I change all of these hadoop-env.sh? and copy it to the other nodes?

Thanks.

Solution

  • Cloudera Manager handle settings for you. If you edit the settings files manually, it will erase by CM. If you want make some change, do it by CM.