Search code examples
hadoopupgraderhelambari

HDP 2.1 to 2.2 upgrade RHEL6

I have a cluster with 1 NameNode and 4 DataNodes on Red Hat Linux Enterprise 6. My HDP version is 2.1. Ambari version was 1.7 but I upgraded it to 2.1. I want to upgrade HDP to version 2.2. I read that if I want to upgrade HDP from 2.1 to 2.2 I have to do it before I upgrade Ambari to 2.1. When I am upgrading hdp to 2.2 ambari does not see any changes and everything is not working. I am using this tutorial: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/HDP_Man_Upgrade_v22/index.html#Item1

How can I do it? I tried to downgrade ambari to 1.7 but I got many errors. What if I try upgrade now hdp to 2.2 and then my ambari from 2.1 to 2.1.1. Will it work? The problem is that I have very little time. Thank you in advance

Solution

  • I am upgrading from HDP-2.0/Hadoop-2.2 to HDP-2.2/Hadoop-2.6 (maybe temporarily, on the way to HDP-2.3) on a development/testing cluster. So far I have gotten the updated HDFS up and running. Not yet starting/stopping HDFS via Ambari, nor do I have YARN running yet. Update: I got YARN, MapReduce, and Hive to run after finding HDP-2.2 documentation (current links added below).

    Here are my rough notes on how I got this far:

    1. upgrade ambari-1.4 to 1.7
    2. update HDP yum repo file on each node and update with yum (cssh) http://public-repo-1.hortonworks.com/HDP/centos6/2.x/updates/2.2.6.0/hdp.repo
    3. hdp-select
    4. sudo su -l hdfs -c "/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh start namenode -upgrade"
      /etc/hadoop/conf.empty/core-site.xml.rpmsave, hdfs-site.xml
      sudo su -l hdfs -c "/usr/hdp/current/hadoop-hdfs-datanode/../hadoop/sbin/hadoop-daemon.sh start datanode"
      hadoop dfsadmin -finalizeUpgrade

    [update]

    1. upgrade ambari-1.7 to 2.1
    2. configure yarn & mapreduce - yarn-site.xml, yarn-env.sh[?], container-executor.cfg, mapred-site.xml
      sudo ln -s /usr/hdp/2.2.6.0-2800/hadoop/libexec/ /usr/lib/hadoop/ # ambari insisting /usr/lib/hadoop
      stop-start resourcemanager, start nodemanagers
    3. hive-site.xml in both conf.dist & conf.server; manual start

    The following resources were useful: https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+1.7.0+from+Public+Repositories
    https://developer.ibm.com/hadoop/blog/2015/10/08/back-up-and-restore-ambari-server-postgresql-database/
    http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_Installing_HDP_AMB/content/_hdp_stack_repositories.html
    http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_upgrading_Ambari/content/_Upgrade_HDFS_mamiu.html
    https://wiki.apache.org/hadoop/Hadoop_Upgrade
    http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_upgrading_Ambari/content/_complt_upgrd_21-23_upgrade_hdfs.html
    http://solaimurugan.blogspot.com/2014/11/upgrade-hadoop-with-latest-version.html
    https://cwiki.apache.org/confluence/display/AMBARI/Install+Ambari+2.0.2+from+Public+Repositories
    [update]
    http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/bk_upgrading_hdp_manually/content/index.html
    http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/bk_upgrading_hdp_manually/content/configure-yarn-mr-21.html
    http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.2.0/bk_upgrading_hdp_manually/content/start-hive-hcat-21.html
    https://brucebcampbell.wordpress.com/2014/12/11/hortonworks-fix-missing-jar-error-in-hive-after-upgrade-to-hdp-2-2/