Search code examples
hadoophdfshadoop-yarnhadoop2

Various ports in hadoop cluster?

I am trying to understand the various ports at which various daemons / processes listens at in a Hadoop cluster.

core-site.xml

    <property>
            <name>fs.defaultFS</name>
            <value>hdfs://master.hadoop.cluster:54310</value>
    </property>

yarn-site.xml

    <property>
            <name>yarn.resourcemanager.address</name>
            <value>master.hadoop.cluster:8032</value>
    </property>

I see we have three other ports, which are:

1) 50070 --> To see hdfs GUI
2) 8088 --> To see RM GUI
3) 8042 --> Not sure which GUI we can see at this port

As there are so many ports, I am not clear which port is meant for which thing. If I make HTTP request to port, say at 8032, it says this is Hadoop IPC port.

Can anyone help me understand this, what are the main port numbers which we should be aware of, and what processes listens at those ports.

Solution

  • The port defined in fs.defaultFS is for file system metadata operations. You cannot use it for accessing the Web UI. 8042 is for NodeManager Web UI and 8032 is for ResourceManager IPC.

    Refer