Search code examples
hadoophadoop2hadoop-streaminghadoop-partitioning

who will create the block ids for blocks in hadoop?


I wanted to know who will create the block ids for blocks in hadoop either HDFS client or Name node.Please let me know.


Solution

  • The NameNode allocates the block ID and gives it to the client. The client then uses this block ID while communicating with a DataNode to write data into the block.

    Apache JIRA HDFS-4645 documents the current design for allocation of block IDs. It uses a monotonically increasing ID starting from a specific constant. If you're interested in seeing the code for this, refer to the BlockIdManager and SequentialBlockIdGenerator classes in the Apache Hadoop codebase.