Search code examples
hadoopdistributed-cache

Hadoop Distributed Cache error message interpretation


I'm trying to place 3 files in the Distributed Cache.

I place one of them programmatically, using:

  DistributedCache.addLocalFiles(conf, "local/path/to/file");

And I also place 2 other files using the -files option.

When running locally, in standalone mode, everything works fine.

When trying to run it in pseudo-distributed mode, I get this error and I don't understand its meaning. I googled for it but with no success.

  Distributed cache entry arrays have different lengths

Does anyone know what it means and how it could be solved ?


Solution

  • The problem stemmed from the fact that I was adding to the cache local files.

    Instead I moved those files to the HDFS and added the HDFS version to the DistributedCache. It worked!