Search code examples
hadoopmapreducehadoop-yarnelastic-map-reduce

Which YARN configuration parameters are re-read on each application?

I've got one job that's much bigger than the other 50 or so that run in my daily workflow. I'd like the property yarn.app.mapreduce.am.resource.mb to be larger for just the big job. Am I in luck? How can I tell which properties require a complete restart of which pieces of YARN across my cluster to be changed?

Solution

  • This setting can be changed on job level without YARN service restart. It's hard to tell in general when you can change each MR property. Usually job level properties like mapper/reducer/appmaster memory settings can be updated on job level.

    You can check this helpful post to get a deeper look on this topic.