On starting spark-thrift-server using
spark-submit showing the job is submitted, running and completed within few seconds successfully however getting error as below -
ERROR SparkContext: Error initializing SparkContext.
MultiException[java.lang.IllegalStateException: class org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter is not a javax.servlet.Filter, java.lang.IllegalStateException: class org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter is not a javax.servlet.Filter]
Suppressed: java.lang.IllegalStateException: class org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter is not a javax.servlet.Filter
[CIRCULAR REFERENCE:java.lang.IllegalStateException: class org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter is not a javax.servlet.Filter]
On checking, hadoop-yarn-server-web-proxy jar, found hadoop-yarn-server-web-proxy-3.3.6.jar is there in hadoop as well as in spark both ->
$ jar -tf $SPARK_HOME/jars/hadoop-yarn-server-web-proxy-3.3.6.jar | grep AmIpFilter
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/opt/disk1/tmp
org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.class
$ jar -tf $HADOOP_HOME/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.3.6.jar | grep AmIpFilter
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/opt/disk1/tmp
org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.class
$ echo $HADOOP_CONF_DIR && echo $SPARK_HOME && echo $HADOOP_HOME
.../hadoop-3.3.6/etc/hadoop
.../spark-3.5.0-bin-hadoop3
.../hadoop-3.3.6
The reported class exists in two jars ->
This error is fixed by removing the hadoop-client-minicluster-3.3.6.jar from spark-3.5.0-bin-hadoop3/jars directory
The weird thing is this is not causing any problem on other servers but on one server.
How do i find existence of reported class in jars: -
find . -name "*.jar" -print | while read jar; do
if jar tf "$jar" | grep -q "org/apache/hadoop/yarn/server/webproxy/amfilter/AmIpFilter.class"; then
echo "$jar"
fi
done