Force Python2 with spark-submit

Question

I am creating a Spark application with AWS EMR but spark-submit runs with Python 3 instead of Python 2. But when I run pyspark instead, it is Python 2.

How can I force spark-submit to use Python 2?

I tried to do

export PYSPARK_PYTHON=/usr/bin/python2 

but it didn't work.

Thanks

Answer 1

Actually I had this in my code

from __future__ import print_function

and when I was running print 'hello world' it was crashing because it's not the default print function. But I thought it was crashing because it was using Python 3 instead of Python 2.