I have used poetry to create a wheel file. I am running following spark-submit command , but it is not working. I think I am missing something
spark-submit --py-files /path/to/wheel
Please note that I have referred to below as well, but did not get much details as I am new to Python. how to pass python package to spark job and invoke main file from package with arguments
Wheel file can be executed as a part of below spark-submit command
spark-submit --deploy-mode cluster --py-files /path/to/wheel main_file.py