Suppress traceback message in spark

I am building a wheel package for using within spark using Azure synapse. During validation, I raise custom exceptions.

When exception occurs, spark includes the traceback in the notebook cells. How can we suppress the traceback and request spark to just display the error message?

I understand print, but this is an error message.

Solution

You can do it like below.

My code which gives error because of invalid path:

df = spark.read.options(header='True', inferSchema='True', delimiter=',').csv("/tmp/mycsv2.csv")

Error with traceback:

enter image description here

For whatever the Custom Exceptions, use the below code which worked for me to avoid the traceback.

try:
    df = spark.read.options(header='True', inferSchema='True', delimiter=',').csv("/tmp/mycsv2.csv")
except  Exception  as err:
    print(err)

enter image description here