Search code examples
apache-spark

Show the options set on a Spark DataFrame Reader

We have set a number of options via a combination of spark.read.option(optName, optVal) and spark.read.options(dictOfOptions). How can the values that have been set be read back from the DataFrameReader instance? The API does not show anything Spark DataFrame Reader API

Solution

  • Options are kept in private field extraOptions and by looking into code I don't see there is a way of reading it.