Search code examples
pysparkdatabricksazure-databricks

PySpark - Generate StructType from string


Is there a simple way to generate a schema from a structype definition from a string ?

For example I actualy do :

from pyspark.sql.types import *
customSchema = StructType([StructField("Date",StringType(),True)]) 

And i'm looking to store the schema définition in a table and load it dynamically like :

from pyspark.sql.types import *
stringShema = 'StructType([StructField("Date",StringType(),True)])'
customSchema = SomeFunctionToConvertTextToStruct(stringShema)

Any hint ? Regards,


Solution

  • Using eval?

    from pyspark.sql.types import *
    stringSchema = 'StructType([StructField("Date",StringType(),True)])'
    customSchema = eval(stringSchema)
    

    Olivier.