I'm trying to ETL a table with a UUID in one column into a postgres table but I'm struggling to load the UUID column into the destination postgres table as a UUID type.
Using the following line of code:
applymapping1 = ApplyMapping.apply(frame = foo, mappings = [("id", "string", "id", "uuid")], transformation_ctx = "applymapping1")
I receive the following error when I run the glue job:
IllegalArgumentException: 'Invalid type name guid'
Does anyone know how I can store the id as a UUID in the destination database table?
Also, I can't seem to find the list of supported data types that I am allowed to use in the ApplyMapping call. Does anyone know where to find these?
AWS Glue does not support uuid as datatype. Following document gives the details about all the supported datatypes in AWS glue ETL, which are extension to pyspark types :