Add .csv extension in output file AWS Python Glue Job

Question

I am trying to save csv file from python glue dynmaic frame. Below is my code -

glueContext.write_dynamic_frame.from_options(
frame=splender_df,
connection_type="s3",
connection_options={"path": 's3://splender_df/'},
format="csv",

But it is creating 20 files and none of them has csv extension(.csv). How can I have only one file with extension .csv in the name of the file.

Answer 1

Did you specified the extra options such as separators and quote_char? Like the following example:

glueContext.write_dynamic_frame.from_options(
frame = datasource1,
connection_type = "s3", 
connection_options = {
    "path": "s3://s3path"
    }, 
format = "csv", 
format_options={
    "quoteChar": -1, 
    "separator": "|"
    }, 
transformation_ctx = "datasink2")

You can find these options in this link.