Search code examples
amazon-web-servicesaws-databrew

In AWS Databrew, how can I stop the Databrew job from partitioning the result file?


All Databrew jobs that saves the result in S3 creates partitions of the resultant file. Due to this, we need to merge this partitioned files before we can use them in Excel. Is there any way by which I can stop the Databrew job from partitioning the resultant file?


Solution

  • When you go to create a DataBrew recipe job, there is a way to setup "File partitioning" option for each of your outputs. The option you are looking for is called "Single file output", select this option and run the job to create a single file per custom column based partitions out of your DataBrew recipe job.

    enter image description here

    AWS Documentation link: https://docs.aws.amazon.com/databrew/latest/dg/jobs.recipe.html#singlefileoutput