How to run flow with minimal configuration in mosaic

Question

I'm developing some ETL jobs using Mosaic Decisions. While running the job, it's submitting the job to Spark using the default configurations. This default configuration is really huge and I don't need that much for development (as I am using less number of records for unit testing while development).

Is there a way I can instruct Mosaic to use less Spark resources for my development? So that I won't unnecessarily block the resources of the cluster?

Answer 1

Yes, it is possible to achieve that. To do so, you will have to create a new run configuration with the desired resource configuration from the Manager persona (LTI Mosaic Manager). Then, simply execute the flow with the newly created run configuration.

Follow the steps below to create a new run configuration:

1. Log in to Mosaic Decisions and on the top right corner, click on Projects, and then on Manager.

2. In Mosaic Manager, click on the Runconfig tab in the left navigation panel.
3. Click on Add New Configuration. Provide the desired configurations and click Save.

4. Go back to Mosaic Decisions, and execute the desired flow with the newly created run configuration