I just started following the dagster tutorial. I managed to get the hello_cereal job running with dagit and the Python API, but for some reason when trying with dagster CLI
dagster job execute -f hello_cereal.py
I am getting a DagsterUnmetExecutorRequirementsError:
2021-10-22 15:33:22 - dagster - ERROR - hello_cereal_job - 272b37fb-9f39-44dc-b63a-dcd3dfbb7956 - 880 - RUN_FAILURE - Execution of run for "hello_cereal_job" failed. Pipeline failure during initialization for pipeline "hello_cereal_job". This may be due to a failure in initializing the executor or one of the loggers.
dagster.core.errors.DagsterUnmetExecutorRequirementsError: You have attempted to use an executor that uses multiple processes with an ephemeral DagsterInstance. A non-ephemeral instance is needed to coordinate execution between multiple processes. You can configure your default instance via $DAGSTER_HOME or ensure a valid one is passed when invoking the python APIs. You can learn more about setting up a persistent DagsterInstance from the DagsterInstance docs here: https://docs.dagster.io/deployment/dagster-instance#default-local-behavior
Indeed, I don't have $DAGSTER_HOME set, but since it is working with the web UI and the Python API versions, I was wondering if I made a mistake elsewhere ?
I am on macOS BigSur (11.6), on a fresh miniconda installation (v4.10.3), Python 3.9.5 and dagster 0.13.0.
thanks for the report - this issue was fixed in the 0.13.1 release that went out today on 10/25. That command should work without needing to worry about setting DAGSTER_HOME.