I am trying to get a a simple Azure ML pipeline with the dogs vs cats data set following the steps - documented here
My notebook contains the following -
import azureml.core
from azureml.core import Workspace, Datastore
from azureml.core import Environment
from azureml.core.environment import CondaDependencies
from azureml.pipeline.steps import PythonScriptStep
ws = Workspace.from_config()
myenv = Environment(name="myenv")
conda_dep = CondaDependencies()
conda_dep.add_conda_package("keras")
conda_dep.add_conda_package("PIL")
myenv.python.conda_dependencies=conda_dep
myenv.register(workspace=ws)
After setting up the data reference and the compute, here's how I am creating the pipeline -
trainStep = PythonScriptStep(
script_name="dogs_vs_cats.py",
arguments=["--input", blob_input_data, "--output", output_data1],
inputs=[blob_input_data],
outputs=[output_data1],
compute_target=compute_target,
source_directory="../dogs-vs-cats"
)
Steps = [trainStep]
from azureml.pipeline.core import Pipeline
pipeline1 = Pipeline(workspace=ws, steps=[Steps])
from azureml.core import Experiment
pipeline_run1 = Experiment(ws, 'dogs_vs_cats_exp').submit(pipeline1)
pipeline_run1.wait_for_completion()
Once this steps is executed, the experiment fails and I get the following error after a bunch of information -
Traceback (most recent call last):
File "dogs_vs_cats.py", line 30, in <module>
import keras
ModuleNotFoundError: No module named 'keras'
The terminal shows my conda environment set to azureml_py36 and Keras seems be listed in the output of conda list.
Am I setting up the environment correctly? What is mising
From the way you have specified your environment, it's hard to see if it's a proper RunConfiguration object. If it is, it should be a matter of adding it to you PythonScriptStep.
trainStep = PythonScriptStep(
script_name="dogs_vs_cats.py",
arguments=["--input", blob_input_data, "--output", output_data1],
inputs=[blob_input_data],
outputs=[output_data1],
compute_target=compute_target,
source_directory="../dogs-vs-cats",
runconfig=myenv
)
Right now you're defining the environment, but no using it anywhere it seems. If your trouble persists maybe try defining your RunConfiguration like they do under the "Specify the environment to run the script" step in this notebook: