Search code examples
pythonpython-3.xazure-machine-learning-service

log metric from inside PythonScriptStep to parent PipelineRun


SDK version: 1.0.43

To minimize clicking and compare accuracy between PipelineRuns, I'd like to log a metric from inside a PythonScriptStep to the parent PipelineRun. I thought I could do this like:

from azureml.core import Run
run = Run.get_context()
foo = 0.80
run.parent.log("accuracy",foo)

however I get this error.

Traceback (most recent call last):
  File "get_metrics.py", line 62, in <module>
    run.parent.log("geo_mean", top3_runs)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 459, in parent
    return None if parent_run_id is None else get_run(self.experiment, parent_run_id)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 1713, in get_run
    return next(runs)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/run.py", line 297, in _rehydrate_runs
    yield factory(experiment, run_dto)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/run.py", line 325, in _from_dto
    return PipelineRun(experiment=experiment, run_id=run_dto.run_id)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/run.py", line 74, in __init__
    service_endpoint=_service_endpoint)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_graph_context.py", line 46, in __init__
    service_endpoint=service_endpoint)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 118, in create_provider
    service_endpoint=service_endpoint)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 133, in create_service_caller
    service_endpoint = _AevaWorkflowProvider.get_endpoint_url(workspace, experiment_name)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/pipeline/core/_aeva_provider.py", line 153, in get_endpoint_url
    workspace_name=workspace.name, workspace_id=workspace._workspace_id)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/workspace.py", line 749, in _workspace_id
    self.get_details()
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/workspace.py", line 594, in get_details
    self._subscription_id)
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/_project/_commands.py", line 507, in show_workspace
    AzureMachineLearningWorkspaces, subscription_id).workspaces,
  File "/azureml-envs/azureml_ffecfef6fbfa1d89f72d5af22e52c081/lib/python3.6/site-packages/azureml/core/authentication.py", line 112, in _get_service_client
    all_subscription_list, tenant_id = self._get_all_subscription_ids()
TypeError: 'NoneType' object is not iterable

Update

On further investigation, I tried just printing the parent attribute of the run with the line below and got the same Traceback

print("print run parent attribute", run.parent)

The get_properties() method the below. I'm guessing that azureml just uses the azureml.pipelinerunid property for pipeline tree hierarchy, and that the parent attribute has been left for any user-defined hierarchies.

{
    "azureml.runsource": "azureml.StepRun",
    "ContentSnapshotId": "45bdecd3-1c43-48da-af5c-c95823c407e0",
    "StepType": "PythonScriptStep",
    "ComputeTargetType": "AmlCompute",
    "azureml.pipelinerunid": "e523d575-c373-46d2-a4bc-1717f5e34ec2",
    "_azureml.ComputeTargetType": "batchai",
    "AzureML.DerivedImageName": "azureml/azureml_dfd7f4f952ace529f986fe919909c3ec"
}

Solution

  • Please upgrade your SDK to the latest version. Seems like this issue was fixed sometime after 1.0.43.