Search code examples
pythonpython-3.xpandaspipelinekedro

Kedro : Failed to find the pipeline named '__default__'


Having issues with kedro. The 'register_pipelines' function doesn't seem to be running or creating the default Pipeline that I'm returning from it.

The error is

(kedro-environment) C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files>kedro run
2021-03-22 13:30:28,201 - kedro.framework.session.store - INFO - `read()` not implemented for `BaseSessionStore`. Assuming empty store.
fatal: not a git repository (or any of the parent directories): .git
2021-03-22 13:30:28,447 - kedro.framework.session.session - WARNING - Unable to git describe C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files
2021-03-22 13:30:28,476 - root - INFO - ** Kedro project dcs_files
2021-03-22 13:30:28,486 - kedro.framework.session.store - INFO - `save()` not implemented for `BaseSessionStore`. Skipping the step.
Traceback (most recent call last):
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\context\context.py", line 304, in _get_pipeline
    return pipelines[name]
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\dynaconf\utils\functional.py", line 17, in inner
    return func(self._wrapped, *args)
KeyError: '__default__'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Anaconda3\envs\kedro-environment\Scripts\kedro-script.py", line 9, in <module>
    sys.exit(main())
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\cli\cli.py", line 228, in main
    cli_collection(**cli_context)
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\cc667216\OneDrive\DCS_Pipeline\dcs_files\src\dcs_package\cli.py", line 240, in run
    pipeline_name=pipeline,
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\session\session.py", line 344, in run
    pipeline = context._get_pipeline(name=pipeline_name)
  File "C:\Anaconda3\envs\kedro-environment\lib\site-packages\kedro\framework\context\context.py", line 310, in _get_pipeline
    ) from exc
kedro.framework.context.context.KedroContextError: Failed to find the pipeline named '__default__'. It needs to be generated and returned by the 'register_pipelines' function.

My src\dcs_package\pipeline_registry.py looks like this:

# Copyright 2021 QuantumBlack Visual Analytics Limited
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND
# NONINFRINGEMENT. IN NO EVENT WILL THE LICENSOR OR OTHER CONTRIBUTORS
# BE LIABLE FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF, OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
#
# The QuantumBlack Visual Analytics Limited ("QuantumBlack") name and logo
# (either separately or in combination, "QuantumBlack Trademarks") are
# trademarks of QuantumBlack. The License does not grant you any right or
# license to the QuantumBlack Trademarks. You may not use the QuantumBlack
# Trademarks or any confusingly similar mark as a trademark for your product,
# or use the QuantumBlack Trademarks in any other manner that might cause
# confusion in the marketplace, including but not limited to in advertising,
# on websites, or on software.
#
# See the License for the specific language governing permissions and
# limitations under the License.

"""Project pipelines."""
from typing import Dict
from kedro.pipeline import Pipeline, node
from .pipelines.data_processing.pipeline import create_pipeline
import logging

def register_pipelines() -> Dict[str, Pipeline]:
    """Register the project's pipelines.

    Returns:
        A mapping from a pipeline name to a ``Pipeline`` object.
    """
    log = logging.getLogger(__name__)
    log.info("Start register_pipelines") 
    data_processing_pipeline = create_pipeline()
    log.info("create pipeline done") 
    

    return {
        "__default__": data_processing_pipeline,
        "dp": data_processing_pipeline
    }

Then I have a "src\dcs_package\pipelines\data_processing\pipeline.py" file with a real simple function that outputs "test string" and nothing else.

I was able to read a few items from my catalog (a csv and a xlsx) so I think all the dependencies are working fine.


Solution

  • What version of kedro are you on? There is a bit of a problem with kedro 0.17.2 where the true error is masked and will return the exception that you're seeing instead. It's possible that the root cause of the error is actually some other ModuleNotFoundError or AttributeError. Try doing a kedro install before kedro run and see if that fixes it.