Search code examples
pythonmachine-learningpersistenceneuraxle

Error loading neuraxle pipeline with execution context

When I save a pipeline, which has an ExecutionContext associated with it, and try to load it again, I get the error shown below.

from neuraxle.base import ExecutionContext, Identity
from neuraxle.pipeline import Pipeline

PIPELINE_NAME = 'saved_pipeline_name'
cache_folder = 'cache_folder'

pipeline = Pipeline([
    Identity()
]).with_context(ExecutionContext(cache_folder))

pipeline.set_name(PIPELINE_NAME).save(ExecutionContext(cache_folder), full_dump=True)
loaded_pipeline = ExecutionContext(cache_folder).load(PIPELINE_NAME)

Error message:

Traceback (most recent call last):
  File "save_example.py", line 12, in <module>
    loaded_pipeline = ExecutionContext(cache_folder).load(PIPELINE_NAME)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 555, in load
    ).load(context_for_loading, True)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 3621, in load
    return loaded_self.load(context, full_dump)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 1708, in load
    return self._load_step(context, savers)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 1717, in _load_step
    loaded_self = saver.load_step(loaded_self, context)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 3644, in load_step
    step.apply('_assert_has_services', context=context)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 2316, in apply
    results: RecursiveDict = self._apply_childrens(results=results, method=method, ra=ra)
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 2327, in _apply_childrens
    for children in self.get_children():
  File ".env/lib/python3.7/site-packages/neuraxle/base.py", line 2530, in get_children
    return [self.wrapped]
AttributeError: 'StepWithContext' object has no attribute 'wrapped'

Without the with_context(ExecutionContext(cache_folder)) the loading works fine. Is this expected behaviour, or is it a bug? What would be the best practice for saving pipelines, when working with execution contexts?

Solution

  • There was an erroneous call to a function in StepWithContext's saver. There will be a hotfix pushed on Neuraxle's main repository in the next day or so. If you can wait until then your code should execute with no problems.

    If not, I'd suggest you to bypass StepWithContext by calling the save directly on StepWithContext's wrapped step (i.e. your pipeline instance):

    pipeline.wrapped.set_name(PIPELINE_NAME).save(ExecutionContext(cache_folder), full_dump=True)
loaded_pipeline = ExecutionContext(cache_folder).load(PIPELINE_NAME)

    You'll then have to re-wrap the loaded_pipeline instance with a StepWithContext using the .with_context() call.

    When the hotfix will be available, keep in my mind that ExecutionContext instances are not getting saved at all and that, on loading, StepWithContext's context attribute is getting replace with whatever context is used for the loading.

    Feel free to ask me any other questions! I'll be glad to answer them.

    Cheers