Search code examples
prefectgreat-expectations

RunGreatExpectationsValidation execution returns an exception

I am struggling on a great_expectations integration problem.
I obviously use RunGreatExpectationsValidation task with:

validation_task = RunGreatExpectationsValidation()
with Flow(
    "GE_pull_and_run",
) as GE_pull_and_run_flow:
    .......
    validation_task(
        context_root_dir=root_dir,
        checkpoint_name=expectation_checkpoint_name
    )

When I run the command on GE (great_expectations --V3-api checkpoint run my_checkpoint), it works, but on prefect task, I have an exception: With GE V3 api:

    .....
   File "c:\Users\vincent2\DK\prefect.data.pipeline\venv\lib\site- 
   packages\prefect\tasks\great_expectations\checkpoints.py", line 246, in run
        for batch in ge_checkpoint["batches"]:
    TypeError: 'Checkpoint' object is not subscriptable

The same with GE V2 api

    ...
        for batch in ge_checkpoint["batches"]:
    TypeError: 'LegacyCheckpoint' object is not subscriptable

Great_expectations=0.13.43 (also tried with 0.12.10 version)
prefect=0.15.9
Anyone experienced this pb? Thanks

Solution

  • To provide an update on this: the issue has been fixed as part of this PR in Prefect. Feel free to give it a try now and if something still doesn't work for you, let us know.