Search code examples
pythonobject-detection

Persist/retrieve evaluation results in FiftyOne

I am using the amazing FiftyOne to analyze machine learning model evaluations.

I have an existing (persistent) dataset. In a previous session, I ran an evaluation on it:

python
>>> import fiftyone as fo
>>> dataset = fo.load_dataset("existing_dataset")
>>> dataset.list_evaluations()
[eval]

I would like to retrieve the results object associated with this evaluation without running the dataset.evaluate_detections method.

We have multiple teams are will analyze these results, and we need to ensure that all teams have the same data/values. It also would be convenient to access results without waiting for them to be generated.

What I Tried

  1. The command dataset.get_evaluation_info("eval") returns high level information about the evaluation, but not the results themselves.

  2. When I generated the results, doing a pickle on the results object gives a PicklingError associated with the mongoengine.base.metaclasses.

Solution

  • I'm glad you like the tool!

    What you're looking for is the dataset.load_evaluation_results("eval") method. It will return the results object even after persisting and reloading a dataset.

    get_evaluation_info() is used to access the info and arguments used to create the evaluation. Also just FYI, load_evaluation_view() can be used to load the exact View that was evaluated.

    Accessing results while they are being generated could be quite useful. Please feel free to create a feature request on GitHub: https://github.com/voxel51/fiftyone/issues