I've trained an image model on Ultralytics Hub using yolov8x. I then converted this model to a best.mlpackage file by running yolo export model={model_url} format=coreml nms=true on a linux machine, and then transfered it to a macbook air m2.
When I use this model on an XCode project, there is a preview tab where I can upload photos, and the model also behaves as expected there.
In a swift playground running on this same mac, when I run
guard let imageUrl = Bundle.main.url(forResource: "test_image", withExtension: "jpg") else { return }
guard let output = try? best().prediction(input: bestInput.init(imageAt: imageUrl, iouThreshold: 0.45, confidenceThreshold: 0.25)) else { return }
print(output.confidence)
It generates the same number of object detections (15) as the preview tab, which is also expected.
However, when I run this same code on an iOS device (I've tried on both iPhone and iPad), the predictions are just nonsense, it is detecting 115 objects on arbitrary locations, none of which are correct.
At first I figured this might be a problem with iOS in general, but the model also works when I use the simulator instead of an actual device. I've also tried it on both iOS 17 and 18.
The result is still the same if I use VNImageRequestHandler instead of model.prediction.
The last thing I tried was changing the compute units of the model to use cpu only. This changes the result compared to using neural engine, but it is still gibberish.
One more information that might be useful, Ultralytics has an iOS app where you can load a model that was trained in the hub to your iPhone and it runs live detection. The model behaves wildly in their app too, even tho it works just fine when viewing the model page in a browser, so I'm convinced it has something to do with how iOS runs these models.
Can anyone help me figure this out?
I'm facing the same issue. I trained a YOLO11 model with only one class. The model performed well during training, validation, and testing.
Validating runs/detect/train/weights/best.pt... Ultralytics 8.3.49 🚀 Python-3.9.6 torch-2.2.2 CPU (Intel Core(TM) i3-8100B 3.60GHz) YOLO11n summary (fused): 238 layers, 2,582,347 parameters, 0 gradients, 6.3 GFLOPs Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 3/3 [00:09<00:00, 3.29s/it] all 79 1229 0.955 0.908 0.971 0.861 Speed: 3.1ms preprocess, 111.1ms inference, 0.0ms loss, 0.8ms postprocess per image
If you export the model with format=coreml and nms=true, it generates a .mlpackage file. In Xcode, this allows the preview to analyze images and detect objects correctly. However, it does not work when using model.prediction(input: input) from the class or with VNCoreMLRequest(model: YOLOModel).
To resolve this issue, you need to export the model using format=mlmodel with coremltools==6.2. This ensures compatibility with both methods, and everything works as expected, it's work with the two methods