Seeking Guidance on Core ML Coordinate System Discrepancy

[tim-ns97]

Dear otmb,

I hope this message finds you well. I’m an enthusiast learning Core ML and recently came across your shared source code. It’s incredibly well-structured and easy to follow—thank you for making it available!

I’ve been experimenting with my own trained Core ML model using your implementation, but encountered an issue where the bounding box predictions from the Vision framework don’t align with my expected results. It seems the coordinate system might have been transformed, but I couldn’t find explicit documentation on Apple’s official resources to clarify this behavior.

Would you happen to know of any additional tutorials, documentation, or best practices that address coordinate system handling in Vision + Core ML workflows? Alternatively, if you’ve dealt with similar challenges, I’d greatly appreciate any insights you might share.

Thank you in advance for your time and expertise!

Best regards,
tim

[otmb]

Hi.
I'd love to hear about any great documentation from Apple if you have any published.

The coreml coordinate system originates at the lower left.
Please correct it to a common top left origin.

I don't know much about it, but it seems to be a method like this.

guard let observations = request.results as? [VNRecognizedObjectObservation] else { fatalError() }

for observation in observations {
        boundingbox = VNImageRectForNormalizedRect(
                observation.boundingBox.flipped, Int(image.width), Int(image.height))
}

extension CGRect {
  var flipped: CGRect {
    return CGRect(x: origin.x,
                  y: 1 - origin.y - height,
                  width: width, height: height)
  }
}

[tim-ns97]

Dear otmb,

Thank you for your thorough and thoughtful guidance. I truly appreciate the time and effort you’ve taken to explain the coordinate system conversion.

While I’ve already attempted solutions along these lines, the issue unfortunately persists. I’ll continue investigating the problem further—perhaps there’s an overlooked detail in my implementation.

If I successfully resolve it, I’d be happy to document the solution and share it with you as a token of gratitude for your kind assistance.

Warm regards,
tim

[otmb]

I'm sorry I couldn't be of any help.

Referring to transform_bbox , the method I proposed is implemented when using xycwh format.

• xyxy
• xywh
• xycwh

I don't know if this will help you, but I think this might be the problem.

I hope it works.

[tim-ns97]

Dear otmb,

Thank you once again for your previous response. I truly appreciate your insights.

When you originally developed this code, did you happen to reference any specific Core ML documentation or resources? If so, I’d be grateful if you could point me to them—it might provide valuable context for my current implementation.

Your guidance has been incredibly helpful, and any additional references would be greatly appreciated.

Best regards,
tim

[otmb]

I learned about this information from an issue with coremltools.
apple/coremltools#2265

Much of the information seems to be undocumented.

[otmb]

@tim-ns97
I recently wrote such a conversion code for yolo v9.
https://gist.github.com/otmb/a6867efe8a93b82ef650ce3d20f94b21

The code around CoremlPostProcess may be helpful.

Thanks.

[tim-ns97]

Dear otmb,

Thank you very much for sharing. You've helped me identify areas for improvement. My model's output format is xyxy, not xywh—perhaps this is the root of the issue. I need to test it out.

Best regards,
tim

[otmb]

Of course, it can also run on xyxy.
Here is the code that works on xyxy.
https://gist.github.com/otmb/0699625681e9d97cc47cc42e3e8a8a8c