Search code examples
iosswiftmetal

Flipped picture after render in Metal

I'm trying to make animation rendered in MTKView and experiencing a strange issue. In the simulator, I see flipped picture but on a device all is fine. I also render video with the same methods and picture has the same orientation on the device and the simulator.

Here is the most simplified example which reproduces my issue: 

class TestVC: UIViewController {
    var output: MTKView!
    let device: MTLDevice
    let context: CIContext
    let colorSpace = CGColorSpaceCreateDeviceRGB()
    let queue: MTLCommandQueue?
    let image: CIImage

    init() {
        let defaultDevice = MTLCreateSystemDefaultDevice()!
        self.device = defaultDevice
        self.context = CIContext(mtlDevice: defaultDevice)
        self.queue = defaultDevice.makeCommandQueue()
        self.image = CIImage(image: UIImage(named: "lena.png")!)!
        super.init(nibName: nil, bundle: nil)
    }

    required init?(coder: NSCoder) {
        fatalError("init(coder:) has not been implemented")
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = .red
        makeOutputView()
    }

    func makeOutputView() {
        let side = min(view.frame.size.width, view.frame.size.height) * 0.9
        let viewFrame = CGRect(x: 0, y: 0, width: side, height: side)
        output = MTKView(frame: viewFrame, device: device)
        view.addSubview(output)
        output.center = view.center
        output.framebufferOnly = false
        output.delegate = self
    }
}

extension TestVC: MTKViewDelegate {
    func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {}

    func draw(in view: MTKView) {
        let buffer = queue?.makeCommandBuffer()
        guard let drawable = view.currentDrawable else { return }
        let dpi = UIScreen.main.nativeScale
        let width = view.bounds.width * dpi
        let height = view.bounds.height * dpi
        let rect = CGRect(x: 0, y: 0, width: width, height: height)
        let extent = image.extent
        let xScale = extent.width > 0 ? width  / extent.width  : 1
        let yScale = extent.height > 0 ? height / extent.height : 1
        let scale = max(xScale, yScale)
        let tx = (width - extent.width * scale) / 2
        let ty = (height - extent.height * scale) / 2
        let transform = CGAffineTransform(a: scale, b: 0, c: 0, d: scale, tx: tx, ty: ty)
        let filter = CIFilter(name: "CIAffineTransform",
                              parameters: ["inputImage": image, "inputTransform": transform])!
        let scaledImage = filter.outputImage!
        context.render(scaledImage,
                       to: drawable.texture,
                       commandBuffer: buffer,
                       bounds: rect,
                       colorSpace: colorSpace)
        buffer?.present(drawable)
        buffer?.commit()
    }
}

And that's what I see: enter image description here

Solution

  • This issue has to do with the simulator using the GPU on your Mac which uses a different coordinate system. A quick workaround is to flip the image when running it in the simulator.

    var scaledImage = filter.outputImage!
    
#if targetEnvironment(simulator)
    scaledImage = scaledImage.transformed(by: CGAffineTransform(scaleX: 1, y: -1))
        .transformed(by: CGAffineTransform(translationX: 0, y: scaledImage.extent.height))
#endif

context.render(scaledImage,
          to: drawable.texture,
          commandBuffer: buffer,
          bounds: rect,
          colorSpace: colorSpace)
 buffer?.present(drawable)
 buffer?.commit()