Search code examples
ioscore-imagemetalkit

[[stitchable]] Metal core image CIKernel fails

It looks like [[stitchable]] Metal Core Image kernels fails to either link on compile time due to Core Image sampler.coord() and sampler.sample(coord) symbols not found, OR, fails to load at run time if I add other Metal linker flags -fcikernel (which is not anyways required for stitchable kernels).

Here is the full link of test project on github. Feel free to download it and play around.

I found the main culprit is this line in Metal file which linker can not resolve:

 float2 srcCoord = inputImage.coord();

Excerpts from my code:

import CoreImage

class FilterTwo: CIFilter {
var inputImage: CIImage?
var inputParam: Float = 0.0

static var kernel: CIKernel = { () -> CIKernel in
   let url = Bundle.main.url(forResource: "default",
                            withExtension: "metallib")!
   let data = try! Data(contentsOf: url)

    let kernelNames = CIKernel.kernelNames(fromMetalLibraryData: data)
    NSLog("Kernels \(kernelNames)")
   return try! CIKernel(functionName: "secondFilter", fromMetalLibraryData: data) //<-- This fails!
}()

override var outputImage : CIImage? {
    guard let inputImage = inputImage else {
        return nil
    }
    
    return FilterTwo.kernel.apply(extent: inputImage.extent, roiCallback: { (index, rect) in
        return rect }, arguments: [inputImage])
}

}

Here is the Metal file:

 #include <CoreImage/CoreImage.h> // includes CIKernelMetalLib.h

 using namespace metal;

 [[ stitchable ]] half4 secondFilter (coreimage::sampler inputImage, coreimage::destination dest)
{
   float2 srcCoord = inputImage.coord();
   half4 color =  half4(inputImage.sample(srcCoord));

   return color;
}

And here is the usage:

       class ViewController: UIViewController {

override func viewDidLoad() {
    super.viewDidLoad()
    // Do any additional setup after loading the view.
    
    let filter = FilterTwo()
    filter.inputImage = CIImage(color: CIColor.red)
    let outputImage = filter.outputImage!
    
    NSLog("Output \(outputImage)")
}

}

And the output:

     StitchableKernelsTesting/FilterTwo.swift:15: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=CIKernel Code=1 "(null)" UserInfo={CINonLocalizedDescriptionKey=Function does not exist in library data. …•∆}
   Kernels []

   reflect Function 'secondFilter' does not exist.
Solution

  • Though the build process is much easier with [[ stitchable ]] CI kernels, you still need to tell the Metal linker to link against Core Image. This was mentioned in the WWDC21 session about the topic.

    To do so, you need to add the -framework CoreImage flag to Other Metal Linker Flags as below. Note that you need to add it in two separate lines, otherwise it won't work.

    enter image description here