Using custom CIFilter on CALayer shows no change to CALayer

Question

We are trying to create a custom CIFilter to add on top of our CALayer's. How ever only the default CIFilters seem to work on a CALayer.

We created a small new project on the ViewController.swift we added:

import Cocoa
import CoreImage

class ViewController: NSViewController {

    override func viewDidLoad() {
        super.viewDidLoad()
        
        // Create some layers to work with! (square with gradient color)
        let mainLayer = CALayer()
        let shapeLayer = CAShapeLayer()
        let gradientLayer = CAGradientLayer()
        gradientLayer.colors = [NSColor.red.cgColor, NSColor.white.cgColor, NSColor.yellow.cgColor, NSColor.black.cgColor]
        
        shapeLayer.path = CGPath(rect: CGRect(x: 0, y: 0, width: 500, height: 500), transform: nil)
        shapeLayer.fillColor = CGColor.black
        
        gradientLayer.frame = CGRect(x: 0, y: 0, width: 500, height: 500)
        gradientLayer.mask = shapeLayer
        
        gradientLayer.setAffineTransform(CGAffineTransform(translationX: 50, y: 50))
        mainLayer.addSublayer(gradientLayer)
        
        mainLayer.filters = []
        self.view.layer?.addSublayer(mainLayer)

        // Register the custom filter
        CustomFilterRegister.register()
        
        // Test with a normal image file, WORKS!
//      if let image = NSImage(named: "test"), let cgImage = image.cgImage(forProposedRect: nil, context: nil, hints: nil) {
//          if let filter = CIFilter(name: "CustomFilter") {
//              filter.setValue(CIImage(cgImage: cgImage), forKey: kCIInputImageKey)
//              let output = filter.outputImage
//              // WORKS! Image filtered as expected!
//          }
//      }
        
        // Does NOT work. No change in color of the layer!
        if let filter = CIFilter(name: "CustomFilter") {
            filter.name = "custom"
            mainLayer.filters?.append(filter)
        }

        // This works: mainLayer and sublayers are blurred!
//      if let filter = CIFilter(name: "CIGaussianBlur") {
//          filter.name = "blur"
//          mainLayer.filters?.append(filter)
//      }


    }
}
}

We created a simple custom CIFilter to give it a first try before we start building our custom CIFilter.

class CustomFilter: CIFilter {
    
    // Error in xcode if you don't add this in!
    override class var supportsSecureCoding: Bool {
        return true
    }
        
    @objc dynamic var inputImage: CIImage?
    @objc dynamic var inputSaturation: CGFloat = 1
    @objc dynamic var inputBrightness: CGFloat = 0
    @objc dynamic var inputContrast: CGFloat = 1
    override func setDefaults() {
        inputSaturation = 1
        inputBrightness = 0
        inputContrast = 2
    }
    
    override public var outputImage: CIImage? {
        guard let image = inputImage else {
            return nil
        }
        return image.applyingFilter("CIPhotoEffectProcess")
            .applyingFilter("CIColorControls", parameters: [
                kCIInputSaturationKey: inputSaturation,
                kCIInputBrightnessKey: inputBrightness,
                kCIInputContrastKey: inputContrast
            ])
    }
}

class CustomFilterRegister: CIFilterConstructor {
    static func register() {
        CIFilter.registerName(
            "CustomFilter", constructor: CustomFilterRegister(),
            classAttributes: [
                kCIAttributeFilterCategories: [kCICategoryBlur, kCICategoryVideo, kCICategoryStillImage]
            ])
    }
    func filter(withName name: String) -> CIFilter? {
        switch name {
        case "CustomFilter":
            return CustomFilter()
        default:
            return nil
        }
    }
}

In the ViewController we added code to test with a normal image. This DOES work so the filter seems to be ok. We also tried a default CIGaussianBlur and that does work on the CALayer.

We are lost as to what is needed to get a custom CIFilter working with CALayer, and can't seem to find any information on it.

Please note that we are NOT looking for this type of CIFilter or a different way to get the filters result. We need a custom CIFilter to work on a CALayer.

Answer 1

As @DonMag points out it should have worked with the changes he described. How ever unfortunately we heard back from Apple today;

At this time, there is a bug preventing custom CIFilters on a CALayer from working. There is no workaround for this bug at this time.

When we file the bug I will add the link here for those interested. But at this time you can not add a custom CIFilter to a CALayer on macOS 11.

Let’s hope they fix it for all of you reading this for a solution.

---

EDIT:

So bad news... currently on macOS 12.2.1, and it still has the same issue, nothing has happened based on our ticket. Doesn't seem like Apple want's to fix this. For those of you out there looking: This still does NOT work on a CALayer even with all the options on like described in the other answers. A builtin CIFilter works as expected.

Note that using the same custom CIFilter on a CALayer for an export using AVVideoCompositionCoreAnimationTool does work!