Search code examples
swiftshaderarkitmetalmetal-performance-shaders

Chaining Metal Performance Shaders, Error: can not operate in place


I am writing an AR application with which im testing different shaders for a university project. Some of them are already available via the Metal Performance Shaders (right now I am only using Sobel and Gaussian Blur on an image. I also have a couple of custom kernel shaders written. First I do a pass over the original image, i.e. ARFrame in my case, with a shader A to distort it and in the second pass I apply another shader B to try and fix the previous damage again.

Original frame --> A --> B (result)

Plain and simple and it works great with my custom shaders (they aren't too sophisticated yet). Now, when I replace either A or B with an MPS it also works like a charm. But once I replace both A and B with an MPS, the program crashes and I get the following error message:

failed assertion `[MPSImageGaussianBlur encodeToCommandBuffer:sourceTexture:destinationTexture: can not operate in place.'

Since this is an AR application, I am assigning a callback function to PostProcessContext and handle everything else in there. The code looks like below (I obviously stripped some boilerplate out but I hope you get the idea of what I am trying to accomplish).

func initShaders() {
    // makeFunction and obtain computePipelineState
    // load textures
    // initialize buffers   
    
    renderCallbacks.postProcess = { context in            
        for (i, shaderDescriptor) in shaders.enumerated() {


// ======================================================================================== //

            // If the current shader is a MPS we should be able to directly run it
            if shaderDescriptor.shader.type == .metalPerformanceShader {
                let gaussianBlur = MPSImageGaussianBlur(device: context.device, sigma: sigma)
                gaussianBlur.encode(
                    commandBuffer: context.commandBuffer,
                    sourceTexture: context.sourceColorTexture,
                    destinationTexture: context.targetColorTexture
                )
                continue
            }
            
// ======================================================================================== //


// ======================================================================================== //

            // Otherwise use our custom shader with the regular workload
            
            guard let encoder = context.commandBuffer.makeComputeCommandEncoder(descriptor: computePassDescriptor) else {
                continue
            }
            encoder.setComputePipelineState(computePipelineState!)
            encoder.setTexture(context.sourceColorTexture, index: 0)
            encoder.setTexture(context.targetColorTexture, index: 1)
            
            // Assign some arguments
            ...
            encoder.setBuffer(argumentBuffer, offset: MemoryLayout<Float>.size * j, index: j)
            ...

            // Assign some textures
            ...
            encoder.setTexture(self.loadedTextures[name], index: i)
            ...

            // Dispatch the threadgroups
            let threadsPerThreadgroup = MTLSize(...)
            let threadgroupsPerGrid = MTLSize(...)
            encoder.dispatchThreadgroups(threadgroupsPerGrid, threadsPerThreadgroup: threadsPerThreadgroup)

            // end the encoding and proceed with the loop to the second pass
            encoder.endEncoding()

// ======================================================================================== //

        }
        
        // Does not need to be run, p
        // context.commandBuffer.commit()
    }
}

Solution

  • The problem was not within the above code but the way I handled the processing which looks like this:

    renderCallbacks.postProcess = { context in
        var context = context
        let correctionCallback = self.postProcessCallbacks[.correction]
        let simulationCallback = self.postProcessCallbacks[.simulation]
    
        if correctionCallback != nil {
            correctionCallback!!(context)
        }
    
        if simulationCallback != nil {
            simulationCallback!!(context)
        }
    }
    

    Between the two if's I needed to use a temporary, intermediate texture to store everything from the callback and then pass it into the second pass.