CGImage to MPSTexture or MPSImage

Question

I have an CGImage which is constructed out of a CVPixelbuffer (ARGB). I want to convert that CGImage into a MTLTexture. I use:

  let texture: MTLTexture = try m_textureLoader.newTexture(with: cgImage, options: [MTKTextureLoaderOptionSRGB : NSNumber(value: true)] )

Later I want to use the texture in an MPSImage having 3 channels:

let sid   = MPSImageDescriptor(channelFormat: MPSImageFeatureChannelFormat.float16, width: 40, height: 40, featureChannels: 3)
preImage    = MPSTemporaryImage(commandBuffer:  commandBuffer, imageDescriptor: sid)
lanczos.encode(commandBuffer: commandBuffer, sourceTexture: texture!, destinationTexture: preImage.texture)
scale.encode  (commandBuffer: commandBuffer, sourceImage: preImage, destinationImage: srcImage)

Now my questions: How does textureLoader.newTexture(...) map the four ARGB channels to the 3 channels specified in the MPSImageDescriptor ? How can I ensure that the RGB components are used and not e.g. ARG ? Is there a way to specify that channel mapping ?

Thanks, Chris

Answer 1

Why not construct the MTLTexture from the CVPixelBuffer directly? Is much quicker!

Do this once at the beginning of your program:

// declare this somewhere, so we can re-use it
var textureCache: CVMetalTextureCache?

// create the texture cache object
guard CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, device, nil, &textureCache) == kCVReturnSuccess else {
  print("Error: could not create a texture cache")
  return false
}

Do this once your have your CVPixelBuffer:

let width = CVPixelBufferGetWidth(pixelBuffer)
let height = CVPixelBufferGetHeight(pixelBuffer)

var texture: CVMetalTexture?
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache,
      pixelBuffer, nil, .bgra8Unorm, width, height, 0, &texture)

if let texture = texture {
  metalTexture = CVMetalTextureGetTexture(texture)
}

Now metalTexture contains an MTLTexture object with the contents of the CVPixelBuffer.