Search code examples
sprite-kitscenekit

Why the SceneKit Material looks different, even when the image is the same?

The material content support many options to be loaded, two of these are NSImage (or UIImage) and SKTexture.

I noticed when loading the same image file (.png) with different loaders, the material is rendered different.

I'm very sure it is an extra property loaded from SpriteKit transformation, but I don't know what is it.

Why the SceneKit Material looks different, even when the image is the same?

This is the rendered example:

enter image description here

About the code:

let plane = SCNPlane(width: 1, height: 1)
plane.firstMaterial?.diffuse.contents = NSColor.green

let plane = SCNPlane(width: 1, height: 1)
plane.firstMaterial?.diffuse.contents = NSImage(named: "texture")

let plane = SCNPlane(width: 1, height: 1)
plane.firstMaterial?.diffuse.contents = SKTexture(imageNamed: "texture")

The complete example is here: https://github.com/Maetschl/SceneKitExamples/tree/master/MaterialTests

Solution

  • I think this has something to do with color spaces/gamma correction. My guess is that textures loaded via the SKTexture(imageNamed:) initializer aren't properly gamma corrected. You would think this would be documented somewhere, or other people would have noticed, but I can't seem to find anything.

    Here's some code to swap with the last image in your linked sample project. I've force unwrapped as much as possible for brevity:

          // Create the texture using the SKTexture(cgImage:) init 
      // to prove it has the same output image as SKTexture(imageNamed:)
      let originalDogNSImage = NSImage(named: "dog")!
      var originalDogRect = CGRect(x: 0, y: 0, width: originalDogNSImage.size.width, height: originalDogNSImage.size.height)
      let originalDogCGImage = originalDogNSImage.cgImage(forProposedRect: &originalDogRect, context: nil, hints: nil)!
      let originalDogTexture = SKTexture(cgImage: originalDogCGImage)

      // Create the ciImage of the original image to use as the input for the CIFilter 
      let imageData = originalDogNSImage.tiffRepresentation!
      let ciImage = CIImage(data: imageData)
      
      // Create the gamma adjustment Core Image filter
      let gammaFilter = CIFilter(name: "CIGammaAdjust")!
      gammaFilter.setValue(ciImage, forKey: kCIInputImageKey)
      // 0.75 is the default. 2.2 makes the dog image mostly match the NSImage(named:) intializer
      gammaFilter.setValue(2.2, forKey: "inputPower")
      
      // Create a SKTexture using the output of the CIFilter
      let gammaCorrectedDogCIImage = gammaFilter.outputImage!
      let gammaCorrectedDogCGImage = CIContext().createCGImage(gammaCorrectedDogCIImage, from: gammaCorrectedDogCIImage.extent)!
      let gammaCorrectedDogTexture = SKTexture(cgImage: gammaCorrectedDogCGImage)
      
      // Looks bad, like in StackOverflow question image.
//        let planeWithSKTextureDog = planeWith(diffuseContent: originalDogTexture)
      // Looks correct
        let planeWithSKTextureDog = planeWith(diffuseContent: gammaCorrectedDogTexture)

    Using a CIGammaAdjust filter with an inputPower of 2.2 makes the SKTexture almost? match the NSImage(named:) init. I've included the original image being loaded through SKTexture(cgImage:) to rule out any changes caused by using that initializer versus the SKTexture(imageNamed:) you asked about.