Why are sampled colors from a CGImage different depending on whether it's a thumbnail?

Question

I have some code that samples pixels in a CGImage. The image that I'm sampling is entirely red. When I sample the full-size image, the resulting colors are all shades of red, but if I sample a thumbnail created from the same URL, the resulting colors are all shades of yellow. I'm pulling all of the constants from the image itself, so really at a loss here.

 func analayze(imageAtURL url: URL) async throws -> [Color: Int] {
        guard let imgSource = CGImageSourceCreateWithURL(url as CFURL, nil) else {
            throw Error.cantLoadImage(url)
        }
        
        let cgImage: CGImage
        switch optimization {
        case .fullSize:
            guard let img = CGImageSourceCreateImageAtIndex(imgSource, 0, nil) else {
                throw Error.cantInstantiateImage(url)
            }
            cgImage = img
        case let .scale(maxSide):
            let options: [CFString: Any] = [
                kCGImageSourceCreateThumbnailFromImageAlways: true as CFBoolean,
                kCGImageSourceThumbnailMaxPixelSize: maxSide as CFNumber
            ]
            guard let img = CGImageSourceCreateThumbnailAtIndex(imgSource, 0, options as CFDictionary) else {
                throw Error.cantInstantiateImage(url)
            }
            cgImage = img
        default:
            throw Error.unsupportedOptimization(optimization)
        }


        let width = cgImage.width
        let height = cgImage.height

        let bytesPerRow = cgImage.bytesPerRow
        let bytesPerPixel = bytesPerRow / width
        let bitsPerComponent = cgImage.bitsPerComponent
        assert(bitsPerComponent == 8, "Expected 8 bits per component, got \(bitsPerComponent)")

        guard let pixelData = cgImage.dataProvider?.data else {
            throw Error.contextHasNoData
        }

        guard let unsafePointer = CFDataGetBytePtr(pixelData) else {
            throw Error.cantCreatePointer
        }

        let unsafeRawPointer = UnsafeRawPointer(unsafePointer)

        let points = pointsInRect(width: width, height: height)

        @Sendable
        func findClosestColorInTable(for point: DiscretePoint) -> Color? {
            let offset = (point.y * bytesPerRow) + (point.x * bytesPerPixel)
            let red = CGFloat(unsafeRawPointer.load(fromByteOffset: offset, as: UInt8.self)) / 255
            let green = CGFloat(unsafeRawPointer.load(fromByteOffset: offset + 1, as: UInt8.self)) / 255
            let blue = CGFloat(unsafeRawPointer.load(fromByteOffset: offset + 2, as: UInt8.self)) / 255
            let imgColor = Color(r: red, g: green, b: blue)
//            let nearestNeighbor = self.colorTree.nearest(to: imgColor)
//            return nearestNeighbor
            return imgColor
        }

Here's the image and results for the full-size approach:

And for the scaled approach:

Answer 1

As @HangarRash notes, CGImageSourceCreateThumbnailAtIndex doesn't promise it'll return RGB(A) data with a linear RGB colorspace, which is what you seem to assume here. While you could dig into all the details of the data format, if performance is not absolutely critical, I would recommend just re-drawing the thumbnail into a CGContext that has the pixel layout and colorspace you want. That will be much simpler and let CoreGraphics do the work for you.

If you need to avoid an extra round-trip, see HangarRash's comment about where to look for the information, though you'll likely also need to check colorspace to be certain you're using the right conversion. This gets complicated, so I only recommend it if you absolutely need it. Drawing into a context is going to be much easier to keep consistent.