Scaling Images: how can the accelerate be the slowest method?

Question

I am testing several methods to rescale a UIImage.

I have tested all these methods posted here and measured the time they take to resize an image.

1) UIGraphicsBeginImageContextWithOptions & UIImage -drawInRect:

let image = UIImage(contentsOfFile: self.URL.path!)

let size = CGSizeApplyAffineTransform(image.size, CGAffineTransformMakeScale(0.5, 0.5))
let hasAlpha = false
let scale: CGFloat = 0.0 // Automatically use scale factor of main screen

UIGraphicsBeginImageContextWithOptions(size, !hasAlpha, scale)
image.drawInRect(CGRect(origin: CGPointZero, size: size))

let scaledImage = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()

2) CGBitmapContextCreate & CGContextDrawImage

let cgImage = UIImage(contentsOfFile: self.URL.path!).CGImage

let width = CGImageGetWidth(cgImage) / 2
let height = CGImageGetHeight(cgImage) / 2
let bitsPerComponent = CGImageGetBitsPerComponent(cgImage)
let bytesPerRow = CGImageGetBytesPerRow(cgImage)
let colorSpace = CGImageGetColorSpace(cgImage)
let bitmapInfo = CGImageGetBitmapInfo(cgImage)

let context = CGBitmapContextCreate(nil, width, height, bitsPerComponent, bytesPerRow, colorSpace, bitmapInfo.rawValue)

CGContextSetInterpolationQuality(context, kCGInterpolationHigh)

CGContextDrawImage(context, CGRect(origin: CGPointZero, size: CGSize(width: CGFloat(width), height: CGFloat(height))), cgImage)

let scaledImage = CGBitmapContextCreateImage(context).flatMap { UIImage(CGImage: $0) }

3) CGImageSourceCreateThumbnailAtIndex

import ImageIO

if let imageSource = CGImageSourceCreateWithURL(self.URL, nil) {
    let options: [NSString: NSObject] = [
        kCGImageSourceThumbnailMaxPixelSize: max(size.width, size.height) / 2.0,
        kCGImageSourceCreateThumbnailFromImageAlways: true
    ]

    let scaledImage = CGImageSourceCreateThumbnailAtIndex(imageSource, 0, options).flatMap { UIImage(CGImage: $0) }
}

4) Lanczos Resampling with Core Image

let image = CIImage(contentsOfURL: self.URL)

let filter = CIFilter(name: "CILanczosScaleTransform")!
filter.setValue(image, forKey: "inputImage")
filter.setValue(0.5, forKey: "inputScale")
filter.setValue(1.0, forKey: "inputAspectRatio")
let outputImage = filter.valueForKey("outputImage") as! CIImage

let context = CIContext(options: [kCIContextUseSoftwareRenderer: false])
let scaledImage = UIImage(CGImage: self.context.createCGImage(outputImage, fromRect: outputImage.extent()))

5) vImage in Accelerate

let cgImage = UIImage(contentsOfFile: self.URL.path!).CGImage

// create a source buffer
var format = vImage_CGImageFormat(bitsPerComponent: 8, bitsPerPixel: 32, colorSpace: nil, 
    bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.First.rawValue), 
    version: 0, decode: nil, renderingIntent: CGColorRenderingIntent.RenderingIntentDefault)
var sourceBuffer = vImage_Buffer()
defer {
    sourceBuffer.data.dealloc(Int(sourceBuffer.height) * Int(sourceBuffer.height) * 4)
}

var error = vImageBuffer_InitWithCGImage(&sourceBuffer, &format, nil, cgImage, numericCast(kvImageNoFlags))
guard error == kvImageNoError else { return nil }

// create a destination buffer
let scale = UIScreen.mainScreen().scale
let destWidth = Int(image.size.width * 0.5 * scale)
let destHeight = Int(image.size.height * 0.5 * scale)
let bytesPerPixel = CGImageGetBitsPerPixel(image.CGImage) / 8
let destBytesPerRow = destWidth * bytesPerPixel
let destData = UnsafeMutablePointer<UInt8>.alloc(destHeight * destBytesPerRow)
defer {
    destData.dealloc(destHeight * destBytesPerRow)
}
var destBuffer = vImage_Buffer(data: destData, height: vImagePixelCount(destHeight), width: vImagePixelCount(destWidth), rowBytes: destBytesPerRow)

// scale the image
error = vImageScale_ARGB8888(&sourceBuffer, &destBuffer, nil, numericCast(kvImageHighQualityResampling))
guard error == kvImageNoError else { return nil }

// create a CGImage from vImage_Buffer
let destCGImage = vImageCreateCGImageFromBuffer(&destBuffer, &format, nil, nil, numericCast(kvImageNoFlags), &error)?.takeRetainedValue()
guard error == kvImageNoError else { return nil }

// create a UIImage
let scaledImage = destCGImage.flatMap { UIImage(CGImage: $0, scale: 0.0, orientation: image.imageOrientation) }

After testing this for hours and measure the time every method took for rescaling the images to 100x100, my conclusions are completely different from NSHipster. First of all the vImage in accelerate is 200 times slower than the first method, that in my opinion is the poor cousin of the other ones. The core image method is also slow. But I am intrigued how method #1 can smash methods 3, 4 and 5, some of them in theory process stuff on the GPU.

Method #3 for example, took 2 seconds to resize a 1024x1024 image to 100x100. On the other hand #1 took 0.01 seconds!

Am I missing something?

Something must be wrong or Apple would not take time to write accelerate and CIImage stuff.

NOTE: I am measuring the time from the time the image is already loaded on a variable to the time a scaled version is saved to another variable. I am not considering the time it takes to read from the file.

Answer 1

Accelerate can be the slowest method for a variety of reasons:

1. The code you show may spend a lot of time just extracting the data from the CGImage and making a new image. You didn't, for example, use any features that would allow the CGImage to use your vImage result directly rather than make a copy. Possibly a colorspace conversion was also required as part of some of those extract / create CGImage operations. Hard to tell from here.
2. Some of the other methods may not have done anything, deferring the work until later when absolutely forced to do it. If that was after your end time, then the work wasn't measured.
3. Some of the other methods have the advantage of being able to directly use the contents of the image without having to make a copy first.
4. Different resampling methods (e.g. Bilinear vs. Lanczos) have different cost
5. The GPU can actually be faster at some stuff, and resampling is one of the tasks it is specially optimized to do. On the flip side, random data access (such as occurs in resampling) is not a nice thing to do to the vector unit.
6. Timing methods can impact the result. Accelerate is multithreaded. If you use wall clock time, you will get one answer. If you use getrusage or a sampler, you'll get another.

If you really think Accelerate is way off the mark here, file a bug. I certainly would check with Instruments Time Profile that you are spending the majority of your time in vImageScale in your benchmark loop before doing so, though.