Apply a mask to AVCaptureStillImageOutput

Question

I'm working on a project where I'd like to mask a photo that the user has just taken with their camera. The mask is created at a specific aspect ratio to add letterboxes to a photo.

I can successfully create the image, create the mask, and save both to the camera roll, but I can't apply the mask to the image. Here's the code I have now

func takePhoto () {
  dispatch_async(self.sessionQueue) { () -> Void in
    if let photoOutput = self.output as? AVCaptureStillImageOutput {
      photoOutput.captureStillImageAsynchronouslyFromConnection(self.outputConnection) { (imageDataSampleBuffer, err) -> Void in
        if err == nil {
          let imageData = AVCaptureStillImageOutput.jpegStillImageNSDataRepresentation(imageDataSampleBuffer)
          let image = UIImage(data: imageData)

          if let _ = image {
            let maskedImage = self.maskImage(image!)

            print("masked image: \(maskedImage)")

            self.savePhotoToLibrary(maskedImage)
          }
        } else {
          print("Error while capturing the image: \(err)")
        }
      }
    }
  }
}

func maskImage (image: UIImage) -> UIImage {
  let mask = createImageMask(image)

  let maskedImage = CGImageCreateWithMask(image.CGImage, mask!)

  return UIImage(CGImage: maskedImage!)
}

func createImageMask (image: UIImage) -> CGImage? {
  let width = image.size.width
  let height = width / CGFloat(store.state.aspect.rawValue)
  let x = CGFloat(0.0)
  let y = (image.size.height - height) / 2
  let maskRect = CGRectMake(0.0, 0.0, image.size.width, image.size.height)
  let maskContents = CGRectMake(x, y, width, height)

  var color = UIColor(white: 1.0, alpha: 0.0)

  UIGraphicsBeginImageContextWithOptions(CGSizeMake(maskRect.size.width, maskRect.size.height), false, 0.0)

  color.setFill()
  UIRectFill(maskRect)

  color = UIColor(white: 0.0, alpha: 1.0)
  color.setFill()
  UIRectFill(maskContents)

  let maskImage = UIGraphicsGetImageFromCurrentImageContext()

  UIGraphicsEndImageContext()

  print("mask: \(maskImage)")
  savePhotoToLibrary(image)
  savePhotoToLibrary(maskImage)

  let mask = CGImageMaskCreate(
    CGImageGetWidth(maskImage.CGImage),
    CGImageGetHeight(maskImage.CGImage),
    CGImageGetBitsPerComponent(maskImage.CGImage),
    CGImageGetBitsPerPixel(maskImage.CGImage),
    CGImageGetBytesPerRow(maskImage.CGImage),
    CGImageGetDataProvider(maskImage.CGImage),
    nil,
    false)

  return mask
}

From what I understand, CGImageCreateWithMask requires that the image to be masked has an alpha channel. I've tried everything I've seen here to add an alpha channel to the jpeg representation, but I'm not having any luck. Any help would be super.

Answer 1

This may be a bug, or maybe it's just a bit misleading. CGImageCreateWithMask() doesn't actually modify the image - it just associates the mask data with the image data, and uses the mask when you draw the image to a context (such as in a UIImageView), but not when you save the image to disk.

There are a couple approaches to generating a "rendered" version of the masked image, but if I understand your intent, you don't really want a "mask" ... you want a letter-boxed version of the image.

Here is one option that will effectively draw black bars on the top and bottom of your image (the bars / frame color is an optional parameter, if you don't want black). You can then save the modified image.

In your code above, replace

  let maskedImage = self.maskImage(image!)

with

  let height = image.size.width / CGFloat(store.state.aspect.rawValue)
  let maskedImage = self.doLetterBox(image!, visibleHeight: height)

and add this function:

func doLetterBox(sourceImage: UIImage, visibleHeight: CGFloat, frameColor: UIColor?=UIColor.blackColor()) -> UIImage! {

    // local rect based on sourceImage size
    let imageRect: CGRect = CGRectMake(0.0, 0.0, sourceImage.size.width, sourceImage.size.height)

    // rect for "visible" part of letter-boxed image
    let clipRect: CGRect = CGRectMake(0.0, (imageRect.size.height - visibleHeight) / 2.0, imageRect.size.width, visibleHeight)

    // setup the image context, using sourceImage size
    UIGraphicsBeginImageContextWithOptions(imageRect.size, true, UIScreen.mainScreen().scale)

    let ctx: CGContextRef = UIGraphicsGetCurrentContext()!
    CGContextSaveGState(ctx)

    // fill new empty image with frameColor (defaults to black)
    CGContextSetFillColorWithColor(ctx, frameColor?.CGColor)
    CGContextFillRect(ctx, imageRect)

    // set Clipping rectangle to allow drawing only in desired area
    UIRectClip(clipRect)

    // draw the sourceImage to full-image-size (the letter-boxed portion will be clipped)
    sourceImage.drawInRect(imageRect)

    // get new letter-boxed image
    let resultImage: UIImage = UIGraphicsGetImageFromCurrentImageContext()

    // clean up
    CGContextRestoreGState(ctx)
    UIGraphicsEndImageContext()

    return resultImage

}