ProRes encoding on M1 Max fails for high bit depth buffers

Question

I have code that has worked for many years for writing ProRes files, and it is now failing on the new M1 Max MacBook. Specifically, if I construct buffers with the pixel type "kCVPixelFormatType_64ARGB", after a few frames of writing, the append operation fails with error -12905. This indicates the writing system was unable to do a conversion on this buffer type, which is very odd and unexpected behavior. Again, this code works just fine on non Max processors (Intel and base M1 natively). Here's a sample main that demonstrates the problem. Am I doing something wrong here?

//
//  main.m
//  TestProresWriting
//

#import <Foundation/Foundation.h>
#import <AVFoundation/AVFoundation.h>

int main(int argc, const char * argv[]) {
@autoreleasepool {
    int timescale = 24;
    int width = 1920;
    int height = 1080;
    NSURL *url = [NSURL fileURLWithPath:@"/Users/diftil/TempData/testfile.mov"];
    
    NSLog(@"Output file = %@", [url absoluteURL]);
    NSFileManager *fileManager = [NSFileManager defaultManager];
    NSError *error = nil;
    [fileManager removeItemAtURL:url error:&error];
    
    // Set up the writer
    AVAssetWriter *trackWriter =
        [[AVAssetWriter alloc] initWithURL:url
                                  fileType:AVFileTypeQuickTimeMovie
                                     error:&error];
    
    // Set up the track
    NSDictionary *videoSettings =
             @{AVVideoCodecKey: AVVideoCodecTypeAppleProRes4444,
               AVVideoWidthKey: @(width),
              AVVideoHeightKey: @(height)};
    
    AVAssetWriterInput *track =
        [AVAssetWriterInput assetWriterInputWithMediaType:AVMediaTypeVideo
                                           outputSettings:videoSettings];
    
    // Set up the adapter
    // This pixel type causes problems on M1 Max
    NSDictionary *attributes =
        @{(id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_64ARGB),
          (id)kCVPixelBufferWidthKey: @(width),
          (id)kCVPixelBufferHeightKey: @(height)};
    
    // This pixel type works on M1 Max
    //  NSDictionary *attributes =
    //      @{(id)kCVPixelBufferPixelFormatTypeKey: @(kCVPixelFormatType_32ARGB),
    //        (id)kCVPixelBufferWidthKey: @(width),
    //        (id)kCVPixelBufferHeightKey: @(height)};
    
    AVAssetWriterInputPixelBufferAdaptor *pixelBufferAdaptor =
        [AVAssetWriterInputPixelBufferAdaptor
         assetWriterInputPixelBufferAdaptorWithAssetWriterInput:track
                 sourcePixelBufferAttributes:attributes];
    
    // Add the track and start writing
    [trackWriter addInput:track];
    [trackWriter startWriting];
    
    CMTime startTime = CMTimeMake(0, timescale);
    [trackWriter startSessionAtSourceTime:startTime];
    
    while(!track.readyForMoreMediaData);
    
    CVPixelBufferRef frameBuffer = NULL;
    
    for (int i = 0; i < 100; i++)
    {
        NSLog(@"Frame %d", i);
        CVPixelBufferPoolRef PixelBufferPool = pixelBufferAdaptor.pixelBufferPool;
        if (PixelBufferPool == nil)
        {
            NSLog(@"PixelBufferPool is invalid.");
            exit(1);
        }
        CVReturn ret = CVPixelBufferPoolCreatePixelBuffer(nil, PixelBufferPool,
                                   &frameBuffer);
        if (ret != kCVReturnSuccess)
        {
            NSLog(@"Error creating framebuffer from pool");
            exit(1);
        }
        
        CVPixelBufferLockBaseAddress(frameBuffer, 0);

        void *imagebuf = CVPixelBufferGetBaseAddress(frameBuffer);
        size_t bytesPerRow = CVPixelBufferGetBytesPerRow(frameBuffer);
        memset(imagebuf, 0x00, bytesPerRow * height);

        CVPixelBufferUnlockBaseAddress(frameBuffer, 0);
        
        while(!track.readyForMoreMediaData);
        
        CMTime presentationTime = CMTimeMake(i, timescale);
        BOOL result = [pixelBufferAdaptor appendPixelBuffer:frameBuffer
                                       withPresentationTime:presentationTime];
        if (result == NO)
        {
            NSLog(@"Error appending to track.");
            NSError *error = trackWriter.error;
            if(error!=nil) {
                NSLog(@"Unresolved error %@,%@.", error, [error userInfo]);
            }
            exit(1);
            
        }
        
        CVPixelBufferRelease(frameBuffer);
    }
    
    // Close everything
    if ( trackWriter.status == AVAssetWriterStatusWriting)
        [track markAsFinished];

    __block BOOL isOpen = YES;
    [trackWriter finishWritingWithCompletionHandler:^{
        isOpen = NO;
        NSLog(@"Closed.");
    }];
    while (isOpen);
    
    NSLog(@"Completed.");
}
return 0;
}

Running this code will yield:

2021-11-13 14:11:37.032427-0700 TestProresWriting[56235:2066349] Output file = file:///Users/diftil/TempData/testfile.mov
2021-11-13 14:11:37.054946-0700 TestProresWriting[56235:2066349] Metal API Validation Enabled
2021-11-13 14:11:37.155398-0700 TestProresWriting[56235:2066349] Frame 0
2021-11-13 14:11:37.158434-0700 TestProresWriting[56235:2066349] Frame 1
2021-11-13 14:11:37.158607-0700 TestProresWriting[56235:2066349] Frame 2
2021-11-13 14:11:37.158715-0700 TestProresWriting[56235:2066349] Frame 3
2021-11-13 14:11:37.159803-0700 TestProresWriting[56235:2066349] Frame 4
2021-11-13 14:11:37.172890-0700 TestProresWriting[56235:2066349] Frame 5
2021-11-13 14:11:37.176622-0700 TestProresWriting[56235:2066349] Error appending to track.
2021-11-13 14:11:37.176723-0700 TestProresWriting[56235:2066349] Unresolved error Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12905), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x10602c1e0 {Error Domain=NSOSStatusErrorDomain Code=-12905 "(null)"}},{
    NSLocalizedDescription = "The operation could not be completed";
    NSLocalizedFailureReason = "An unknown error occurred (-12905)";
    NSUnderlyingError = "Error Domain=NSOSStatusErrorDomain Code=-12905 \"(null)\"";
}.
Program ended with exit code: 1

If I change kCVPixelFormatType_64ARGB to kCVPixelFormatType_32ARGB, it runs fine on my MacBook Pro with M1 Max. Again, the original code runs fine on every other computer.

Answer 1

This turns out to have been an Apple bug, which was fixed in MacOS 13 (Ventura). It was related to processing ProRes on Pro and Max M-series silicon. No problems now!