How to display very large Metal-based textures in an NSScrollView?

Question

I have a program that displays an image inside an NSScrollView. I'm using Metal to render the image, so the NSClipView's documentView is a MTKView. I support the ability to zoom in/out of the image. The last steps of my render pipeline wrap my Metal texture into a CIImage where I apply the CIAffineTransform filter to do the scaling. I render the image in drawRect into a drawable like this:

CIImage *image = ...;
CIRenderDestination *renderDest = [[CIRenderDestination alloc] initWithWidth:image.extent.size.width
                                                                      height:image.extent.size.height
                                                                 pixelFormat:self.colorPixelFormat
                                                               commandBuffer:commandBuffer
                                                          mtlTextureProvider:^id<MTLTexture> _Nonnull{
    return drawable.texture;
}];

[self.ciContext startTaskToRender:image
                    toDestination:renderDest
                            error:&error];

When the document view size changes I update the view's drawableSize property. This works great and the scrolling and scroll bars work as expected. The problem is when the magnification factor gets too large the drawable's underlying texture exceeds the GPU memory:

validateTextureDimensions, line 1081: error 'MTLTextureDescriptor has width (18798) greater than the maximum allowed size of 16384.'
validateTextureDimensions:1081: failed assertion `MTLTextureDescriptor has width (18798) greater than the maximum allowed size of 16384.'

so drawing a cropped image into part of the drawable won't work (not that I know how to do that from a CIImage). This happens even if I don't draw any image into the view (i.e. just set the expected drawable size) which suggests the MTKView's drawable is the problem. Of course, I don't need to draw the full image; just the part that's visible in the clip view. I could restrict the drawableSize/document view, but then I'd have to effectively implement scrolling manually and lose the scroll bars.

What is the best path for a standard scroll view while still displaying extremely large images? Alternately, is there a path to use something like CATiledLayer with Metal?

Answer 1

I received a suggestion from an Apple engineer on the developer forums which was to place an MTKView inside the document view of the NSScrollView, i.e.:

NSScrollView
  NSClipView
    NSView <-- document view
      MTKView

This makes sense since Metal and AppKit don't really talk to one another. With this scheme one can manipulate the document view size to get the scroll bars to reflect correctly and manually set the MTKView to be no larger than what is visible in the clip view. Not the ideal solution since there is a lot of NSRect-math involved, but certainly doable. (I haven't had time to implement it yet but will update this answer with any useful information if it comes up.)

The reply on the forum suggested looking at the code that synchronizes two scroll views as a good starting point.