Vulkan VkBufferImageCopy for partial transfer
In a nutshell, my problem is that when I try to update an image based on a set of dirty rectangles that offset/size of this does not match.
So, let's show the problem.
Here's the object properly rendererd:
This comes from Chromium Embedded Framework and gets properly rendered by updating the entire image - something that is usually not necessary and CEF gives you a list of rectangles that changed and need to be updated.
The full copy is succesfully achieved with:
def copyBuffertoImageRegion(self, topleft, size, fullsize):
print(topleft, size, fullsize)
with CmdBuffer(self.interface, True) as cmdbuffers:
region = VkBufferImageCopy(
bufferOffset=0,
bufferRowLength=fullsize[0],
bufferImageHeight=fullsize[1],
imageSubresource=self.subresource,
imageExtent=size,
imageOffset=topleft
)
vkCmdCopyBufferToImage(
cmdbuffers[0],
self.staging.buffer,
self.image,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
1,
region
)
The print in this case gives (0,0,0) (1920, 1080, 1) (1920, 1080) For a full copy and this works.
However, once I try to use the dirty rects, I print for example these values:
(88, 88, 0) (120, 120, 1) (1920, 1080)
(88, 88, 0) (120, 120, 1) (1920, 1080)
(88, 96, 0) (120, 120, 1) (1920, 1080)
(72, 80, 0) (152, 136, 1) (1920, 1080)
(72, 80, 0) (152, 136, 1) (1920, 1080)
(80, 80, 0) (144, 136, 1) (1920, 1080)
(80, 80, 0) (136, 136, 1) (1920, 1080)
(80, 88, 0) (152, 144, 1) (1920, 1080)
(88, 88, 0) (152, 144, 1) (1920, 1080)
Which, if I understood the VkBufferImageCopy command right, should be correct?
The first tuple of the start of the dirty rect; topleft point. the second tuple is the width, height and depth of the rect and the last tuple is the full size of the image and buffer.
But, it looks like this: https://i.sstatic.net/jBHv7.jpg
The offset is "wrong" - the origin of the graphic jumps around and I'm not sure about the extent.
Any help would be much appreciated.
Edit: More info for possible further optimization:
The incoming data gets handled like this:
def fill(self, pointer, rects):
rect = self.combine_rects(rects)
ffi.memmove(self.mappedhostmemory, pointer, self.buffer.size)
with self.interface.main_lock:
self.image.transitionImageLayout(VK_IMAGE_LAYOUT_UNDEFINED, VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL)
self.image.copyBuffertoImageRegion(*rect,self.interface.resolution)
self.image.transitionImageLayout(VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL)
Pointer being this pointer: https://github.com/cztomczak/cefpython/blob/master/api/PaintBuffer.md#getintpointer
and mappedhostmemory being the staging buffer's mapped memory that I keep mapped.
So, even better would be if I could limit the CEF pointer -> vulkan upload even more and not even grab the entire texture and put it into the buffer, but to actually only grab the dirty parts.
Should it matter; combine_rects takes the list of dirty rects, and makes a big one out of it while also keeping the queue families' granularity in mind.
Edit 2: Thanks to the answers so far, getting closer (thanks!), but not quite there yet: https://i.sstatic.net/1qqUw.jpg
It still jumps around, but at least it isn't data salad anymore. This was achieved by setting
bufferOffset = (topleft[0]*fullsize[0]+topleft[1])*4
In this instance I don't need to make sure it is a multiple of 4, as it (at least on my computer, to be fixed later for general-) is on a image granularity of (8,8,8).
def combine_rects(self, rects):
#start rect is *resolution, 0, 0
left, up, width, height = self.start_rect
for rect in rects:
rleft, rup, rwidth, rheight = rect
left = min(left, rleft)
up = min(up, rup)
width = max(width, rwidth)
height = max(height, rheight)
if width == self.interface.resolution[0] or height == self.interface.resolution[1]:
#issue full copy
return (0, 0, 0), (*self.interface.resolution, 1)
if self.granularity_important:#granularity != (1,1,1)
left = (left // self.granularity_x) * self.granularity_x
up = (up // self.granularity_y) * self.granularity_y
#safety buffer, as we may remove up to granularity-1 texels
width += self.granularity_x
height += self.granularity_y
width = width if width % self.granularity_x == 0 else width + self.granularity_x - width % self.granularity_x
height = height if height % self.granularity_y == 0 else height + self.granularity_y - height % self.granularity_y
return (left, up, 0), (width, height, 1)
In case there is an error in this function, putting that here.
When doing copies between buffers and images, you have two sets of parameters. One describes the location of interest within the image; these are defined by the VkBufferImageCopy::image* parameters. The other describes the location of interest within the buffer; these are defined by the VkBufferImageCopy::buffer* parameters.
imageExtent is important to both, as it describes how much data will be transferred. It does so in the space of the image, but it also affects the region of interest within the buffer.
Buffers, of course, don't contain images; they contain arbitrary data. So the way you describe the data in the buffer is different from how you would with an image.
In a buffer, image data is tightly packed; each pixel is directly adjacent to the next. And each pixel element is stored as defined by its format. The buffer part of the copy region is defined by 3 parameters.
The bufferRowLength is the number of pixels from one row to the next. The bufferImageHeight is the number of rows from one texture layer to the next.
These parameters allow you to do sub-selection from within the buffer. For example, if your buffer logically stores an image that is 100x100, and you only want to copy the first 50x50 pixels, you still provide bufferRowLength/bufferImageHeight values of 100x100.
It's the imageExtent that will prevent it from copying past the 50th pixel in each dimension. imageExtent determines how much data is transferred from/to the VkImage to/from the buffer. So if you set this to 50x50, you'll get what you need.
Note that when I say "first 50x50" pixels, I mean the top-left 50x50. If you want to copy from the top-right 50x50, that's a bit more of a challenge.
bufferOffset allows you to specify the byte offset to the start of image data. And because you can specify the row length separately from the image's extent, you can achieve a transfer from/to the top-right 50x50 by providing a bufferOffset of 50 * the element size. The buffer row length/height will be the same as before.
The equation that maps from image coordinates to buffer byte addresses is as follows:
address of (x,y,z) = region->bufferOffset + (((z * imageHeight) + y) * rowLength + x) * elementSize;
So, if you want to transfer from/to the bottom-left 50x50 of the buffer, you can do this. Set the bufferOffset to be:
elementSize * (50 * rowLength)
For the bottom-right 50x50, you would set bufferOffset to:
elementSize * ((50 * rowLength) + 50)
Note however that bufferOffset is required to be a multiple of 4 (and a multiple of the element's size, if the format is not depth/stencil). So if this were an R8 format, this would not work, since 50 is not a multiple of 4.
This also works well for 3D layer copies. The bufferImageHeight specifies how many rows to skip to get to the next layer.
So, to do what you're interested in, you need the following (note: I don't know Python, so I'm just guessing at the syntax):
bufferOffset = VkDeviceSize(((fullsize[0] * topleft.y) + topleft.x) * elementSize)
region = VkBufferImageCopy(
bufferOffset=bufferOffset ,
bufferRowLength=fullsize[0],
bufferImageHeight=fullsize[1],
imageSubresource=self.subresource,
imageExtent=size,
imageOffset=topleft
)
You'll need to calculate elementSize based on the format in question. Also, the above code only works for 2D copies; for 3D copies, you'll need to take into account topleft.y as well.