Using skimage view_as_windows to make image patches and reconstruct patches

I want to extract color image patches from a 512x512 color image, and save them as individual image patches in a folder. How can I reconstruct my original image from these image patches? I have read and looked at afew of the similar questions and they do not solve my problem.

I have done abit of reading up and decided to use the view_as_windows function from SKimage to carry out my image patching. I also managed to save my patches to png files.

Currently using SKimage view_as_window to extract patches from a color image sized 512x512 patch_img = view_as_windows(input_img, (128, 128, 3), step=64) when revealing the details of the output array, I noticed that patch_img has a shape of (7, 7, 1, 128, 128, 3) and a dtype of unint8. To save each patch as an individual image I use the following code.

    for i in range(0, len(patch_img)):    #range should be 0 to 6
        for x in range(0, len(patch_img)):
            fname= 'IMG_test_{}_{}.png'.format(i, x)
            #cv2.imwrite(fname, crop_img[i,x,0,:,:,:])

When loading the entire folder with the saved images using CV2, I am unable to get back the same shape and dtype of patch_img, instead, I get a shape (49, 128, 128, 3). How can I fix this.

Edit: Fixed the shape using savedimg = savedimg.reshape(7,7,128 128, 3)

Also, how can I then use the saved image patches to reconstruct the original image?

Solution

Let's assume we're working on something a bit simpler first. Let's say that instead of a 2D RGB array, we want to split a 2D array of numbers, like this:

>>> image_arr = np.array(list(range(1, 26))).reshape((5,5))
>>> image_arr
array([[ 1.,  2.,  3.,  4.,  5.],
       [ 6.,  7.,  8.,  9., 10.],
       [11., 12., 13., 14., 15.],
       [16., 17., 18., 19., 20.],
       [21., 22., 23., 24., 25.]])

Now, let's say we want to split it to 2x2 windows:

>>> patch_arr = view_as_windows(image_arr, (2,2))

Let's compare the shapes of the two arrays:

>>> image_arr.shape
(5, 5)
>>> patch_arr.shape
(4, 4, 2, 2)

Now, (if I understood you correctly) you're asking how we could reconstruct the image_arr using the patch_arr?

The way we're going to approach it, is we're going to create an empty np.array and then we will be taking each of the 'patches' and pasting them on the image. Since they are overlapping, it means we will be writing the same values to most of the cells multiple times, but of course it is not an issue.

You could also try to optimize this approach to write to each cell only once, but I'm not sure it's worth it in this case.

Let's create the empty array

>>> reconstructed_arr = np.zeros(shape=(5,5))
>>> reconstructed_arr
array([[0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.]])

We will now iterate over the patches and paste them to the reconstructed_arr:

for x in range(patch_arr.shape[0]):
    for y in range(patch_arr.shape[1]):
        reconstructed_arr[x:x + 2, y:y + 2] = patch_arr[x,y]

That's it

>>> reconstructed_arr
array([[ 1.,  2.,  3.,  4.,  5.],
       [ 6.,  7.,  8.,  9., 10.],
       [11., 12., 13., 14., 15.],
       [16., 17., 18., 19., 20.],
       [21., 22., 23., 24., 25.]])

The similar approach needs to be applied to your data (this time with another axis for the RGB values):

Generating a random input_img array

input_img = np.random.rand(512, 512, 3)

Let's create the patches as you did:

patch_img = view_as_windows(input_img, (128, 128, 3), step=64)

The empty array for reconstruction:

>>> reconstructed_arr = np.zeros((512, 512, 3))

The reconstructing for loops with some small adjustments, since you used step=64.

>>> step = 64
>>> for x in range(patch_img.shape[0]):
        for y in range(patch_img.shape[1]):
            x_pos, y_pos = x * step, y * step
            reconstructed_arr[x_pos:x_pos + 128, y_pos:y_pos + 128] = patch_img[x, y, 0, ...]
>>> (input_img == reconstructed_arr).all()
True