How to modify patches made by patchify?

Question

I am using patchify library to patch a big image:

img = cv2.imread("resized.jpg")
patches_img = patchify(img, (224,224,3), step=224)
print(patches_img.shape)

Then I save the patches:

for i in range(patches_img.shape[0]):
    for j in range(patches_img.shape[1]):
        single_patch_img = patches_img[i, j, 0, :, :, :]
        if not cv2.imwrite('patches/images/' + 'image_' + '_'+ str(i)+str(j)+'.jpg', single_patch_img):
            raise Exception("Could not write the image")

Then, I want to make some modification on any of those patches, e.g. draw bounding boxes, so when I use unpatchify to merge patches together, the bounding boxes would be displayed on the reconstructed image.

After making the modifications, I run the following code to merge the patches back together:

reconstructed_image = unpatchify(patches_img, img.shape)
cv2.imwrite("unpatched.jpg", reconstructed_image)

But the reconstructed image generated is the same as the original one, with no change visible. I assume this is because unpatchify reads the variable patches_img, which has still stored the original, unmodified patches.

I tried the following:

patches = 'patches/images/*.jpg'
reconstructed_image = unpatchify(patches, img.shape)
cv2.imwrite("unpatched.jpg", reconstructed_image)

But I am getting AttributeError: 'str' object has no attribute 'shape'

Thanks you!

Answer 1

For reconstructing the image, we have to read the images one by one, and place each image in the original patch position.

There was a bug in the file naming, for example:
i = 1 and j = 11 has the same name as i = 11 and j = 1 ('image__111.jpg').
Better file naming:

cv2.imwrite('patches/images/' + 'image_' + '_'+ str(i).zfill(2) + '_' + str(j).zfill(2) + '.png', single_patch_img)

Note:

• I changed the image file format from JPEG to PNG for keeping the original image quality.
  JPEG is a lossy image format, so every storing and loading we loose some quality.

---

Suggested solution for reconstructing:

• Read test.jpg just for getting the shape (of img):

    img = cv2.imread("test.jpg")
    img = np.zeros_like(img)  # Fill with zeros for the example (start from an empty image).

• Use patchify just for getting the shape (of patches):

    patches = patchify(img, (224,224,3), step=224)  # We could have also used: patches = np.zeros((14, 18, 1, 224, 224, 3), np.uint8)

• Read the images and place them in the original position in patches:

    for i in range(patches.shape[0]):
        for j in range(patches.shape[1]):
            single_patch_img = cv2.imread('patches/images/' + 'image_' + '_'+ str(i).zfill(2) + '_' + str(j).zfill(2) + '.png')  # Read a patch image.
            if single_patch_img is None:
                raise Exception("Could not read the image") 
            patches[i, j, 0, :, :, :] = single_patch_img.copy()  # Copy single path image to patches

• unpatchify

    reconstructed_image = unpatchify(patches, img.shape)

---

Here is a complete code sample that patchify, save patches, load patches, and unpatchify:

import cv2
import numpy as np
from patchify import patchify, unpatchify


img = cv2.imread("test.jpg")
patches_img = patchify(img, (224,224,3), step=224)  # patches_img.shape = (14, 18, 1, 224, 224, 3)

for i in range(patches_img.shape[0]):
    for j in range(patches_img.shape[1]):
        single_patch_img = patches_img[i, j, 0, :, :, :]
        cv2.rectangle(single_patch_img, (30, 30), (224-30, 224-30), (0, 255, 0), 3)  # Draw something (for testing).
        if not cv2.imwrite('patches/images/' + 'image_' + '_'+ str(i).zfill(2) + '_' + str(j).zfill(2) + '.png', single_patch_img):  # Save as PNG, not JPEG for keeping the quality.
            raise Exception("Could not write the image") 

# Store an unpatchified reference for testing
cv2.imwrite("unpatched_ref.jpg", unpatchify(patches_img, img.shape))

# Unpatchify
################################################################################

# Allocate sapces for storing the patches
img = cv2.imread("test.jpg")  # Read test.jpg just for getting the shape
img = np.zeros_like(img)  # Fill with zeros for the example (start from an empty image).

# Use patchify just for getting the size. shape = (14, 18, 1, 224, 224, 3)
# We could have also used: patches = np.zeros((14, 18, 1, 224, 224, 3), np.uint8)
patches = patchify(img, (224,224,3), step=224)

for i in range(patches.shape[0]):
    for j in range(patches.shape[1]):
        single_patch_img = cv2.imread('patches/images/' + 'image_' + '_'+ str(i).zfill(2) + '_' + str(j).zfill(2) + '.png')  # Read a patch image.
        if single_patch_img is None:
            raise Exception("Could not read the image") 
        patches[i, j, 0, :, :, :] = single_patch_img.copy()  # Copy single path image to patches

reconstructed_image = unpatchify(patches, img.shape)

cv2.imwrite("unpatched.jpg", reconstructed_image)

---

Sample output (reduced size):