How can i add .png images modified with Pillow to an OpenCV Video?

Question

I'm new here and i want to solve a problem that makes me sick this past weeks...

This is the problem, i have 2 codes, one that grab a video of a directory (it could be a web camera to) with OpenCV and read frame by frame drawing text on each frame, plotting one variable (just for testing), show me the video and save it in the python files directory, this is the code.... ("MyPath" is the path of the file that is irrelevant for the question)

import cv2
import random

capture = cv2.VideoCapture(r"MyPath\Car.mp4")
out = cv2.VideoWriter('Test.avi',cv2.VideoWriter_fourcc('M','P','4','2'), 25, (1280, 720))

def velocity():
    now = (random.randint(0,100))
    return now

while True:
    ret, frame = capture.read()
    if ret:
        font=cv2.FONT_HERSHEY_SIMPLEX
        cv2.putText(frame,'Velocity:',(15,580),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Distance:',(15,620),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Inclination:',(15,660),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Orientation:',(15,700),font,0.7,(255,255,255),1)
        cv2.putText(frame, str(velocity()), (130,580),font,0.7,(255,255,255),1)
        cv2.putText(frame,'KM/H',(165,580),font,0.7,(255,255,255),1)
        out.write(frame)
        cv2.imshow("Testing", frame)

    else:
        break

capture.release()
out.release()
cv2.destroyAllWindows()

This works fine, no problems here, and i have another code with pillow that open a background image (jpg) and 4 images in png that resize and repositions them, then the code paste the images above the background, (i made a background just for testing, this 4 png's images have to plot into the videoframes but now it's just testing...) then show me and save the background with the png's above... and again, it works perfectly!

from PIL import Image

back = Image.open(r"MyPath\Eagle.jpg")
vel = Image.open(r"MyPath\Velocímeter.png")
dis = Image.open(r"MyPath\Distance.png")
inp = Image.open(r"MyPath\Inclination.png")
orz = Image.open(r"MyPath\Orientation.png")

vel = vel.resize((60, 60), Image.LANCZOS)
dis = dis.resize((55, 55), Image.LANCZOS)
inp = inp.resize((60, 60), Image.LANCZOS)
orz = orz.resize((60, 60), Image.LANCZOS)

back.paste(vel, (10, 420), vel)
back.paste(dis, (10, 510), dis)
back.paste(inp, (10, 583), inp)
back.paste(orz, (10, 655), orz)

back.show()
back.save("Test.jpg")

The problem is that Pillow is an image library so i can't open a video and paste the images, and OpenCV not accept PNG images easily like Pillow... is there any way to mix this 2 codes in 1 and do what i want to do? (plotting the png images in videoframes to get a render video with text and the images) this project it's for get the information of sensors and plotting, that's why i made a test function just for see,(the images it's a detail that i want to add to the project). If my code likes you and it's useful for you feel free to use it! thank you very much for reading!!! i hope you can help me (i don't speak fluenty english sooo, sorry for posibbles mistakes, but i can read perfectly your answers).

Answer 1

You can mix PIL/Pillow and OpenCV. There are three things you need to know...

To convert an OpenCV image into a PIL image:

pilimage = Image.fromarray(opencvimage)

To convert from a PIL image into an OpenCV image:

opencvimage = np.array(pilimage)

And lastly, OpenCV stores images in BGR order while PIL uses RGB, so reds and blues will get swapped if you don't watch out - cvtColor(BGR2RGB) is your friend here.

So we can now mix up those two pieces of code:

#!/usr/bin/env python3

import cv2
import random
import numpy as np
from PIL import Image

# Load a speedo image which has transparency
speedo = Image.open('speedo.png').convert('RGBA')

capture = cv2.VideoCapture("movie.mov")
out = cv2.VideoWriter('test.mov',cv2.VideoWriter_fourcc('a','v','c','1'), 25, (1280, 720))

def velocity():
    now = (random.randint(0,100))
    return now

while True:
    ret, frame = capture.read()
    if ret:
        font=cv2.FONT_HERSHEY_SIMPLEX
        cv2.putText(frame,'Velocity:',(15,580),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Distance:',(15,620),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Inclination:',(15,660),font,0.7,(255,255,255),1)
        cv2.putText(frame,'Orientation:',(15,700),font,0.7,(255,255,255),1)
        cv2.putText(frame, str(velocity()), (130,580),font,0.7,(255,255,255),1)
        cv2.putText(frame,'KM/H',(165,580),font,0.7,(255,255,255),1)

        # Make PIL image from frame, paste in speedo, revert to OpenCV frame
        pilim = Image.fromarray(frame)
        pilim.paste(speedo,box=(80,20),mask=speedo)
        frame = np.array(pilim)

        out.write(frame)
        cv2.imshow("Testing", frame)
        cv2.waitKey(1)

    else:
        break

capture.release()
out.release()
cv2.destroyAllWindows()

Here's the speedo I used:

---

If you want the colours to go across to OpenCV correctly from PIL, you need to re-order the channels to what OpenCV expects, namely BGRA. So, you could change the image loading to this:

# Open speedo image, split into separate RGBA channels, then re-combine in BGRA order for OpenCV
speedo = Image.open('speedo.png').convert('RGBA')
R,G,B,A = speedo.split()
speedo = Image.merge('RGBA',(B,G,R,A))