python bokeh: get image from webcam and show it in dashboard

Question

I want to display an image - e.g. capture with the webcam - in bokeh. I tried image_url and image_rgba, but both are not working. Image_url is showing nothing, image_rgb shows something, but there seems to be some index shift.

    # -*- coding: utf-8 -*-

from bokeh.plotting import figure, show
import scipy.misc
import cv2
import matplotlib.pyplot as plt
import os

# capture video frame from webcam
#vc = cv2.VideoCapture(1)
vc = cv2.VideoCapture(-0)
rval, frame = vc.read()
vc.release()
                   
# show captured image (figure 1)
fig = plt.figure()
ax = plt.Axes(fig,[0,0,1,1])
ax.set_axis_off()
fig.add_axes(ax)
ax.imshow(frame)
plt.show()

# save captured image
scipy.misc.imsave('outfile.jpg', frame)
mypath = os.path.join(os.getcwd(),'outfile.jpg')

# 1. try using image_url ... not working (figure 2)
p = figure()
p.image_url(url=[mypath],x=[0],y=[1],w=[10],h=[10])
show(p)

# 2. try using image_rgba ... also not working (figure 3)
p = figure(x_range=(0,10), y_range=(0,10))
p.image_rgba(image=[frame], x=0, y=0, dw=10, dh=10)
show(p)

# 3. solution provided by @bigreddot ... flipped but also not working (figure4)
img = scipy.misc.imread(mypath)[::-1] # flip y-direction for Bokeh
p = figure(x_range=(0,10), y_range=(0,10))
p.image_rgba(image=[img], x=0, y=0, dw=10, dh=10)
show(p)

update: according to @bigreddot 's comment

figure 1

figure 2

figure 3

figure 4

the bokeh version I use is 0.13 the python version 3.6.0

Answer 1

After investigation, the return result from OpenCV is a Numpy array of bytes with shape (M, N, 3), i.e. RGB tuples. What Bokeh expects is a Numpy array of shape (M, N) 32-bit integers representing RGBA values. So you need to convert from one format to the other. Here is a complete example:

from bokeh.plotting import figure, show
import scipy.misc
import cv2
import os
import numpy as np

#vc = cv2.VideoCapture(1)
vc = cv2.VideoCapture(-0)
rval, frame = vc.read()
vc.release()

M, N, _ = frame.shape
img = np.empty((M, N), dtype=np.uint32)
view = img.view(dtype=np.uint8).reshape((M, N, 4))
view[:,:,0] = frame[:,:,0] # copy red channel
view[:,:,1] = frame[:,:,1] # copy blue channel
view[:,:,2] = frame[:,:,2] # copy green channel
view[:,:,3] = 255

img = img[::-1] # flip for Bokeh

p = figure()
p.image_rgba(image=[img],x=0,y=0, dw=10, dh=10)
show(p)

Generates: