Search code examples
pythoncssimageplotlyjupyter-lab

Fast plot of pixelated image in plotly

I use plotly to visualize images with additional data in python jupyter notebooks (jupyter-lab).

I have currently three implementations, two show the desired result but are slow, the third one is fast but shows blurry images.

Especially when it comes to investigating behavior of image-processing networks, I need to see each pixel of the image. CSS already supports that simply by setting this style parameter

style="image-rendering: pixelated;"

I would like to include this as part of my plotly visualizations.

These are the options I know of. (One and two seem to do something similar under the hood)

  1. plotly.express.imshow
  2. since I work with grayscale images, I can also use HeatMap to plot images
  3. encoding the image to base64 and setting it in go.Layout.images f"data:image/png;base64,{base64.b64encode(res)}"

Again: The option 1 and 2 work, but are very slow compared to option 3. (Edit: >30x slower for 1024x1024 image, see below) This impacts the development when doing this on a regular basis. The option 3 is nice, but has the problem that png images are blurred.

Here an example run in jupyter lab

import cv2
import base64
import numpy as np
from plotly import graph_objects as go
import plotly.express as px
from IPython.display import display, HTML

image = np.random.uniform(0, 1, size=(50, 50))

display(px.imshow(image))

_, res = cv2.imencode(".png", image * 255)
data = b"data:image/png;base64," + base64.b64encode(res)

fig = go.FigureWidget()
n_y_pixel, n_x_pixel = image.shape[:2]
layout = go.Layout(
    xaxis=dict(range=(0, n_x_pixel)),
    yaxis=dict(range=(n_y_pixel, 0), scaleanchor="x", scaleratio=1.0),
    images=[
        dict(
            source=data.decode(),
            xref="x",
            yref="y",
            x=-0.5,
            y=-0.5,
            sizex=n_x_pixel,
            sizey=n_y_pixel,
        )
    ],
)
fig.update_layout(layout)
display(fig)
Option 1: Option 3: Option 3 + manually hacking afterwards:
plotly express image without blur layout image, fast but unnecessarily blurry enter image description here

The manual hack is basically executing this in a separate cell afterwards. (It is not reliable, so you may want to reload the window and only plot option two)

display(HTML("<script>var imgElement = document.querySelector('image'); imgElement.style.imageRendering = 'pixelated'; </script>"))

My goal is to have option 3 with pixelated accuracy, without the need to hack it every time I plot something in every notebook. I don't see how this is possible, or even if this is simply a feature request to plotly.

Things I tried

  • I couldn't find a parameter in Layout where I could override the style
  • running the display(HTML( hack in my plot function before the figure has no impact. This would be necessary if want to integrate it into my plotting function

Thanks for any help!

I am using

  • python 3.10
  • plotly 5.6.0

Time measurements with an image of size (1024, 1024) in my browser (simply using the code above, but using image = np.random.uniform(0, 1, size=(1024, 1024)). plotly v5.15.0; chrome 114.0.5735.198, jupyter lab 3.3.2

  • 6.7 seconds using px.imshow
  • 0.2 seconds using my method with base64
Solution

  • As @EricLavault pointed out, binary_string=True does the job.

    import plotly.express as px

px.imshow(image, binary_string=True)

    This shows a 2D image (no-color) as a grayscale image including the pixelated effect, including a massive speedup