Search code examples
opencvimage-processingopen3d

How to capture Open3d screen without saving


I want to use the open3d image screen right away without saving it as a file.

However, the 'capture_screen_image' function provided by open3d must save the image. (http://www.open3d.org/docs/release/python_api/open3d.visualization.Visualizer.html)

This causes the problem of having to read the saved image back to 'cv2.imread'.

I wonder if there are other ways to solve this problem or other functions provided by open3d.


Solution

  • I would use capture_screen_float_buffer() instead of capture_screen_image().

    Remember the result is a (normalised) float numpy mat, not a typical np.uint8 typed one so depending on the use, you might also need to scale it back up to 0-255 range and cast as np.uint8:

    # get the image
    o3d_screenshot_mat = visualizer.capture_screen_float_buffer()
    # scale and convert to uint8 type
    o3d_screenshot_mat = (255.0 * np.asarray(o3d_screenshot_mat)).astype(np.uint8)
    # use as required
    # cv2.imshow("screenshot", o3d_screenshot_mat) , PIL.Image.fromarray(o3d_screenshot_mat , "RGB"), etc.
    

    (e.g. if you want to visualise, remap to 0-255 range, otherwise leave data as is and simply save/load as needed)