Search code examples
javascriptpythonopencvimread

Why read the same picture with python and javascript with the help of opencv, the results are inconsistent？I think it should be same

picture_4000*4000:

enter image description here

the python code is:

import cv2
pic_data = cv2.imread(path_to_picture)

the pic_data is:

pic_data
Out[21]: 
array([[[125, 189, 204],
        [125, 189, 204],
        [125, 189, 204],
        ...,
        [125, 189, 204],
        [125, 189, 204],
        [125, 189, 204]],

the javascript is:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <meta name="theme-color" content="#000000" />
    <meta
      name="description"
      content="Web site created using create-react-app"
    />
    <link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
    <!--
      manifest.json provides metadata used when your web app is installed on a
      user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
    -->
    <link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
    <!--
      Notice the use of %PUBLIC_URL% in the tags above.
      It will be replaced with the URL of the `public` folder during the build.
      Only files inside the `public` folder can be referenced from the HTML.

      Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
      work correctly both with client-side routing and a non-root public URL.
      Learn how to configure a non-root public URL by running `npm run build`.
    -->
    <title>React App</title>
  </head>
  <body>
    <noscript>You need to enable JavaScript to run this app.</noscript>
    <div id="root"></div>
    <!--
      This HTML file is a template.
      If you open it directly in the browser, you will see an empty page.

      You can add webfonts, meta tags, or analytics to this file.
      The build step will place the bundled scripts into the <body> tag.

      To begin the development, run `npm start` or `yarn start`.
      To create a production bundle, use `npm run build` or `yarn build`.
    -->
      <input type="file" id="js-input">
      <img id="js-img" />
    <script>
      function onOpenCvReady() {
        const input = document.getElementById('js-input');
        const img = document.getElementById('js-img');
        input.onchange = function () {
          const file = event.target.files[0];
          img.src = URL.createObjectURL(file);
          console.log(file)
        }
        img.onload = function () {
          const res = cv.imread(img);
          console.log(res)
        }
        console.log('ready', cv)
      }
    </script>
    <script src="https://docs.opencv.org/4.7.0/opencv.js" onload="onOpenCvReady()" type="text/javascript"></script>
  </body>
</html>

Can use Chrome to open javascript code. and load the image, the read data is:

enter image description here

Problem

(Ignore channel order inconsistencies)

The python code reads the first pixel of the data is [125, 189, 204]

The javascript code reads the first pixel of the data is [126, 190, 205]

I read the same picture separately through python and javascript, and get different results. I do not why, I want they hava the same results.

Update

Thanks to everyone who paid attention to this question. After reading rotem's answer and Christoph Rackwitz's comment. I randomly found several other pictures and did the same operation, and found that the reading results of python and javascript are consistent. This is very confusing, only picture_4000*4000 has the problem, did not reproduce the problem on other pictures.

Eg. the picture 1.png:

The python code reads the first pixel of 1.png is [245, 213, 154]

The javascript code reads the first pixel of 1.png is [245, 213, 154]

picture 1.png

enter image description here

picture 2.png

enter image description here

picture 3.jpg

enter image description here

Solution

  • It could be a bug in OpenCV.js, but it's more likely related to the exif metadata of the PNG image.

    When getting the exif information of the image using ExifTool:

    exiftool Picture4000x4000.png

    There are all kind of metadata that may affect the values of the pixels.
    Example:

    White Point X                   : 0.3127
White Point Y                   : 0.329
Red X                           : 0.64
Red Y                           : 0.33
Green X                         : 0.3
Green Y                         : 0.6
Blue X                          : 0.15
Blue Y                          : 0.06

    As far as I know, OpenCV in Python (and C++) ignores the metadata, and keep the "original" pixel values (without color conversion).

    I suppose OpenCV.js respects the metadata, and applies some kind of color conversion process (OpenCV.js reads the metadata and adjust the colors according to the metadata).

    We may use ImageMagick for striping the exif data:

    magick.exe Picture4000x4000.png -strip Picture4000x4000_copy.png

    When loading the copy without the exif data, the value of the first pixel is [125, 204, 189] - same value as in Python.

    The fact that OpenCV.js respects the metadata, is undocumented, and I can't find a way to prevent it, for getting same the values as in Python.

    Update:

    As commented by Christoph Rackwitz, in Python OpenCV uses libpng for reading the PNG image, and in OpenCV.js, the Web browser reads the image to canvas element and OpenCV reads the image from the canvas element.
    The image reading process is different (the browser uses the exif data and may apply color conversion...).

    In the JavaScript code, we can see that OpenCV.js reads the image from the canvas element and not from the PNG file:

    const img = document.getElementById('js-img');
const res = cv.imread(img);

    I can't find any attribute or CSS property for forcing the browser to ignore the exif data.