Search code examples
c++opencvtensorfloweigen

In tensorflow c++ api, how to save a picture or an array from a tensor

I am trying to run a saved model for image segmentation, using tf 1.5.0 c++ api. My model gets an input image with size 1*256*256*3, and feeds to a tensor like this:

  for (int x = 0; x < 256; x++) {
    for (int y = 0; y <256; y++) {
      data_(0, x, y, 0) =
          (float) image_out.at<cv::Vec3b>(x, y)[0];
      data_(0, x, y, 1) =
          (float) image_out.at<cv::Vec3b>(x, y)[1];
      data_(0, x, y, 2) =
          (float) image_out.at<cv::Vec3b>(x, y)[2];
    }
  }

Then I run the model using sess->Run(), and get the output:

input Tensor type: float shape: [1,224,224,3] values: [[[254 254 254]]]... Output Tensor type: float shape: [1,224,224,1] values: [[[0.160249829][0.0639446825][0.0414313935]]]...

I want to save the out put to a image using cv::imwrite(). However, a tensor can't be saved directly. So I tried to convert the tensor like this: tensorflow::tensor->eigen::mat->cv::mat. Code is:

auto m = Eigen::Map<Eigen::Matrix<
             float,           /* scalar element type */
             Eigen::Dynamic,  /* num_rows is a run-time value */
             Eigen::Dynamic,  /* num_cols is a run-time value */
             Eigen::RowMajor  /* tensorflow::Tensor is always row-major */
              >>(
                 outputs[0].flat<float>().data(),  /* ptr to data */
                 outputs[0].dim_size(1),           /* num_rows */
                 outputs[0].dim_size(2)            /* num_cols */);

  //std::cout << "m " << m << std::endl;

  cv::Mat rotMatrix;
  cv::eigen2cv(m, rotMatrix);

This raises an error when compiling:

note: template void cv::eigen2cv(const Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>&, cv::Mat&) void eigen2cv( const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, Mat& dst ) ^ /usr/local/opencv3.1/include/opencv2/core/eigen.hpp:63:6: note: template argument deduction/substitution failed: src/demo/demo.cpp:152:28: note:
\u2018Eigen::Map >\u2019 is not derived from \u2018const Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>\u2019 cv::eigen2cv(m, rotMatrix); ^ In file included from src/demo/demo.cpp:11:0: /usr/local/opencv3.1/include/opencv2/core/eigen.hpp:81:6: note: template void cv::eigen2cv(const Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>&, cv::Matx<_Tp, m, n>&) void eigen2cv( const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, ^ /usr/local/opencv3.1/include/opencv2/core/eigen.hpp:81:6: note: template argument deduction/substitution failed: src/demo/demo.cpp:152:28: note:
\u2018Eigen::Map >\u2019 is not derived from \u2018const Eigen::Matrix<_Scalar, _Rows, _Cols, _Options, _MaxRows, _MaxCols>\u2019 cv::eigen2cv(m, rotMatrix); ^ make: *** [obj/demo.o] Error 1

What's the matter?

On the other hand, I don't think this is a good way to get the picture from a tensor. I've sacnned the tf's c++ api doc, and didn't find a good way. https://www.tensorflow.org/api_docs/cc/class/tensorflow/tensor#classtensorflow_1_1_tensor_1a6afab48885080a80ff0b52437959d929

So, is there a convenient way to do this?

Solution

  • I think you can directly wrap tensor data with OpenCV matrix header this way:

    cv::Mat rotMatrix(outputs[0].dim_size(1), outputs[0].dim_size(2), CV_32FC1, outputs[0].flat<float>().data())