How to overly depth data on 2D image?

Question

All of this is done in Matlab. I have a 2D RGB image, with some depth data for key vertexes. That is, I have three vectors (m x 1): X, Y, Z. Together, [X(i), Y(i)] specifies the point in the image with depth Z(i).

The crux of my problem is this:

I would like to plot the image "warped" with the depth information. But, each time I keep calling functions like 'mesh(X,Y,Z,RGBImage)' and 'surf', I get weird errors like "Z need to be matrix not vector". Also, I haven't even been able to implement the 'warp' function, as I am not sure how to translate my data into a usable format.

Any help would be really appreciated.

EDIT: I finally got it to work the way I would like. The only thing that needed to be changed, that was not answered, was to include the line

s = surf(Xmat,Ymat,Zmat, T1, 'edgecolor', 'none', 'FaceColor', ... 
'texturemap');

Where T1 = rgb2gray(OrignialRGBImage);. Much thanks!

Answer 1

I don't have access to MATLAB right now, so I am giving the outline of the solution.

First, create two matrices using meshgrid: [Xmat,Ymat] = meshgrid(1:imgCols,1:imgRows);. Now, you can find every row of [X,Y] at [Xmat(i,j),Ymat(i,j)] for some (i,j). In the next step, create Zmat such that Zmat(i,j) is the correct depth value for [Ymat(i,j),Xmat(i,j)]. You should be able to do this as follows:

Zmat = zeros(imgRows,imgCols);
for i=1:size(X,1)
   Zmat(Y(i),X(i))=Z(i);
end

Now you can do surf with Xmat,Ymat,Zmat as follows. Then overlay image as texture.

s = surf(Xmat,Ymat,Zmat);
set(s, 'faceColor', 'texture','edgecolor', 'none','cdata', subimage); %not tested,
                                                                      %see if it works

Overlaying part taken from here.

If the above part doesn't work, then you can overlay Zmat in the heatmap style, see this question.