Search code examples
numpytensorflowkerasbackendloss

How to convert this numpy one-liner into Tensorflow backend code?

I have multiple depthmaps which show a car from different angles. I need to calculate how well they match together in my loss function, so I have to reproject them into a different view. The depthmaps live in a cube that is relative to the length of the vehicle. The images have the shape (256,256). I already wrote the code to convert them to a pointcloud with backend functions (256*256,3). I can reproject this pointcloud to the side view with numpy like this:

reProj = np.zeros((256, 256), np.float32)
reProj[pointCloud[:, 1], pointCloud[:, 2]] = pointCloud[:, 0]

How can I convert this into keras backend code? I suspect there should be a gather somewhere in there, but I just cannot get it working.

Example:

Source depth image:

enter image description here

Reprojected:

enter image description here

Thanks for your help!

Edit: Minimal working example with data: https://easyupload.io/rwutwa

Solution

  • So i finally figured it out, I was just thinking about it wrong. It is not a gather operation, is it a scatter. This works perfect now!

    indices = K.stack([p[:, 1], p[:, 2]], -1)
indices = K.reshape(indices, (256, 256, 2))
indices = K.clip(indices, 0, 256 - 1)
updates = K.reshape(p[:,0], (256,256))
reProj = tf.tensor_scatter_nd_max(tf.zeros((256, 256), tf.int32), indices, updates)