I'm struggling to work with different tensor types and operations between them. For example, a basic division tf.divide(a, b) is giving me the following error:
TypeError: Failed to convert elements of SparseTensor(indices=Tensor("inputs_8_copy:0", shape=(None, 2), dtype=int64), values=Tensor("cond/Cast_1:0", shape=(None,), dtype=float64), dense_shape=Tensor("inputs_10_copy:0", shape=(2,), dtype=int64)) to Tensor. Consider casting elements to a supported type. See https://www.tensorflow.org/api_docs/python/tf/dtypes for supported TF dtypes.
I was able to work around this by calling tf.sparse.to_dense on a and b. But the approach doesn't scale when the dataset is large. Nor does it work in general because I don't know the tensor type of all of the features (I'm working within a preprocessing_fn in TFT and the data comes from BigQuery).
This seems like a very common issue that should have a simple answer but I'm not able to find any information on it. Something like basic divisions shouldn't cause this much trouble?
It is a difficult question, in fact.
For element-wise division in particular, let say ai and bi are scalars. if ai = 0 and bi is not zero, then ai/bi = 0, but what if ai = 0 and bi = 0, ai/bi = ? 0?
Even worse, if ai is not zero and bi = 0 then ai/bi is NaN!
So if the divisor is a sparse tensor, it will raise (possibly lots of) NaNs, unless the indices of both sparse matrices are the same. Same problem if you divide a dense matrix by a sparse matrix.
There is a nice a workaround to multiply two sparse tensors element-wise here, based on the relation (a+b)^2 = a^2 + b^2 + 2 ab.
It is also possible to compute the inverse of a sparse tensor C: tf.SparseTensor(indices=C.indices, values=1/C.values, dense_shape=C.dense_shape).
So there is this NaN issue for division, and concerning the mixture of dense tensor and sparse tensor, one option consists in converting the sparse tensor to a dense tensor. But we want to avoid this. In the other direction, from converting the dense tensor to a sparse tensor, this can be very ineffective if the tensor is not really sparse.
All this to say that it does not seem to be a simple problem.