How to define positional arguments: 'op', 'value_index', and 'dtype' in a tensor?

Question

when I run the below piece of code

from tensorflow.python.ops.numpy_ops import np_config
np_config.enable_numpy_behavior()
import pandas as pd

df = pd.DataFrame(
    {'x':[1.,2.,3.,4.],
     'y':[1.59,4.24,2.38,0.53]}
)

data = tf.data.Dataset.from_tensor_slices(df.to_numpy())
data = data.flat_map(lambda x: x.reshape((2,1)))

I receive: TypeError: init() missing 3 required positional arguments: 'op', 'value_index', and 'dtype' . I understand why this is happening as I didn't define values for 'op', 'value_index', and 'dtype' amd that tensorflow cant produce tensors. Basically I want to use flat_map function to create tensors with shape = (1,) and dtype = tf.float64 such that when I run the below code the printed tensors look like:

for item in data:
    print(item)

tf.Tensor([1.], shape=(1,), dtype=float64)
tf.Tensor([2.], shape=(1,), dtype=float64)
tf.Tensor([3.], shape=(1,), dtype=float64)
tf.Tensor([4.], shape=(1,), dtype=float64)
tf.Tensor([1.59], shape=(1,), dtype=float64)
tf.Tensor([4.24], shape=(1,), dtype=float64)
tf.Tensor([2.38], shape=(1,), dtype=float64)
tf.Tensor([0.53], shape=(1,), dtype=float64)

How can I specify those values inside flat_map function or any other function which I can pass into flat_map function?

I checked here https://www.tensorflow.org/api_docs/python/tf/Tensor but unfortunately I couldn't come up with a solution.

Thanks!

Answer 1

The function passed as argument to flat_map should return a Dataset. In your case, you are returning df.to_numpy() which is a numpy array, and hence you are receiving this error.

Solution is simple. Try:

import tensorflow as tf

data = data.flat_map(lambda x: tf.data.Dataset.from_tensor_slices(x.reshape((2,1)))).