Search code examples
pythontensorflowdataloader

Generator dataloader in models with the same inputs and outputs


I am trying to implement a model whose output is the same as its input. It's a simple part of an extensive model, I deleted complicated parts. I wrote a generator dataloader for generating random numbers.

def random_generator():
    tf.random.set_seed(43)
    while True:
        yield tf.random.uniform((3,), 0, 1, dtype=tf.dtypes.float32, seed=32)

random_dataset = tf.data.Dataset.from_generator(
    random_generator,
    output_types=tf.float32,
    output_shapes=(3,)
)

I need to use the same dataloader for input and output, but I'll get different inputs and outputs as I zip it.

dataloader = tf.data.Dataset.zip((random_dataset, random_dataset))
model.fit(dataloader, epochs=200, batch_size=32)

Is there any way to copy the dataset or generate random arrays of numbers so that it produces the same result in the second call?


Solution

  • You can use TensorFlow generator seed setter and set both seeds to one number.

    def random_generator():
        generator = tf.random.Generator.from_seed(43)
        while True:
            yield tf.round(
                generator.uniform((3,), 0, 1, dtype=tf.dtypes.float32)
            )
    

    Now making generator data loader gives us the same results in the second call.