How to load tfrecord in pytorch?

Question

How to use tfrecord with pytorch?

I have downloaded "Youtube8M" datasets with video-level features, but it is stored in tfrecord. I tried to read some sample from these file to convert it to numpy and then load in pytorch. But it failed.

    reader = YT8MAggregatedFeatureReader()
    files = tf.gfile.Glob("/Data/youtube8m/train*.tfrecord")
    filename_queue = tf.train.string_input_producer(
        files, num_epochs=5, shuffle=True)
    training_data = [
        reader.prepare_reader(filename_queue) for _ in range(1)
    ]

    unused_video_id, model_input_raw, labels_batch, num_frames = tf.train.shuffle_batch_join(
        training_data,
        batch_size=1024,
        capacity=1024 * 5,
        min_after_dequeue=1024,
        allow_smaller_final_batch=True  ,
        enqueue_many=True)

    with tf.Session() as sess:
        label_numpy = labels_batch.eval()
        print(type(label_numpy))

But this step have no result, just stuck for a long while without any response.

Answer 1

Maybe this can help you: TFRecord reader for PyTorch