Using Python to parse and render Kinesis Video Streams and get an image representation of the input frame

Question

I have set up a pipeline in which, I live stream the video to Kinesis Video Stream (KVS), which sends the frames to Amazon Rekognition for face recognition, which further sends them to Kinesis Data Stream (KDS). Finally, KDS sends the results to a lambda.

For a frame on which face recognition has been conducted, I get the JSON of the following format: https://docs.aws.amazon.com/rekognition/latest/dg/streaming-video-kinesis-output-reference.html

My AIM is: Using this JSON, I somehow want to get an image representation of the frame which was recorded by the KVS.

What have I tried:

This JSON provides me with the Fragment Number.

I use this fragment number and make a call to the get_media_for_fragment_list

The above call returns a key called Payload in response.

I have been trying to somehow render this payload into an image.

However, I fail to do this every time as I do not know how to make sense out of this payload and decode it.

Following is the code snippet.

    def getFrameFromFragment(fragment):
         client = boto3.client('kinesis-video-archived-media',endpoint_url=data_endpoint_for_kvs)
         response = client.get_media_for_fragment_list(
             StreamName='kvs1',
             Fragments=[
                fragment,
             ]
         )
         payload = response['Payload']
         print(payload.read())

How do I use this payload to get an image?

I know of parsers that exist in Java: https://docs.aws.amazon.com/kinesisvideostreams/latest/dg/examples-renderer.html

However, I wanted to know of a solution in Python.

In case my question statement is wrong or doesn't make sense, feel free to ask me more about this issue.

Thanks for the help. :)

Answer 1

After receiving the payload using the following code,

kvs_stream = kvs_video_client.get_media(
                 StreamARN="ARN", 
                 StartSelector= 
                              {'StartSelectorType':'FRAGMENT_NUMBER',
                               'AfterFragmentNumber': decoded_json_from_stream['InputInformation']['KinesisVideo']['FragmentNumber']
                              }
                                       )

you can use,

 frame = kvs_stream['Payload'].read()

to receive to get the frame from the payload. Now you can open an mvi file and write this frame to it and then extract a particular frame using openCV from this mvi file.

with open('/tmp/stream.avi', 'wb') as f:
                f.write(frame)
                cap = cv2.VideoCapture(file.mvi)
                #use frame for further processing