IMFSourceReaderCallback::OnReadSample Function

Question

To read samples from your webcam and write them to file you can implement IMFSourceReaderCallback and the call back function OnReadSample will send IMFSample *pSample with some others parameters like the timestamp you can use them with ISinkWriter to write them on file(You can check MSDN example MFCaptureToFile Sample )

STDMETHODIMP OnReadSample(
    HRESULT hrStatus,
    DWORD dwStreamIndex,
    DWORD dwStreamFlags,
    LONGLONG llTimestamp,
    IMFSample *pSample

);

I have set my encoded parameters to H264 .I convert pSample to rawdata(set of bytes) and removed ISinkWriter. I have a tricky question *"is IMFSample pSample encoded or they become encoded inside ISinkWriter as when I convert them to RawData I got a huge file so are they encoded?" One more question how can I test them after writing them to file(they are in a raw format)

Answer 1

The sample will be in a raw format such as YUV or RGB. You can't set the output media type to H264 on a video device source reader (unless that device happens to natively support H264). When you introduce the SinkWriter with an .mp4 file as the destination the Media Foundation will take care of resolving the topology which in this case would mean including the H264 Media Foundation Transform.

One way I use to test the raw output from a SourceReader is to specify an output format of MFVideoFormat_IYUV and then after writing to a file stream use ffmpeg to extract a jpeg or convert it to a avi video file.

    std::ofstream outputBuffer("rawframes.yuv", std::ios::out | std::ios::binary);

    IMFMediaBuffer *buf = NULL;
    DWORD bufLength;
    pSample->ConvertToContiguousBuffer(&buf);
    buf->GetCurrentLength(&bufLength);

    printf("Writing sample %i, sample time %i, sample duration %i, sample size .\n", sampleCount, llVideoTimeStamp, llSampleDuration);

    byte *byteBuffer;
    DWORD buffCurrLen = 0;
    DWORD buffMaxLen = 0;
    buf->Lock(&byteBuffer, &buffMaxLen, &buffCurrLen);
    outputBuffer.write((char *)byteBuffer, bufLength);
    outputBuffer.flush();

And then the ffmpeg commands:

ffmpeg -vcodec rawvideo -s 640x480 -pix_fmt yuv420p -i rawframes.yuv -vframes 1 output.jpeg
ffmpeg -vcodec rawvideo -s 640x480 -pix_fmt yuv420p -i rawframes.yuv out.avi

Sample console app that uses the above snippet available on github.