Search code examples
c#directshowdirectshow.net

DirectShow.Net Filter Graph Black Screen

QUESTION: Why does my filter graph work in GraphEditPlus but not in my program?

Hi Guys, I am building a gimbal system for a Sony FCB-EV7500 and have run into a bit of a problem. I just purchased a hauppauge colossus to take the cameras component out into my pc. After many hours of trying to figure out how DirectShow works I eventually got a filter graph that worked.

Filter Graph 1

The C# code looks like this:

 //graph builder
        ICaptureGraphBuilder2 pBuilder = (ICaptureGraphBuilder2)new CaptureGraphBuilder2();
        hr = pBuilder.SetFiltergraph(pGraph);
        checkHR(hr, "Can't SetFiltergraph");

        Guid CLSID_WDMStreamingCrossbars = new Guid("{A799A801-A46D-11D0-A18C-00A02401DCD4}");
        Guid CLSID_WDMStreamingCaptureDevices = new Guid("{65E8773D-8F56-11D0-A3B9-00A0C9223196}"); 
        Guid CLSID_WDMStreamingEncoderDevices = new Guid("{19689BF6-C384-48FD-AD51-90E58C79F70B}");
        Guid CLSID_ArcSoftMPEGDemux = new Guid("{D5F9370B-E232-464F-BE29-A885B1249190}"); //ArcDemux.ax
        Guid CLSID_ArcSoftVideoDecoder = new Guid("{B793E9A8-C53E-4845-9DE9-C32326EACCAD}"); //ASVid.ax
        Guid CLSID_VideoRenderer = new Guid("{6BC1CFFA-8FC1-4261-AC22-CFB4CC38DB50}"); //quartz.dll

        //add Hauppauge Colossus Crossbar 0
        IBaseFilter pHauppaugeColossusCrossbar0 = CreateFilterByName(@"Hauppauge Colossus Crossbar 0", CLSID_WDMStreamingCrossbars);
        hr = pGraph.AddFilter(pHauppaugeColossusCrossbar0, "Hauppauge Colossus Crossbar 0");
        checkHR(hr, "Can't add Hauppauge Colossus Crossbar 0 to graph");

        //add Hauppauge Colossus Capture 0
        IBaseFilter pHauppaugeColossusCapture0 = CreateFilterByName(@"Hauppauge Colossus Capture 0", CLSID_WDMStreamingCaptureDevices);
        hr = pGraph.AddFilter(pHauppaugeColossusCapture0, "Hauppauge Colossus Capture 0");
        checkHR(hr, "Can't add Hauppauge Colossus Capture 0 to graph");

        //add Hauppauge Colossus TS Encoder 0
        IBaseFilter pHauppaugeColossusTSEncoder0 = CreateFilterByName(@"Hauppauge Colossus TS Encoder 0", CLSID_WDMStreamingEncoderDevices);
        hr = pGraph.AddFilter(pHauppaugeColossusTSEncoder0, "Hauppauge Colossus TS Encoder 0");
        checkHR(hr, "Can't add Hauppauge Colossus TS Encoder 0 to graph");

        //add ArcSoft MPEG Demux
        IBaseFilter pArcSoftMPEGDemux = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_ArcSoftMPEGDemux));

        IPin outPutPin;
        AMMediaType PinType = new AMMediaType();
        PinType.majorType = MediaType.Video;
        PinType.subType = MediaSubType.H264;
        hr = (pArcSoftMPEGDemux as IMpeg2Demultiplexer).CreateOutputPin(PinType, "Video", out outPutPin);
        checkHR(hr, "Can't add Output Pin to ArcSoft MPEG Demux");

        hr = pGraph.AddFilter(pArcSoftMPEGDemux, "ArcSoft MPEG Demux");
        checkHR(hr, "Can't add ArcSoft MPEG Demux to graph");

        //add ArcSoft Video Decoder
        IBaseFilter pArcSoftVideoDecoder = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_ArcSoftVideoDecoder));
        hr = pGraph.AddFilter(pArcSoftVideoDecoder, "ArcSoft Video Decoder");
        checkHR(hr, "Can't add ArcSoft Video Decoder to graph");

        //add Video Renderer
        IBaseFilter pVideoRenderer = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_VideoRenderer));
        hr = pGraph.AddFilter(pVideoRenderer, "Video Renderer");
        checkHR(hr, "Can't add Video Renderer to graph");

        //connect Hauppauge Colossus Crossbar 0 and Hauppauge Colossus Capture 0
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusCrossbar0, "0: Video Decoder Out"), GetPin(pHauppaugeColossusCapture0, "Analog Video In"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus Crossbar 0 and Hauppauge Colossus Capture 0");

        //connect Hauppauge Colossus Capture 0 and Hauppauge Colossus TS Encoder 0
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusCapture0, "Virtual Video Out"), GetPin(pHauppaugeColossusTSEncoder0, "Virtual Video Out"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus Capture 0 and Hauppauge Colossus TS Encoder 0");

        //connect Hauppauge Colossus TS Encoder 0 and ArcSoft MPEG Demux
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusTSEncoder0, "AVC TS Out"), GetPin(pArcSoftMPEGDemux, "Input"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus TS Encoder 0 and ArcSoft MPEG Demux");

        //map pinPID for ArcSoft MPEG Demux
        IMPEG2PIDMap pidmap2 = (IMPEG2PIDMap)outPutPin;
        hr = pidmap2.MapPID(1, new int[1] { 0x1011 }, MediaSampleContent.ElementaryStream);
        checkHR(hr, "Can't map video output pin pid to ArcSoft MPEG Demux");

        //connect ArcSoft MPEG Demux and ArcSoft Video Decoder
        hr = pGraph.ConnectDirect(GetPin(pArcSoftMPEGDemux, "Video"), GetPin(pArcSoftVideoDecoder, "Video In"), null);
        checkHR(hr, "Can't connect MPEG-2 Demultiplexer and ArcSoft Video Decoder");

        //connect ArcSoft Video Decoder and Video Renderer
        hr = pGraph.ConnectDirect(GetPin(pArcSoftVideoDecoder, "Video Out"), GetPin(pVideoRenderer, "VMR Input0"), null);
        checkHR(hr, "Can't connect ArcSoft Video Decoder and Video Renderer");

Using this graph everything works fine. I can display the camera's video on a form then create a transparent form on top of that and use GDI+ to build my on screen display. The 2 things I would still like to add to my OSD are the current FPS count, and when I pull the trigger on my joystick I would like to capture the screen and save it to disk.

So in order to do this I believe that I need to add a sample grabber filter to time the FPS manually and pull out individual frames. I added a sample grabber to the graph along with an AVI Decompressor and tried to run it.

Filter Graph 2

With all the trouble I have been having with direct-show I didn't expect it to work but surprisingly it did. In GraphEditPlus the video displayed and I could see the sample-grabber capturing the frames. SO I exported it to C# and attempted to run it again.

int hr = 0;

        //graph builder
        ICaptureGraphBuilder2 pBuilder = (ICaptureGraphBuilder2)new CaptureGraphBuilder2();
        hr = pBuilder.SetFiltergraph(pGraph);
        checkHR(hr, "Can't Set Filtergraph");

        Guid CLSID_WDMStreamingCrossbars = new Guid("{A799A801-A46D-11D0-A18C-00A02401DCD4}");
        Guid CLSID_WDMStreamingCaptureDevices = new Guid("{65E8773D-8F56-11D0-A3B9-00A0C9223196}"); 
        Guid CLSID_WDMStreamingEncoderDevices = new Guid("{19689BF6-C384-48FD-AD51-90E58C79F70B}");
        Guid CLSID_ArcSoftMPEGDemux = new Guid("{D5F9370B-E232-464F-BE29-A885B1249190}"); //ArcDemux.ax
        Guid CLSID_ArcSoftVideoDecoder = new Guid("{B793E9A8-C53E-4845-9DE9-C32326EACCAD}"); //ASVid.ax
        Guid CLSID_SampleGrabber = new Guid("{C1F400A0-3F08-11D3-9F0B-006008039E37}"); //qedit.d11

        //add Hauppauge Colossus Crossbar 0
        IBaseFilter pHauppaugeColossusCrossbar0 = CreateFilterByName(@"Hauppauge Colossus Crossbar 0", CLSID_WDMStreamingCrossbars);
        hr = pGraph.AddFilter(pHauppaugeColossusCrossbar0, "Hauppauge Colossus Crossbar 0");
        checkHR(hr, "Can't add Hauppauge Colossus Crossbar 0 to graph");

        //add Hauppauge Colossus Capture 0
        IBaseFilter pHauppaugeColossusCapture0 = CreateFilterByName(@"Hauppauge Colossus Capture 0", CLSID_WDMStreamingCaptureDevices);
        hr = pGraph.AddFilter(pHauppaugeColossusCapture0, "Hauppauge Colossus Capture 0");
        checkHR(hr, "Can't add Hauppauge Colossus Capture 0 to graph");

        //add Hauppauge Colossus TS Encoder 0
        IBaseFilter pHauppaugeColossusTSEncoder0 = CreateFilterByName(@"Hauppauge Colossus TS Encoder 0", CLSID_WDMStreamingEncoderDevices);
        hr = pGraph.AddFilter(pHauppaugeColossusTSEncoder0, "Hauppauge Colossus TS Encoder 0");
        checkHR(hr, "Can't add Hauppauge Colossus TS Encoder 0 to graph");

        //add ArcSoft MPEG Demux
        IBaseFilter pArcSoftMPEGDemux = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_ArcSoftMPEGDemux));
        //add Video output map
        IPin outPutPin;
        AMMediaType PinType = new AMMediaType();
        PinType.majorType = MediaType.Video;
        PinType.subType = MediaSubType.H264;
        hr = (pArcSoftMPEGDemux as IMpeg2Demultiplexer).CreateOutputPin(PinType, "Video", out outPutPin);
        checkHR(hr, "Can't add Output Pin to ArcSoft MPEG Demux");
        hr = pGraph.AddFilter(pArcSoftMPEGDemux, "ArcSoft MPEG Demux");
        checkHR(hr, "Can't add ArcSoft MPEG Demux to graph");

        //add ArcSoft Video Decoder
        IBaseFilter pArcSoftVideoDecoder = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_ArcSoftVideoDecoder));
        hr = pGraph.AddFilter(pArcSoftVideoDecoder, "ArcSoft Video Decoder");
        checkHR(hr, "Can't add ArcSoft Video Decoder to graph");

        //add SampleGrabber
        IBaseFilter pSampleGrabber = (IBaseFilter)Activator.CreateInstance(Type.GetTypeFromCLSID(CLSID_SampleGrabber));
        hr = pGraph.AddFilter(pSampleGrabber, "SampleGrabber");
        checkHR(hr, "Can't add SampleGrabber to graph");
        (pSampleGrabber as ISampleGrabber).SetCallback(this, 0);

        /*
        AMMediaType pSampleGrabber_pmt = new AMMediaType();
        pSampleGrabber_pmt.majorType = MediaType.Video;
        pSampleGrabber_pmt.subType = MediaSubType.YUY2;
        pSampleGrabber_pmt.formatType = FormatType.VideoInfo;
        pSampleGrabber_pmt.fixedSizeSamples = true;
        pSampleGrabber_pmt.formatSize = 88;
        pSampleGrabber_pmt.sampleSize = 4147200;
        pSampleGrabber_pmt.temporalCompression = false;
        VideoInfoHeader pSampleGrabber_format = new VideoInfoHeader();
        pSampleGrabber_format.SrcRect = new DsRect();
        pSampleGrabber_format.SrcRect.right = 1920;
        pSampleGrabber_format.SrcRect.bottom = 1080;
        pSampleGrabber_format.TargetRect = new DsRect();
        pSampleGrabber_format.TargetRect.right = 1920;
        pSampleGrabber_format.TargetRect.bottom = 1080;
        pSampleGrabber_format.BitRate = 4000000;
        pSampleGrabber_format.AvgTimePerFrame = 333667;
        pSampleGrabber_format.BmiHeader = new BitmapInfoHeader();
        pSampleGrabber_format.BmiHeader.Size = 40;
        pSampleGrabber_format.BmiHeader.Width = 1920;
        pSampleGrabber_format.BmiHeader.Height = 1080;
        pSampleGrabber_format.BmiHeader.Planes = 1;
        pSampleGrabber_format.BmiHeader.BitCount = 16;
        pSampleGrabber_format.BmiHeader.Compression = 844715353;
        pSampleGrabber_format.BmiHeader.ImageSize = 4147200;
        pSampleGrabber_pmt.formatPtr = Marshal.AllocCoTaskMem(Marshal.SizeOf(pSampleGrabber_format));
        Marshal.StructureToPtr(pSampleGrabber_format, pSampleGrabber_pmt.formatPtr, false);
        hr = ((ISampleGrabber)pSampleGrabber).SetMediaType(pSampleGrabber_pmt);
        DsUtils.FreeAMMediaType(pSampleGrabber_pmt);
        checkHR(hr, "Can't set media type to sample grabber");
        */

        //add AVI Decompressor
        IBaseFilter pAVIDecompressor2 = (IBaseFilter)new AVIDec();
        hr = pGraph.AddFilter(pAVIDecompressor2, "AVI Decompressor");
        checkHR(hr, "Can't add AVI Decompressor to graph");

        //connect Hauppauge Colossus Crossbar 0 and Hauppauge Colossus Capture 0
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusCrossbar0, "0: Video Decoder Out"), GetPin(pHauppaugeColossusCapture0, "Analog Video In"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus Crossbar 0 and Hauppauge Colossus Capture 0");

        //connect Hauppauge Colossus Capture 0 and Hauppauge Colossus TS Encoder 0
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusCapture0, "Virtual Video Out"), GetPin(pHauppaugeColossusTSEncoder0, "Virtual Video Out"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus Capture 0 and Hauppauge Colossus TS Encoder 0");

        //connect Hauppauge Colossus TS Encoder 0 and ArcSoft MPEG Demux
        hr = pGraph.ConnectDirect(GetPin(pHauppaugeColossusTSEncoder0, "AVC TS Out"), GetPin(pArcSoftMPEGDemux, "Input"), null);
        checkHR(hr, "Can't connect Hauppauge Colossus TS Encoder 0 and ArcSoft MPEG Demux");

        //map pinPID for ArcSoft MPEG Demux
        IMPEG2PIDMap pidmap2 = (IMPEG2PIDMap)outPutPin;
        hr = pidmap2.MapPID(1, new int[1] { 0x1011 }, MediaSampleContent.ElementaryStream);
        checkHR(hr, "Can't map video output pin pid to ArcSoft MPEG Demux");

        //connect ArcSoft MPEG Demux and ArcSoft Video Decoder
        hr = pGraph.ConnectDirect(GetPin(pArcSoftMPEGDemux, "Video"), GetPin(pArcSoftVideoDecoder, "Video In"), null);
        checkHR(hr, "Can't connect MPEG-2 Demultiplexer and ArcSoft Video Decoder");

        //connect ArcSoft Video Decoder and SampleGrabber
        hr = pGraph.ConnectDirect(GetPin(pArcSoftVideoDecoder, "Video Out"), GetPin(pSampleGrabber, "Input"), null);
        checkHR(hr, "Can't connect ArcSoft Video Decoder and SampleGrabber");

        //connect SampleGrabber and AVI Decompressor
        hr = pGraph.ConnectDirect(GetPin(pSampleGrabber, "Output"), GetPin(pAVIDecompressor2, "XForm In"), null);
        checkHR(hr, "Can't connect SampleGrabber and AVI Decompressor");

        //add Video Renderer
        IBaseFilter pVideoRenderer = (IBaseFilter)new VideoRenderer();
        hr = pGraph.AddFilter(pVideoRenderer, "Video Renderer");
        checkHR(hr, "Can't add Video Renderer to graph");

        //connect AVI Decompressor and Video Renderer
        hr = pGraph.ConnectDirect(GetPin(pAVIDecompressor2, "XForm Out"), GetPin(pVideoRenderer, "Input"), null);
        checkHR(hr, "Can't connect AVI Decompressor and Video Renderer");

Now all I get is a black screen and I can't figure out why. I have tried to make sure that the media type for the sample grabber is correct but to be honest I am not really sure what I am doing.(This is my first time using direct show) Any help is appreciated to solve this problem. Thanks for the help!

UPDATE I successfully connected to my graph remotely via the instructions of Peter Kostov and I think I figured out why it is not working.

Image 3

It appears that the decoder is not connecting to the sample grabber. I stepped through the method but got no errors. 

  //connect ArcSoft Video Decoder and SampleGrabber
  hr = pGraph.ConnectDirect(GetPin(pArcSoftVideoDecoder, "Video Out"),     GetPin(pSampleGrabber, "Input"), null);
  checkHR(hr, "Can't connect ArcSoft Video Decoder and SampleGrabber");

It appears to run fine but the output is still not connecting.

Solution

  • Well I finally solved it by adding a null renderer to the output of the sample grabber. I then changed the media type of the sample grabber from YUY2 to RGB24 and used the call back from the sample grabber to convert the data to a bitmap then all I had to do was display it on a picture box. Thanks for the help guys!