Search code examples
cgstreamergtk3tegra

Gstreamer video in GTK Window

UPDATE 2018.03.12: The code has been updated and now runs as expected

I am struggeling a bit with creating a GTK+ Window equivalent to the following gst-launch command gst-launch-1.0 udpsrc port=6000 caps="application/x-rtp" ! rtph264depay ! h264parse ! omxh264dec ! autovideosink

My code is as following:

int main (int argc, char **argv)
{
  GdkWindow *video_window_xwindow;
  GtkWidget *window, *video_window;
  gulong embed_xid;
  GstStateChangeReturn sret;

  gst_init (&argc, &argv);
  gtk_init (&argc, &argv);

  GstElement *pipeline, *udpsrc, *appxrtp, *depay, *parse, *omxh264dec, *videoConvert, *sink;

  pipeline = gst_pipeline_new ("xvoverlay");
  udpsrc = gst_element_factory_make ("udpsrc", NULL); g_assert(udpsrc);

  //Set CAPS
  g_object_set (G_OBJECT (udpsrc), "port", 6000, NULL);
  GstCaps * xrtpcaps = gst_caps_from_string("application/x-rtp,encoding-name=H264");
  g_object_set (udpsrc, "caps", xrtpcaps, NULL);

  depay = gst_element_factory_make ("rtph264depay", NULL); g_assert(depay);
  parse = gst_element_factory_make ("h264parse", NULL); g_assert(parse);
  omxh264dec = gst_element_factory_make ("omxh264dec", NULL); g_assert(omxh264dec);
  videoConvert = gst_element_factory_make ("videoconvert", NULL); g_assert(videoConvert);
  sink = gst_element_factory_make ("xvimagesink", NULL); g_assert(sink);

  //ADD
  gst_bin_add_many (GST_BIN (pipeline), udpsrc, depay, parse, omxh264dec, videoConvert, sink, NULL);

  //LINK
  g_assert(gst_element_link (udpsrc, depay));
  g_assert(gst_element_link (depay, parse));
  g_assert(gst_element_link (parse, omxh264dec));
  g_assert(gst_element_link (omxh264dec, videoConvert));
  g_assert(gst_element_link (videoConvert, sink));

  /* prepare the ui */
  window = gtk_window_new (GTK_WINDOW_TOPLEVEL);   

  g_signal_connect (G_OBJECT (window), "delete-event", G_CALLBACK (window_closed), (gpointer) pipeline);
  gtk_window_set_default_size (GTK_WINDOW (window), 600, 400);
  gtk_window_set_title (GTK_WINDOW (window), "GstVideoOverlay Gtk+ demo");

  video_window = gtk_drawing_area_new ();
  gtk_container_add (GTK_CONTAINER (window), video_window);
  gtk_container_set_border_width (GTK_CONTAINER (window), 2);

  gtk_widget_show_all (window);

  video_window_xwindow = gtk_widget_get_window (video_window);
  embed_xid = GDK_WINDOW_XID (video_window_xwindow);
  gst_video_overlay_set_window_handle (GST_VIDEO_OVERLAY (sink), embed_xid);

  /* run the pipeline */
  sret = gst_element_set_state (pipeline, GST_STATE_PLAYING);
  if (sret == GST_STATE_CHANGE_FAILURE)
    gst_element_set_state (pipeline, GST_STATE_NULL);
  else
    gtk_main ();

  gst_object_unref (pipeline);
  return 0;
}

When running it I get a black/empty GTK Window. blank screen

Console output is:

NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1223: BeginSequence  320x240, bVPR = 0
TVMR: LowCorner Frequency = 100000 
TVMR: cbBeginSequence: 1622: DecodeBuffers = 17, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1693: Display Resolution : (320x240) 
TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (320x240) 
TVMR: cbBeginSequence: 1762: ColorFormat : 5 
TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601
TVMR: cbBeginSequence: 1904: SurfaceLayout = 3
TVMR: cbBeginSequence: 2005: NumOfSurfaces = 24, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 2007: BeginSequence  ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2
Allocating new output: 320x240 (x 24), ThumbnailMode = 0
OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 320, nFrameHeight = 240

I am running this as the video source (which seems fine, I am able to consume this with gst-launch without any issues)

gst-launch-1.0 videotestsrc  ! omxh264enc control-rate=2 bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! h264parse ! rtph264pay mtu=1400 ! udpsink host=127.0.0.1 port=6000

Can anyone give me some pointers? Do I need to link differently (when pads are added)?

Solution

  • Adding videoconvert between omxh264dec and sink fixed the problem. The video is now rendered in the GTK Window. Have updated the original code.