Java gstreamer linking two textoverlay elements is not working

Question

I have a rtsp player application written in java and built on top of gstreamer 1.When I try to display a text on top of the playing video with only one textoverlay element in the pipeline it is working fine.But I need to display different texts on all the corners of the window.

The first think I though was chaining overlay elements which actually worked from command line with gst-launch-1.0 as follows;

gst-launch-1.0 -v rtspsrc location=rtsp://10.0.5.41:8554  ! rtpjitterbuffer  ! rtph264depay ! vaapiparse_h264 ! vaapidecodebin !  textoverlay text = "live video" halignment=left ! textoverlay text="action camera 1" ! xvimagesink

When I try to construct the same pipeline with java as follows;

this.sourceElement = ElementFactory.make(RTSP_SOURCE, RTSP_SOURCE);
    final Element videoQueue = ElementFactory.make(QUEUE, QUEUE);
    final Element videoDepay = ElementFactory.make(RTP_H264_DEPAY, RTP_H264_DEPAY);
    final Element videoParser = ElementFactory.make(VAAPI_H264_PARSE, VAAPI_H264_PARSE);
    final Element videoDecoder = ElementFactory.make(VAAPI_DECODE, VAAPI_DECODE);
    videoTypeOverlay = ElementFactory.make(TEXT_OVERLAY, TEXT_OVERLAY);
    videoSourceOverlay = ElementFactory.make(TEXT_OVERLAY, TEXT_OVERLAY);
    sinkElement = ElementFactory.make(XV_IMAGE_SINK, XV_IMAGE_SINK);

    pipe.addMany(sourceElement, videoQueue, videoDepay, videoParser, videoDecoder, videoSourceOverlay, videoTypeOverlay, sinkElement);
    Element.linkMany(videoQueue, videoDepay, videoParser, videoDecoder, videoSourceOverlay, videoTypeOverlay, sinkElement);

      sourceElement.connect((Element.PAD_ADDED) (element, pad) -> {
        if (pad.isLinked()) {
            return;
        }
        Caps caps = pad.getCaps();
        if (caps.size() > 0) {
            String mediaType = caps.getStructure(0).getString("media");
            if ("video".equalsIgnoreCase(mediaType)) {
                pad.link(videoQueue.getStaticPad("sink"));
            }
        }
    });

 @Override
        public void play(PlaySettings playSettings) {
            videoTimeOverlay.set("text", text);
            videoTimeOverlay.set("valignment", valign);
            videoTimeOverlay.set("halignment", halign);
            ...
    }

I get a blank canvas. The only way to make application at least working is to remove one of the overlays from linkMany and addMany lines as follows;

pipe.addMany(sourceElement, videoQueue, videoDepay, videoParser, videoDecoder, videoSourceOverlay,  sinkElement);
Element.linkMany(videoQueue, videoDepay, videoParser, videoDecoder, videoSourceOverlay,  sinkElement);

So how can I put more than one static overlay strings on video canvas?

Answer 1

I solved this problem by creating a Bin element and the two overlay elements are wrapped in it. The method below is a working code example for creating Bin container element with ghost pads.

  private Bin createTextOverlayBin() {
    Bin textOverlayBin = new Bin();
    videoTypeOverlay = ElementFactory.make(TEXT_OVERLAY, "video-type-overlay");

    Optional<Pad> textOverlaySinkPad = playIdOverlay.getSinkPads().stream().filter(pad -> pad.getName().equals("video_sink")).findAny();
    Optional<Pad> textOverlaySrcPad = videoTypeOverlay.getSrcPads().stream().filter(pad -> pad.getName().equals("src")).findAny();

    if (textOverlaySinkPad.isPresent() && textOverlaySrcPad.isPresent()) {
        textOverlayBin.add(playIdOverlay);
        textOverlayBin.add(videoTypeOverlay);

        GhostPad ghostSinkPad = new GhostPad("sink", textOverlaySinkPad.get());
        textOverlayBin.addPad(ghostSinkPad);

        GhostPad ghostSrcPad = new GhostPad("src", textOverlaySrcPad.get());
        textOverlayBin.addPad(ghostSrcPad);


        playIdOverlay.link(videoTypeOverlay);
    } else {
        LOGGER.error("Video text overlay element creation is failed!");
    }
    return textOverlayBin;
}