vlcj JPanel draw over mediaPlayerComponent

Question

I have fully a functional VLCj-based video player as shown bellow.

Working code

import java.awt.BorderLayout;
import java.awt.Dimension;
import java.awt.Graphics;

import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

import uk.co.caprica.vlcj.component.EmbeddedMediaPlayerComponent;
import uk.co.caprica.vlcj.discovery.NativeDiscovery;

/**
 * Minimal quick-start example.
 */
public class Example1 {

    private final JFrame frame;

    private final EmbeddedMediaPlayerComponent mediaPlayerComponent;

    public static void main(String[] args) {
        new NativeDiscovery().discover();

        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                new Example1().start("file:///C:/video.avi");
            }
        });
    }

    public Example1() {
        mediaPlayerComponent = new EmbeddedMediaPlayerComponent();

        frame = new JFrame("vlcj quickstart");
        frame.setLocation(50, 50);
        frame.setSize(1400, 800);
        frame.getContentPane().setLayout(new BorderLayout());
        frame.getContentPane().add(mediaPlayerComponent, BorderLayout.CENTER);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setPreferredSize(new Dimension(320, 240));
        frame.pack();
        frame.setVisible(true);
    }

    private void start(String mrl) {
        mediaPlayerComponent.getMediaPlayer().playMedia(mrl);
    }
}

Modifications to the code in order to draw over the video

What I need is to draw over the video (for example a rectangle). For this purpose I have created MyJPanel.

class MyPanel extends JPanel {
    private EmbeddedMediaPlayerComponent comp;

    public MyPanel(EmbeddedMediaPlayerComponent mediaPlayerComponent) {
        add(this.comp = mediaPlayerComponent);
    }
    
    @Override
    public void paint(Graphics g) {
        super.paint(g);
        g.drawRect(10, 10, 200, 100);
    }
}

And instead of the line:

frame.getContentPane().add(mediaPlayerComponent, BorderLayout.CENTER);

I added:

frame.getContentPane().add(new MyPanel(mediaPlayerComponent), BorderLayout.CENTER);

Problem:

After adding MyPanel I am getting this error: and no any video is being displayed.

[0000000029d930e0] avi demux error: no key frame set for track 0

[0000000029e035d0] core vout display error: Failed to set on top

Answer 1

You simply can not use Java2D to draw on top of the heavyweight AWT Canvas video surface.

There are however a number of other approaches you can use to render on top of the video:

1. Use the native marquee API - this is limited to text rendering.
2. Use the native logo API - of course this is limited to rendering bitmapped images , not drawing primitives like a rectangle.
3. Use a 'direct' media player - here you are rendering the video yourself and you can use Java2D to render on top of the video. However direct rendering media players are more demanding in terms of performance.
4. Overlay a transparent top-level Window on top of the video surface and render into that using Java2D in the usual way. This is not ideal, as you must keep the overlay window in sync with the video surface position and size and in doing so it can lag. vlcj has an API that helps with this, and a sample application that demonstrates it: https://github.com/caprica/vlcj/blob/d4c6dd5a0cb9f617fe236ea55fcf21b3fea588e8/src/test/java/uk/co/caprica/vlcj/test/overlay/OverlayTest.java

I think #4, whilst not ideal, is probably the best you can do and is closest to what you're asking for.