Search code examples
javamultithreadingjavafxfxmljavacv

Flashing White Screen In Different Resolution of JavaFx Application

We are developing video streaming app using JavaFx and JavaCv. We Have GridPane of 8X8 and each cell contains Pane layout and each pane containing ImageView.

Every time i am calling play function whenever user drag url to particular cell. we have 64 Urls,So we are calling play function 64 times(all 64 url are playing in background thread).

for(int i=0;i<8;i++){
        for(int j=0;j<8;j++){
            //grid08(8X8)
            Pane p=new Pane();
            ImageView im=new ImageView();
            Label l=new Label("");
            im.fitWidthProperty().bind(p.widthProperty());
            im.fitHeightProperty().bind(p.heightProperty());
            l.prefWidthProperty().bind(p.widthProperty());
            l.prefHeightProperty().bind(p.heightProperty());
            p.getChildren().addAll(im,l);
            l.setVisible(false);
            l.setStyle("-fx-text-fill: #fff; -fx-font-size: 12px");
            l.setAlignment(Pos.CENTER);
            p.setStyle("-fx-background-color: black;");
            grid.add(p,j,i);//grid->instance of gridPane
            p.setOnDragOver(new EventHandler<DragEvent>() {
            @Override
            public void handle(DragEvent event) {
            if(!l.isVisible()){
                event.acceptTransferModes(TransferMode.ANY);
                   }
                }
            });
            p.setOnDragDropped(new EventHandler<DragEvent>() {
                @Override
                public void handle(DragEvent event) {
                   play(im,event.getDragboard().getString());
                }
            });
      }
     }

 play(ImageView im,String Url){
   new Thread(new Runnable(){
         frameGrabber = new FFmpegFrameGrabber(Url);
         frameGrabber.setVideoOption("preset","ultrafast");
         frameGrabber.setOption("rtsp_transport","tcp");
         frameGrabber.setOption("stimeout" , "60000");
         frameGrabber.setAudioChannels(0);
         frameGrabber.setAudioCodec(0);
        try {
        frameGrabber.start();
        start=true;
        } catch (Exception e) {
        }
        JavaFXFrameConverter jconverter=new JavaFXFrameConverter();
        while(true){
        Frame frame = frameGrabber.grabImage();
        Image image=jconverter.convert(frame);
        Platform.runlater(new Runnable(){
         im.setImage(image);
         });
        }
      }).start();  
 }

Problem: We are facing White Screen Problem,after increasing some threshold Width and Height. Here i am attaching Snap of my application with different screen size; Screen1 Screen2 Screen3 Screen4

Screen1

Since We are facing this problem from long time. Any Lead will be highly appreciable. Thanks in advance.

Solution

  • Just a guess here, but when the image view is a different size to the image, the image view has quite a lot of work to do when you call setImage(...) (it has to resize the image). Since you have a lot of these happening, potentially every pulse, it's possible you are submitting more runnables to Platform.runLater(...) than it can handle (i.e. you submit another Platform.runLater(...) before the previous one submitted from the same thread can complete). This will cause rendering issues, because the rendering system can't keep up.

    I can't test this, so I'm not sure if it will fix the issue, but it's probably a good idea to only submit a new call to Platform.runLater(...) when the previous one has completed (i.e. you should do this even if it's not the cause of the problem).

    You can do this with

    void play(ImageView im,String Url){
    new Thread(() -> {
        frameGrabber = new FFmpegFrameGrabber(Url);
        frameGrabber.setVideoOption("preset","ultrafast");
        frameGrabber.setOption("rtsp_transport","tcp");
        frameGrabber.setOption("stimeout" , "60000");
        frameGrabber.setAudioChannels(0);
        frameGrabber.setAudioCodec(0);
        try {
            frameGrabber.start();
            start=true;
        } catch (Exception e) {
            e.printStackTrace();
        }
        JavaFXFrameConverter jconverter=new JavaFXFrameConverter();

        AtomicReference<Image> imageHolder = new AtomicReference<>(null);
        while(true){
            Frame frame = frameGrabber.grabImage();
            Image image=jconverter.convert(frame);
            if (imageHolder.getAndSet(image) == null) {
                Platform.runLater(() -> im.setImage(imageHolder.getAndSet(null)));
            }
        }
    }).start();  
}

    Note that in this code (and the original), all the images are being resized on the same thread (the FX Application Thread). If possible, it would probably be more efficient to resize each image in its own thread. You can't absolutely guarantee this (because there's no way to guarantee the image view doesn't change size between the image being resized in the background thread and the image view displaying the image), so you still need to bind the fitWidth and fitHeight properties, but this should reduce the amount of image resizing that happens on the UI thread, and consequently should improve performance. This relies on there being some mechanism in your frame grabber API (which I'm not familiar with) to resize images.

    for(int i=0;i<8;i++){
    for(int j=0;j<8;j++){
        //grid08(8X8)
        Pane p=new Pane();
        AtomicReference<Bounds> bounds = new AtomicReference<>(p.getBoundsInLocal());
        p.boundsInLocalProperty().addListener((obs, oldBounds, newBounds) -> bounds.set(newBounds));
        ImageView im=new ImageView();
        Label l=new Label("");
        im.fitWidthProperty().bind(p.widthProperty());
        im.fitHeightProperty().bind(p.heightProperty());
        l.prefWidthProperty().bind(p.widthProperty());
        l.prefHeightProperty().bind(p.heightProperty());
        p.getChildren().addAll(im,l);
        l.setVisible(false);
        l.setStyle("-fx-text-fill: #fff; -fx-font-size: 12px");
        l.setAlignment(Pos.CENTER);
        p.setStyle("-fx-background-color: black;");
        grid.add(p,j,i);//grid->instance of gridPane
        p.setOnDragOver(new EventHandler<DragEvent>() {
            @Override
            public void handle(DragEvent event) {
                if(!l.isVisible()){
                    event.acceptTransferModes(TransferMode.ANY);
                }
            }
        });
        p.setOnDragDropped(new EventHandler<DragEvent>() {
            @Override
            public void handle(DragEvent event) {
               play(im,bounds,event.getDragboard().getString());
            }
        });
     }
 }

    and

    void play(ImageView im, AtomicReference<Bounds> bounds, String Url){
    new Thread(() -> {
        frameGrabber = new FFmpegFrameGrabber(Url);
        frameGrabber.setVideoOption("preset","ultrafast");
        frameGrabber.setOption("rtsp_transport","tcp");
        frameGrabber.setOption("stimeout" , "60000");
        frameGrabber.setAudioChannels(0);
        frameGrabber.setAudioCodec(0);
        try {
            frameGrabber.start();
            start=true;
        } catch (Exception e) {
            e.printStackTrace();
        }
        JavaFXFrameConverter jconverter=new JavaFXFrameConverter();

        AtomicReference<Image> imageHolder = new AtomicReference<>(null);
        while(true){
            Frame frame = frameGrabber.grabImage();

            Bounds imgBounds = bounds.get();
            double width = imgBounds.getWidth();
            double height = imgBounds.getHeight();

            // resize frame to width, height....

            Image image=jconverter.convert(frame);

            if (imageHolder.getAndSet(image) == null) {
                Platform.runLater(() -> im.setImage(imageHolder.getAndSet(null)));
            }
        }
    }).start();  
}