基于Java Swing+OpenCV的JPanel视频帧重绘失效问题排查

需求与问题背景 #

需求：开发一款支持帧编辑的视频播放器，通过JFrame内嵌JPanel，利用OpenCV捕获视频帧并在JPanel中显示，禁止使用Swing自定义控件。
问题：在视频帧读取循环中调用repaint()方法，但paintComponent仅在首次被调用，后续不再执行重绘。

原代码 #

public class MainPaint extends JFrame {

    static{ System.loadLibrary(Core.NATIVE_LIBRARY_NAME); }
    PaintPanel paintPan = null;
    
    public MainPaint() {
        setTitle("Video Panel");
        setSize(600, 600);
        setLayout(new BorderLayout());

        paintPan = new PaintPanel();
        JButton testButon = new JButton("Display Video");
        add(paintPan, BorderLayout.CENTER);
        add(testButon, BorderLayout.PAGE_END);

        testButon.addActionListener(new ActionListener() {

            @Override
            public void actionPerformed(ActionEvent arg0) {
                try {
                    paintPan.updateGraphics(50, 50);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                repaint();
            }
        });
        setVisible(true);
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    }

    public static void main(String[] args) {
        new MainPaint();
    }
}

class PaintPanel extends JPanel {

    public PaintPanel() {
       // setBackground(Color.ORANGE);
    }

    int count = 0;
    
    @Override
    public void paintComponent(Graphics g) {
        super.paintComponent(g);
        Graphics2D graphics2d = (Graphics2D)  g.create();
        graphics2d.drawImage(image, null, 0, 0);
        graphics2d.dispose();
    }

    BufferedImage image = null;
    public void updateGraphics(int length, int width) throws InterruptedException {
        File selectedFile = new File("C:\\image_test\\Opencvs\\data\\test.mp4");
        String filePath = selectedFile.getAbsolutePath();
        
        VideoCapture cap = new VideoCapture(filePath);
        Mat frame = new Mat();
        
        while (true) {
            if (cap.read(frame)) {
                image = Mat2bufferedImage(frame);
                repaint();
                Thread.currentThread().wait(100);
            }
        }
    }
    
    private static BufferedImage Mat2bufferedImage(Mat image) {
        MatOfByte bytemat = new MatOfByte();
        Imgcodecs.imencode(".jpg", image, bytemat);
        byte[] bytes = bytemat.toArray();
        InputStream in = new ByteArrayInputStream(bytes);
        BufferedImage img = null;
        try {
            img = ImageIO.read(in);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return img;
    }
}

问题原因分析 #

1. EDT线程被阻塞：Swing的事件处理（包括按钮点击回调）都在**事件调度线程（EDT）**中执行。updateGraphics方法里的while(true)循环直接在EDT中运行，导致EDT被完全占用，无法处理后续的重绘请求和其他UI事件，自然不会再调用paintComponent。
2. wait()方法使用错误：Thread.currentThread().wait(100)没有在同步代码块中调用，会抛出IllegalMonitorStateException；且wait是用于线程间通信的API，单纯延时应该用Thread.sleep()。

解决方案 #

修改要点： #

• 将视频帧读取逻辑放到独立线程中，避免阻塞EDT。
• 用Thread.sleep()替代wait()实现帧间隔控制。
• 确保UI更新相关操作（比如repaint）在EDT中执行（repaint本身线程安全，用SwingUtilities.invokeLater更规范）。

修改后的代码 #

public class MainPaint extends JFrame {

    static{ System.loadLibrary(Core.NATIVE_LIBRARY_NAME); }
    PaintPanel paintPan = null;
    
    public MainPaint() {
        setTitle("Video Panel");
        setSize(600, 600);
        setLayout(new BorderLayout());

        paintPan = new PaintPanel();
        JButton testButon = new JButton("Display Video");
        add(paintPan, BorderLayout.CENTER);
        add(testButon, BorderLayout.PAGE_END);

        testButon.addActionListener(new ActionListener() {
            @Override
            public void actionPerformed(ActionEvent arg0) {
                // 启动新线程处理视频读取，避免阻塞EDT
                new Thread(() -> {
                    try {
                        paintPan.updateGraphics(50, 50);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }).start();
            }
        });
        setVisible(true);
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    }

    public static void main(String[] args) {
        new MainPaint();
    }
}

class PaintPanel extends JPanel {

    public PaintPanel() {
       // setBackground(Color.ORANGE);
    }
    
    @Override
    public void paintComponent(Graphics g) {
        super.paintComponent(g);
        Graphics2D graphics2d = (Graphics2D)  g.create();
        // 避免空指针，image为null时不绘制
        if(image != null){
            graphics2d.drawImage(image, null, 0, 0);
        }
        graphics2d.dispose();
    }

    private BufferedImage image = null;
    public void updateGraphics(int length, int width) throws InterruptedException {
        File selectedFile = new File("C:\\image_test\\Opencvs\\data\\test.mp4");
        String filePath = selectedFile.getAbsolutePath();
        
        VideoCapture cap = new VideoCapture(filePath);
        Mat frame = new Mat();
        
        try {
            while (cap.isOpened()) { // 用isOpened判断视频是否还能读取
                if (cap.read(frame)) {
                    BufferedImage newImage = Mat2bufferedImage(frame);
                    // 在EDT中更新image并调用repaint
                    SwingUtilities.invokeLater(() -> {
                        this.image = newImage;
                        repaint();
                    });
                    Thread.sleep(100); // 控制帧间隔，替代wait
                } else {
                    break; // 读取不到帧时退出循环
                }
            }
        } finally {
            cap.release(); // 释放视频资源
        }
    }
    
    private static BufferedImage Mat2bufferedImage(Mat image) {
        MatOfByte bytemat = new MatOfByte();
        Imgcodecs.imencode(".jpg", image, bytemat);
        byte[] bytes = bytemat.toArray();
        InputStream in = new ByteArrayInputStream(bytes);
        BufferedImage img = null;
        try {
            img = ImageIO.read(in);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return img;
    }
}

额外优化说明 #

• 添加了cap.isOpened()判断视频是否有效，避免无效循环。
• 读取完帧后添加break退出循环，防止视频结束后死循环。
• 在paintComponent中增加image != null判断，避免空指针异常。
• 最后调用cap.release()释放视频捕获资源，防止内存泄漏。

内容的提问来源于stack exchange，提问作者Shailender Jain