I am using Qt version 6.2.4, and I am making a simple video processing software. I need to take every frame of the video and perform a series of OpenCV operations on that frame. Here is my code to get each frame of the image:
Here is some explanation of my code:
processVideoFrame
slot function is used to get each frame of the video in response to the signal.std::vector
to store every frame of the cv::Mat type, where my QImage was converted to cv::Mat and then push_back
into matspositionChanged
signal, if the time reaches the end of the video, it indicates that the video is finished playing, and then the showMats function is called to show each frame that I have stored beforeTo make sure it doesn't end immediately, I've added cv::waitKey
to pause it. In theory, my showMats
function should reproduce the video by going through each Mat, but it's reporting an error at imshow.
class VideoFrameProcessor : public QObject
{
public:
VideoFrameProcessor(QLabel* label) : videoLabel(label) {
mats.clear();
videoWrite.open("video/output.mp4", cv::VideoWriter::fourcc('M', 'J', 'P', 'G'), 25, cv::Size(1566, 1080), true);
}
~VideoFrameProcessor() {
videoWrite.release();
}
void showMats() {
for (auto& frame : mats) {
if (frame.empty() || cv::waitKey(10) == 27) {
break;
}
//cv::imshow("1", frame); //When I use it, I get an error
}
}
public slots:
void processVideoFrame(const QVideoFrame& frame)
{
if (frame.isValid())
{
QImage image = frame.toImage();
height = image.height();
width = image.width();
cv::Mat mt(image.height(), image.width(), CV_8UC4,image.bits(),image.bytesPerLine());
mats.push_back(mt);
cv::imshow("1", mt); // When I use it, it's correct
//
//videoWrite.write(mt);
}
}
private:
cv::VideoWriter videoWrite;
std::vector<cv::Mat> mats;
int height, width;
QLabel* videoLabel;
};
class VideoWidget : public QMainWindow
{
public:
VideoWidget(QWidget* parent = nullptr) : QMainWindow(parent)
{
resize(640, 480);
QLabel* videoLabel = new QLabel(this);
videoLabel->show();
mediaPlayer = new QMediaPlayer(this);
videoWidget = new QVideoWidget(this);
mediaPlayer->setVideoOutput(videoWidget);
videoWidget->show();
// 设置要播放的视频文件
mediaPlayer->setSource(QUrl::fromLocalFile("video/cat.mp4"));
// 播放视频
mediaPlayer->play();
// 创建QVideoSink并连接到videoFrameChanged信号
videoSink = new QVideoSink(this);
frameProcessor = new VideoFrameProcessor(videoLabel);
QObject::connect(videoSink, &QVideoSink::videoFrameChanged, frameProcessor, &VideoFrameProcessor::processVideoFrame);
mediaPlayer->setVideoSink(videoSink);
connect(mediaPlayer, &QMediaPlayer::positionChanged, this, [=](qint64 val) {
if (val == mediaPlayer->duration()) {
//Show a series of Mats
frameProcessor->showMats();
}
});
this->setCentralWidget(videoWidget);
}
private:
QMediaPlayer* mediaPlayer;
QVideoWidget* videoWidget;
QVideoSink* videoSink;
VideoFrameProcessor* frameProcessor;
};
int main(int argc, char* argv[])
{
QApplication app(argc, argv);
VideoWidget w;
w.show();
return app.exec();
}
This is an error image after my imshow And output the error code: -1073741819。
As far as I know: std::vector data does not automatically disappear. The data will remain until your program exits or you explicitly release them.
So I also used an imshow in the processVideoFrame
function to display the image at any time, and this time it worked.
As you can see, this video is of a cat:
This is the correct video frame
I want to know why imshow will report errors in different positions?
You get segmentation fault because you use cv::Mat
constructor version which only stories a reference to already allocated data i.e. the data created and owned by QImage
(we can say that image data are shared between QImage
and cv::Mat
).
data Pointer to the user data. Matrix constructors that take data and step parameters do not allocate matrix data. Instead, they just initialize the matrix header that points to the specified data, which means that no data is copied. This operation is very efficient and can be used to process external data using OpenCV functions. The external data is not automatically deallocated, so you should take care of it.
Becuase QImage
is local, when the code goes out of the scope local QImage
is destroyed and cv::Mat
holds dangling pointer.
QImage image = frame.toImage();
height = image.height();
width = image.width();
cv::Mat mt(image.height(), image.width(), CV_8UC4,image.bits(),image.bytesPerLine());
mats.push_back(mt);
to make the code working, call clone
:
mats.push_back(mt.clone());
by clong
the deep copy of data shared by mat/qimage
is done.