from QCamera or QVideoWidget get video bytes

How can I get an ARRAY OF BYTES (frame) from QCamera or QVideoWidget?

I successfull capture video from camera and audio from microphone and now I want send them to socket TCP channel for creating a simple video/audio call software.

What I've tried:

I have reimplemented QAbstractVideoSurface methods,
but "present" method isn't invocked. What is wrong?
How Can I use this class with QCameraViewfinder?

Here is my code:


#include <QtMultimedia/QAbstractVideoSurface>
class QMyAbstractVideoSurface :
    public QAbstractVideoSurface
{
    Q_OBJECT
public:
    explicit QMyAbstractVideoSurface(QObject* parent = 0);
    ~QMyAbstractVideoSurface();

    QList<QVideoFrame::PixelFormat> supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const;

    bool present(const QVideoFrame& frame);

    bool start(const QVideoSurfaceFormat& format);

    void stop();
};

#include "QMyAbstractVideoSurface.h"
#include <QDebug>

QMyAbstractVideoSurface::QMyAbstractVideoSurface(QObject* parent) {
}

bool QMyAbstractVideoSurface::start(const QVideoSurfaceFormat& format) {
    return QAbstractVideoSurface::start(format);

}

void QMyAbstractVideoSurface::stop() {
    QAbstractVideoSurface::stop();
}

bool QMyAbstractVideoSurface::present(const QVideoFrame& frame) {

    QVideoFrame cloneFrame(frame);
    QByteArray a((char*)cloneFrame.bits());

    qDebug() << "present";
    qDebug() << a.toBase64();

    return true;
}

QList<QVideoFrame::PixelFormat> QMyAbstractVideoSurface::
supportedPixelFormats(QAbstractVideoBuffer::HandleType handleType) const
{
    Q_UNUSED(handleType);
    return QList<QVideoFrame::PixelFormat>()
        << QVideoFrame::Format_ARGB32
        << QVideoFrame::Format_ARGB32_Premultiplied
        << QVideoFrame::Format_RGB32
        << QVideoFrame::Format_RGB24
        << QVideoFrame::Format_RGB565
        << QVideoFrame::Format_RGB555
        << QVideoFrame::Format_ARGB8565_Premultiplied
        << QVideoFrame::Format_BGRA32
        << QVideoFrame::Format_BGRA32_Premultiplied
        << QVideoFrame::Format_BGR32
        << QVideoFrame::Format_BGR24
        << QVideoFrame::Format_BGR565
        << QVideoFrame::Format_BGR555
        << QVideoFrame::Format_BGRA5658_Premultiplied
        << QVideoFrame::Format_AYUV444
        << QVideoFrame::Format_AYUV444_Premultiplied
        << QVideoFrame::Format_YUV444
        << QVideoFrame::Format_YUV420P
        << QVideoFrame::Format_YV12
        << QVideoFrame::Format_UYVY
        << QVideoFrame::Format_YUYV
        << QVideoFrame::Format_NV12
        << QVideoFrame::Format_NV21
        << QVideoFrame::Format_IMC1
        << QVideoFrame::Format_IMC2
        << QVideoFrame::Format_IMC3
        << QVideoFrame::Format_IMC4
        << QVideoFrame::Format_Y8
        << QVideoFrame::Format_Y16
        << QVideoFrame::Format_Jpeg
        << QVideoFrame::Format_CameraRaw
        << QVideoFrame::Format_AdobeDng;
}

QMyAbstractVideoSurface::~QMyAbstractVideoSurface()
{}

Solution

You are probably looking for a QVideoSink: https://doc-snapshots.qt.io/qt6-dev/qvideosink.html.

The QVideoSink class can be used to retrieve video data on a frame by frame basis from Qt Multimedia.

It is supposed to be set on the QMediaCaptureSession: https://doc-snapshots.qt.io/qt6-dev/qmediacapturesession.html#setVideoSink.

For Qt5 you have QAbstractVideoSurface https://doc.qt.io/qt-5/qabstractvideosurface.html or QVideoProbe https://doc.qt.io/qt-5/qvideoprobe.html.