Displaying Qt Quick content inside OpenSceneGraph scene

Question

I'd like to show my Qt Quick content on a virtual screen inside my OpenSceneGraph scene.

The approach I'm using right now is highly inefficient:

1. Render Qt Quick to the offscreen surface using FBO (FrameBufferObject)
2. Download pixels with QOpenGLFramebufferObject::toImage()
3. Upload pixels to the OSG

So it's GPU-CPU-GPU transfer. Source code

A proper solution should somehow utilize existing FBO and be able to transfer data solely inside the GPU.

There are two options exist:

1. Create FBO on the Qt side and use its texture on the OSG side
2. Create FBO on the OSG side and feed it to the Qt Quick renderer

The Qt part is OK. And I'm completely lost with the OSG. Could anyone provide me with some pointers?

Answer 1

Finally made it.

General idea:

1. Render QtQuick to texture using FBO - there are some examples over the internet available.

2. Use this texture inside OpenSceneGraph

The whole thing includes several tricks.

Context Sharing

To perform certain graphics operations, we must initialize OpenGL global state, also known as context.

When a texture is created, context stores it's id. Ids are not globally unique, so when another texture is created within another context, it may get the same id, but with different resource behind it.

If you just pass your texture's id to another renderer (operating within different context), expecting it to show your texture, you end up showing another texture or black screen or crash.

The remedy is context sharing, which effectively means sharing ids.

OpenSceneGraph and Qt abstractions are not compatible, so you need to tell OSG not to use its own context abstraction. This is done by calling setUpViewerAsEmbeddedInWindow

Code:

OsgWidget::OsgWidget(QWidget* parent, Qt::WindowFlags flags)
    : QOpenGLWidget(parent, flags)
    , m_osgViewer(new osgViewer::Viewer)
{
    setFormat(defaultGraphicsSettings());

    // ...

    m_osgGraphicsContext = m_osgViewer->setUpViewerAsEmbeddedInWindow(x(), y(), width(), height());
}

// osg::ref_ptr<osgViewer::GraphicsWindowEmbedded> m_osgGraphicsContext;

From now on, the existing QOpenGLContext instance will be used as an OpenGL context for OSG rendering.

You will need to create another context for QtQuick rendering and set them shared:

void Widget::initializeGL()
{
    QOpenGLContext* qmlGLContext = new QOpenGLContext(this);
    // ...
    qmlGLContext->setShareContext(context());
    qmlGLContext->create();
}

Remember, there can be only one active context at a time.

Your scheme is:

0. create osg::Texture out of QOpenGLFrameBufferObject::texture()
1. make QtQuick context active
2. render QtQuick to texture
3. make primary (OSG) context active
4. render OSG
5. goto 1

Making of a proper osg::Texture

Since OSG and Qt API are incompatible you barely can link QOpenGLFrameBufferObject to osg::Texture2D as it is.

QOpenGLFrameBufferObject has QOpenGLFrameBufferObject::texture() method which returns opengl texture id, but osg::Texture manages all openGL stuff on its own.

Something like osg::Texture2D(uint textureId); could help us but it just doesn't exist.

Let's make one by ourselves.

osg::Texture is backed by osg::TextureObject which stores OpenGL texture id and some other data as well. If we construct osg::TextureObject with a given texture id and pass it to osg::Texture, the latter will use it as its own.

Code:

void Widget::createOsgTextureFromId(osg::Texture2D* texture, int textureId)
{
    osg::Texture::TextureObject* textureObject = new osg::Texture::TextureObject(texture, textureId, GL_TEXTURE_2D);
    textureObject->setAllocated();

    osg::State* state = m_osgGraphicsContext->getState();
    texture->setTextureObject(state->getContextID(), textureObject);
}

---

Complete demo project here