Search code examples
c++openglvertex-array-object

glDrawElements crashes in the driver | debugging hints

Short version: How can a crash (bad memory access exception/nullptr exception) inside glDrawElements/glDrawElementsInstanced be debugged?

Long version: You have a path of OpenGL rendering code which uses VAOs, and commits rendering thru calling glDrawElements or glDrawElementsInstanced. That codepath works correctly most of the time. We are talking "editor code", which means: the data may be any geometry and is highly likely to change frequently.

But sometimes after commiting reproducible data changes it simply crashes within the glDrawElements* driver code (i.e. glDrawElements is called, the function parameters are OK, the crash happens inside glDrawElements).

How could you proceed debugging this problem?

P.S.:

  • self-answered question: all research efforts went into the answer!
  • this is targeted at editor code. For simple demonstrations such crashes are mostly caused by the coder not understanding the requirements of glDrawElements correctly, and as such the codepath will either work, or not - in those cases, see:
Solution

  • At first it should be clear what may cause crashes within the driver. In most cases that's bad memory access.

    What may cause access of bad memory inside the driver?

    • GL_ELEMENT_ARRAY_BUFFER-binding changed somehow (thus the parameters given to glDrawElements may cause access beyond the memory of that object)
    • GL_ELEMENT_ARRAY_BUFFER's content has changed, possibly referencing uninitialized/non-existent vertex data (VBO access out of bounds)
    • any of the associated GL_ARRAY_BUFFER objects' data was altered, thus not containing referenced vertex data anymore (VBO access out of bounds)
    • or even more of such changes. Access violations inside glDrawElements* mostly mean that any of the objects bound within the VAO state was accessed out of bounds.

    Without extra debugging code these access violations are very hard to catch. I suggest inserting debug-output right before calling glDrawElements*. The Debug output should query all the bindings and info available, so you can compare the settings "when it works" with "when it crashes" and figure out what to look for, next.

    My debug function looks like this:

    void debugVAOState(std::string baseMessage)
{
    baseMessage.append( " ... querying VAO state:\n" );
    int vab, eabb, eabbs, mva, isOn( 1 ), vaabb;
    glGetIntegerv( GL_VERTEX_ARRAY_BINDING, &vab );
    glGetIntegerv( GL_ELEMENT_ARRAY_BUFFER_BINDING, &eabb );
    glGetBufferParameteriv( GL_ELEMENT_ARRAY_BUFFER, GL_BUFFER_SIZE, &eabbs );

    baseMessage.append( "  VAO: " + std::to_string( vab ) + "\n" );
    baseMessage.append( "  IBO: " + std::to_string( eabb ) + ", size=" + std::to_string( eabbs )  + "\n" );

    glGetIntegerv( GL_MAX_VERTEX_ATTRIBS, &mva );
    for ( unsigned i = 0; i < mva; ++i )
    {
        glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_ENABLED, &isOn );
        if ( isOn )
        {
            glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_BUFFER_BINDING, &vaabb );
            baseMessage.append( "  attrib #" + std::to_string( i ) + ": VBO=" + std::to_string( vaabb ) + "\n" );
        }
    }
    OutputDebugString( baseMessage.c_str() );
}

    It's still simple and outputs only the most valuable information to be able to see if above mentioned bindings have changed somehow. But this helped me finding numerous crashes that came from aggressive OpenGL optimization.