Search code examples
c++vulkanfrustum

Wrong frustum plane when getting them from view and proj matrix

I've been having some trouble with doing a frustum culling with aabb. All frustums planes seems to be wrong and I don't know where it's comming from.

Here is the code to compute frustum planes.

std::vector<math::Vec4> BaseCamera::GetFrustumPlanes()
{
    //Matrix are Column Major
    math::Matrix4 mat = viewMatrix_ * projectionMatrix_;

    std::vector<math::Vec4> tempFrustumPlane(6);

    // Left Frustum Plane
    tempFrustumPlane[0] = mat.GetColumn(3) + mat.GetColumn(0);

    // Right Frustum Plane
    // Subtract first column of matrix from the fourth column
    tempFrustumPlane[1] = mat.GetColumn(3) - mat.GetColumn(0);

    // Top Frustum Plane
    tempFrustumPlane[2] = mat.GetColumn(3) - mat.GetColumn(1);

    // Bottom Frustum Plane
    tempFrustumPlane[3] = mat.GetColumn(3) + mat.GetColumn(1);

    // Near Frustum Plane
    tempFrustumPlane[4] = mat.GetColumn(3) - mat.GetColumn(2);

    // Far Frustum Plane
    // Subtract third column of matrix from the fourth column
    tempFrustumPlane[5] = mat.GetColumn(3) + mat.GetColumn(2);

    // Normalize plane normals (A, B and C (xyz))
    for(int i = 0; i < 6; i++) {
        const auto length = sqrt((tempFrustumPlane[i].x * tempFrustumPlane[i].x) + (tempFrustumPlane[i].y * tempFrustumPlane[i].y) + (tempFrustumPlane[i].z * tempFrustumPlane[i].z));
        tempFrustumPlane[i].x /= length;
        tempFrustumPlane[i].y /= length;
        tempFrustumPlane[i].z /= length;
        tempFrustumPlane[i].w /= length;
    }

    return tempFrustumPlane;
}

The viewMatrix is compute using this function

Matrix4 Matrix4::LookAt(const Vec3 eye, const Vec3 center, const Vec3 up)
{
    const Vec3 f((center - eye).Normalize());
    const Vec3 s(Vec3::Cross(f, up).Normalize());
    const Vec3 u(Vec3::Cross(s, f));

    Matrix4 result = Identity();
    result[0][0] = s.x;
    result[1][0] = s.y;
    result[2][0] = s.z;
    result[0][1] = u.x;
    result[1][1] = u.y;
    result[2][1] = u.z;
    result[0][2] = -f.x;
    result[1][2] = -f.y;
    result[2][2] = -f.z;
    result[3][0] = -(s * eye);
    result[3][1] = -(u * eye);
    result[3][2] = (f * eye);

    return result;
}

And the perspective if compute using this function

Matrix4 Matrix4::Perspective(
    const float fov,
    const float aspect,
    const float near,
    const float far)
{
    const float tanHalfFov = tan(fov * 0.5f);

    Matrix4 result(0);

    result[0][0] = 1.0f / (aspect * tanHalfFov);
    result[1][1] = 1.0f / tanHalfFov;
    result[2][2] = far / (near - far);
    result[2][3] = -1.0f;
    result[3][2] = -(far * near) / (far - near);

    return result;
}

Finally I'm testing aabbs against the frustum in this function

bool DrawSystem::CullAABB(
    const physics::AABB aabb,
    std::vector<math::Vec4> frustumPlanes)
{

    const float minX = aabb.centerPoint.x - aabb.extent.x;
    const float maxX = aabb.centerPoint.x + aabb.extent.x;

    const float minY = aabb.centerPoint.y - aabb.extent.y;
    const float maxY = aabb.centerPoint.y + aabb.extent.y;

    const float minZ = aabb.centerPoint.z - aabb.extent.z;
    const float maxZ = aabb.centerPoint.z + aabb.extent.z;

    auto& gizmoCommandBuffer = GraphicsEngine::Get().GetGizmoCommandBuffer();

    math::Vec3 center = Camera::Get().GetPosition() + Camera::Get().GetFront() * 2;

    // check box outside/inside of frustum
    for (int i = 0; i < 6; i++)
    {
        math::Vec3 dir{ frustumPlanes[i].x, frustumPlanes[i].y, frustumPlanes[i].z };
        if(i == 0) { //left
            gizmoCommandBuffer.SetColor(Color::red);
        } else if(i == 1) { //right
            gizmoCommandBuffer.SetColor(Color::yellow);
        }
        else if (i == 2) { //top
            gizmoCommandBuffer.SetColor(Color::blue);
        }
        else if (i == 3) { //bottom
            gizmoCommandBuffer.SetColor(Color::green);
        }
        else {
            gizmoCommandBuffer.SetColor(Color::grey);
        }
        for(int j = 0; j < 10; j++) {
            gizmoCommandBuffer.DrawWireSphere(center + (dir * j / 10 )* 0.1f, 0.01f);
        }

        if (frustumPlanes[i] * math::Vec4(minX, minY, minZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(maxX, minY, minZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(minX, maxY, minZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(maxX, maxY, minZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(minX, minY, maxZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(maxX, minY, maxZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(minX, maxY, maxZ, 1.0f) > 0.0f) {
            continue;
        }
        if (frustumPlanes[i] * math::Vec4(maxX, maxY, maxZ, 1.0f) > 0.0f) {
            continue;
        }
        return false;
    }
    return true;
}

In the engine I've made a test. The aabbs are drawn (blue considered outside the camera, red inside) :enter image description here

The cross seems to work but as soon as I'm turning the camera, the cross is wrong.

AABB are working because they are drawn correctly.

If you need anything else just ask

Solution

  • I found a working solution in this repos : https://github.com/EQMG/Acid/blob/master/Sources/Physics/Frustum.cpp