how to use Tango3DR_updateTexture?

Question

I am trying to make 3DR texturing but it always use only vertex colors in texture.

On every frame I store frame as PNG:

    RGBImage frame(t3dr_image, 4);
    std::ostringstream ss;
    ss << dataset_.c_str();
    ss << "/";
    ss << poses_.size();
    ss << ".png";
    frame.Write(ss.str().c_str());
    poses_.push_back(t3dr_image_pose);
    timestamps_.push_back(t3dr_image.timestamp);

---

In the method Save I am trying to process texturing:

1) I extract full mesh from context

    Tango3DR_Mesh* mesh = 0;
    Tango3DR_Status ret;
    ret = Tango3DR_extractFullMesh(t3dr_context_, &mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

2) Create texturing context using extracted mesh

    Tango3DR_ConfigH textureConfig;
    textureConfig = Tango3DR_Config_create(TANGO_3DR_CONFIG_TEXTURING);
    ret = Tango3DR_Config_setDouble(textureConfig, "min_resolution", 0.01);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    Tango3DR_TexturingContext context;
    context = Tango3DR_createTexturingContext(textureConfig, dataset.c_str(), mesh);
    if (context == nullptr)
        std::exit(EXIT_SUCCESS);
    Tango3DR_Config_destroy(textureConfig);

3) Call Tango3DR_updateTexture with data I stored before (this does not work)

    for (unsigned int i = 0; i < poses_.size(); i++) {
        std::ostringstream ss;
        ss << dataset_.c_str();
        ss << "/";
        ss << i;
        ss << ".png";
        RGBImage frame(ss.str());
        Tango3DR_ImageBuffer image;
        image.width = frame.GetWidth();
        image.height = frame.GetHeight();
        image.stride = frame.GetWidth() * 3;
        image.timestamp = timestamps_[i];
        //data are for sure in this format
        image.format = TANGO_3DR_HAL_PIXEL_FORMAT_RGB_888;
        image.data = frame.GetData();
        ret = Tango3DR_updateTexture(context, &image, &poses_[i]);
        if (ret != TANGO_3DR_SUCCESS)
            std::exit(EXIT_SUCCESS);
    }

4) Texturize mesh

    ret = Tango3DR_Mesh_destroy(mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    mesh = 0;
    ret = Tango3DR_getTexturedMesh(context, &mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

5) Save it as OBJ (in the result texture are only data from vertex colors, why?)

    ret = Tango3DR_Mesh_saveToObj(mesh, filename.c_str());
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    ret = Tango3DR_destroyTexturingContext(context);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    ret = Tango3DR_Mesh_destroy(mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

All methods returned TANGO_3DR_SUCCESS.

Full code here: https://github.com/lvonasek/tango

Answer 1

Thanks for reaching out and providing the detailed code breakdown.

The error is on our end - the library currently doesn't support RGB texture inputs. It assumes YUV for all input images. I've opened a ticket to track this bug and we'll fix it for the next release, by allowing RGB input and providing better return values for invalid image formats.

Edit: Found another bug on our end. The API states image_pose should be the pose of the image, but our implementation actually expects the pose of the device. I've opened a bug, and this will be fixed in next release (release-H).

You can try working around this for now by passing in the device pose without multiplying the device-to-camera extrinsic calibration, although of course that's just a temp bandaid.