Search code examples
javalwjglvulkanimgui

How to render image in SpaiR / imgui-java with Vulkan?

In my Vulkan Java app I create textures (it's part of Vulkan tutorial for Java https://github.com/lwjglgamedev/vulkanbook/tree/master/bookcontents)

public TextureDescriptorSet(DescriptorPool descriptorPool, DescriptorSetLayout descriptorSetLayout,
                            Texture texture, TextureSampler textureSampler, int binding) {
    try (MemoryStack stack = MemoryStack.stackPush()) {
        Device device = descriptorPool.getDevice();
        LongBuffer pDescriptorSetLayout = stack.mallocLong(1);
        pDescriptorSetLayout.put(0, descriptorSetLayout.getVkDescriptorLayout());
        VkDescriptorSetAllocateInfo allocInfo = VkDescriptorSetAllocateInfo.calloc(stack)
                .sType(VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO)
                .descriptorPool(descriptorPool.getVkDescriptorPool())

                .pSetLayouts(pDescriptorSetLayout);
        allocInfo.descriptorSetCount();

        LongBuffer pDescriptorSet = stack.mallocLong(1);
        //IntBuffer pDescriptorSet = stack.mallocInt(1); // pDescriptorSet should be LongBuffer according to LWJGL and possibly Vulkan specs
        vkCheck(vkAllocateDescriptorSets(device.getVkDevice(), allocInfo, pDescriptorSet),
                "Failed to create descriptor set");

        vkDescriptorSet = pDescriptorSet.get(0);
        // link this descriptor set with texture (the image view) and the sampler previously created
        // With this structure we combine the image view associated with the texture ad the sampler
        VkDescriptorImageInfo.Buffer imageInfo = VkDescriptorImageInfo.calloc(1, stack)
                .imageLayout(VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL)
                .imageView(texture.getImageView().getVkImageView())
                .sampler(textureSampler.getVkSampler());
        // After that we need to update the contents of the descriptor set to associate it with those resources
        VkWriteDescriptorSet.Buffer descrBuffer = VkWriteDescriptorSet.calloc(1, stack);
        descrBuffer.get(0)
                .sType(VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET)
                .dstSet(vkDescriptorSet)
                .dstBinding(binding)
                .descriptorType(VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER)
                .descriptorCount(1)
                .pImageInfo(imageInfo);

        vkUpdateDescriptorSets(device.getVkDevice(), descrBuffer, null);
    }
}

It looks exactly the same as in ImGui_ImplVulkan_AddTexture method in ImGui C++ library. Commit for this method - https://github.com/ocornut/imgui/pull/914/commits/f1f948bea715754ad5e83d4dd9f928aecb4ed1d3. This method wasn't ported to Java for now, but I use the same principles in the equivalent Java method.

ImTextureID ImGui_ImplVulkan_AddTexture(VkSampler sampler, VkImageView image_view, VkImageLayout image_layout){
VkResult err;

ImGui_ImplVulkan_InitInfo* v = &g_VulkanInitInfo;
VkDescriptorSet descriptor_set;
// Create Descriptor Set:
{
    VkDescriptorSetAllocateInfo alloc_info = {};
    alloc_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO;
    alloc_info.descriptorPool = v->DescriptorPool;
    alloc_info.descriptorSetCount = 1;
    alloc_info.pSetLayouts = &g_DescriptorSetLayout;
    err = vkAllocateDescriptorSets(v->Device, &alloc_info, &descriptor_set);
    check_vk_result(err);
}

// Update the Descriptor Set:
{
    VkDescriptorImageInfo desc_image[1] = {};
    desc_image[0].sampler = sampler;
    desc_image[0].imageView = image_view;
    desc_image[0].imageLayout = image_layout;
    VkWriteDescriptorSet write_desc[1] = {};
    write_desc[0].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET;
    write_desc[0].dstSet = descriptor_set;
    write_desc[0].descriptorCount = 1;
    write_desc[0].descriptorType = VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER;
    write_desc[0].pImageInfo = desc_image;
    vkUpdateDescriptorSets(v->Device, 1, write_desc, 0, NULL);
}

return (ImTextureID)descriptor_set;

}

So, returning to Java, if I understand it right, I can use then this descriptor_set (or in my case vkDescriptorSet value) as textureId to pass to ImGui.image that has the following structure

public static native void image(int textureID, float sizeX, float sizeY); /*
    ImGui::Image((ImTextureID)(intptr_t)textureID, ImVec2(sizeX, sizeY));
*/

As you can see ImGui.image takes int for textureId, not long. Converting it to int doesn't work because the long value of vkDescriptorSet is too big.

When I run application and try to create image using ImGui,

long img = guiRenderActivity.getTextureDescriptorSet().getVkDescriptorSet();
ImGui.imageButton((int) img, 500, 200);

it renders some default image and not mine of course enter image description here

Maybe you know other ways to upload a texture to ImGui in Java (not with OpenGL, but with Vulkan). Or maybe alternative to ImGui?

Solution

  • It seems that for now it's not possible to render an image with Vulkan in ImGui-Java. I found a bug related to this that was created not so long ago. https://github.com/SpaiR/imgui-java/issues/185

    But thanks to enesaltinkaya user on github (who opened a dedicated issue) I found that there is a way to fix it:

    1. make a copy of ImGui-Java repo and change interesintg data types to long.

    public static native void image(long textureID, float sizeX, float sizeY); /*
    ImGui::Image((ImTextureID)(intptr_t)textureID, ImVec2(sizeX, sizeY));
*/

public static native boolean imageButton(long textureID, float sizeX, float sizeY); /*
    return ImGui::ImageButton((ImTextureID)(intptr_t)textureID, ImVec2(sizeX, sizeY));
*/

//and so on

//and we should not forget this one: 

public native long getCmdListCmdBufferTextureId(int cmdListIdx, int cmdBufferIdx); /*
    return (intptr_t)IM_DRAW_DATA->CmdLists[cmdListIdx]->CmdBuffer[cmdBufferIdx].GetTexID();
*/

    2. Then rebuild ImGui-Java with command provided at their repo https://github.com/SpaiR/imgui-java?tab=readme-ov-file#how-to-build-native-libraries

    3. Use compiled classes instead of imgui-java-binding lib and and imgui-java64.dll file insted of imgui-java-natives-windows-<version> lib

    4. In code when before calling vkCmdBindDescriptorSets determine if it's your texture or fonts (default texture)

    long cmdListCmdBufferTextureId = imDrawData.getCmdListCmdBufferTextureId(i, j);
                if (cmdListCmdBufferTextureId == 0) {
                    LongBuffer descriptorSets = stack.mallocLong(1)
                            .put(0, textureDescriptorSet.getVkDescriptorSet());
                    vkCmdBindDescriptorSets(cmdHandle, VK_PIPELINE_BIND_POINT_GRAPHICS,
                            pipeline.getVkPipelineLayout(), 0, descriptorSets, null);
                } else {
                    LongBuffer descriptorSets = stack.mallocLong(1)
                            .put(0, cmdListCmdBufferTextureId);
                    vkCmdBindDescriptorSets(cmdHandle, VK_PIPELINE_BIND_POINT_GRAPHICS,
                            pipeline.getVkPipelineLayout(), 0, descriptorSets, null);
                }

    5. See results

    enter image description here

    p.s. Whoever disliked my question don't be afraid to explain your dislike. Do you find this question too noobie and a have solution? Then make everyone who stuck with this problem happy