Raytracing normal mapping

Question

I am currently working on a raytracer and I just "bumped" in an issue. I implemented texture mapping for planes, cylinders and spheres and it's working pretty well... Except for the normal map part. Here is what I have, the in-world position and the in-world normals of each pixel : world-space normals. And some tangent-space normal map (the usual normal map).

I can't seem to figure out how to convert the tangent-space normals to world-space. I have tried using a "TBN" matrix but the normals are off : normal map projected normals.

And here is my code to compute the new normal :

VEC3    t = vec3_cross(worldnormal, new_vec3(0.0, 1.0, 0.0)); 
VEC3    b;
if (!vec3_length(t))
    t = vec3_cross(worldnormal, new_vec3(0.0, 0.0, 1.0));
t = vec3_normalize(t);
b = vec3_normalize((vec3_cross(worldnormal, t)));
VEC3    map_n = vec3_normalize(get_texture_color(normal_map, texcoords));
MAT3    tbn = new_mat3(t, b, worldnormal);
worldnormal = vec3_normalize(mat3_mult_vec3(tbn, map_n));

get_texture_color() returns the normal map's texture color divided by 255.f

Answer 1

So ! I just found what was wrong with my normal mapping ! After trying to use a constant {0, 0, 1} normal to see if my TBN matrix was right (and it was) I just found out that normal map's tangent space normals had to be "converted"

So the right code is :

VEC3    t = vec3_cross(worldnormal, new_vec3(0.0, 1.0, 0.0)); 
VEC3    b;
if (!vec3_length(t))
    t = vec3_cross(worldnormal, new_vec3(0.0, 0.0, 1.0));
t = vec3_normalize(t);
b = vec3_normalize((vec3_cross(worldnormal, t)));
VEC3    map_n = vec3_normalize(get_texture_color(normal_map, texcoords));
//map_n * 2 - 1
map_n = vec3_sub(vec3_scale(map_n, 2), new_vec3(1, 1, 1));
MAT3    tbn = new_mat3(t, b, worldnormal);
worldnormal = vec3_normalize(mat3_mult_vec3(tbn, map_n));

So close, yet so far ! Here is how it looks now, looking pretty good IMHO !

New (propper) normal mapping using TBN matrix !

With a better material for middle pillar ! (not the other "sort of" water)