Search code examples
c#unity-game-enginenormalsprocedural-generation

How do I determine if i just calculated the normal in the correct direction?

To explain the context, I procedurally generate buildings on unity in C#. I create a mesh and fill in the vertices and triangles, then I calculate the normals of the different points. I have several hundreds of buildings that are generated, but some faces are not in the right direction, the normals point inwards instead of outwards.

The normals are good

enter image description here

The normals are bad

enter image description here

To calculate the normals I create a Plane with the different vertex that make up the triangle, and then I retrieve the normal of the Plane. I also tested the cross product that gives the same result.

Plane plane = new Plane(v1, v2, v3);
normals.Add(plane.normal);

How i generate Triangle ?

enter image description here

Ex: I make a for loop on the points at the base of the building.

 vectors.Add(v1);
 vectors.Add(v2);
 vectors.Add(v3); //v3 = v1+height
 vectors.Add(v4); //v4 = v2+height

 // index values
 int idx1, idx2, idx3, idx4;
 idx4 = vectors.Count - 1;
 idx3 = vectors.Count - 2;
 idx2 = vectors.Count - 3;
 idx1 = vectors.Count - 4;

 // Triangle 1
 indices.Add(idx1);
 indices.Add(idx3);
 indices.Add(idx2);
...
mesh.triangles = indices.ToArray();

So my question is: How to determine if the normal is in the right direction (inside or outside)? If I can determine that, I can then flip the normal and normally it will work.

Solution

  • I do not totally understand your question, but for simple shapes a simple algorithm is ..

    • just take the "center" of the object (the CG is fine) and make a vector from there to your vertex or triangle. That direction is "outside". If you're pointing over 90° away from that, you're pointing in to the guts of the object

    the next more complicated approach ...

    • move along your normal say "one meter". call that point "test point". you should now be "outside" your building, correct?

    • now you just need to check if testPoint is inside or outside your 3D shape.

    • in short, to determine if you are inside or outside a 3D object, you just cast a ray and count how many times you intersect with the walls. if odd, you're inside, if even you're outside.

    You can google hundreds of discussions on this on the www, example https://stackoverflow.com/a/63572837/294884

    Note that there are many, many variations on this, and many problems too. In some cases you start from "just inside" your normal (ie go backwards a little); sometimes it's better to start from the middle of the object (if such a thing is knowable); sometimes there are issues about how the ray casting system works in edge cases (like "right on" the surface, which is what your vertex is)

    the first solution seems to me to be the right one

    It may be. Sometimes it is useless. It depends on the nature of your shapes.

    I wanted to avoid having to do raycast ..

    Can assure you - you will have to constantly raycast during any construction process! If you wish to "avoid" casting, set the idea aside. You will be casting until you are sick of it :)

    I wanted to avoid having to do raycast to limit the generation time

    The good news is this is totally incorrect. Casting is extremely trivial and a very minor burden compared to everything going on in a 3D scene. When you play any 3D scene, casts are being made 100s and 1000s of times each frame.

    but if I have two buildings that are close to each other, the ray will find the wall but from the second building so I couldn't determine the normal is "inside" or "outside", right?

    TBC in the simple system I described. You would certainly ONLY cast against that same building that you are working on! So that's the way to go.

    But indeed, unrelated to what you're asking about. Say you're in a "city". You can indeed cast through "all walls" and the parity (ie: odd or even) will let you know if you are indoors or out. This has problems though, relating to edges, corners etc. But in some case it is relevant to that problem.

    As mentioned on this answer https://stackoverflow.com/a/63593282/294884 ultimately you can investigate convex hulls and more. It's a big topic!

    Note that often when you build dynamic buildings/shapes, you havce a data structure of each "wall" and indeed it knows which side is in and which side out. When you do that first, it's then easy to draw the triangles the correct way. It's a big topic!