How to avoid distance ordering in large scale billboard rendering with transparency

Question

Setting the scene:

I am rendering a height map (vast non-transparent surface) with a large amount of billboards on it (typically grass, flowers and so on).

The billboards thus have a mostly transparent color map applied, with only a few pixels colored to produce the grass or leaf shapes and such. Note that the edges of those shapes use a bit of transparency gradient to make them look smoother, but I have also tried with basic, binary color/transparent textures.

Pseudo rendering code goes like so:

map->render();
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
wildGrass->render();
glDisable(GL_BLEND);

Where the wildGrass render instruction renders multiple billboards at various locations in a single OGL call.

The issue I am experiencing has to do with transparency and the fact that billboards apparently hide each-other, even on their transparent area. However the height-map solid background is correctly displayed on those transparent parts.

Here's the glitch:

• Left is with an explicit fragment shader discard on fully transparent pixels
• Right is without the discard, clearly showing the billboard's flat quad

Based on my understanding of OGL blending and some reading, it seems that the solution is to have a controlled order of rendering, starting from the most distant objects to the closest, so that the color buffer is filled properly in the end.

I am desperately hoping that there is another way... The ordering here would typically vary depending on the point of view, which means it has to be applied in-real-time for each frame. Plus the nature of those particular billboards is to be produced in a -very large- number... Performance alert!

Any suggestions or is my approach of blending wrong?

Answer 1

Did not work for me:

@httpdigest's suggestion to disable depth buffer writing:

It worked essentially for billboards with the same texture (and possibly a specific type of texture, like wild grass for instance), because the depth inconsistencies are not visually noticeable - however introducing another texture, say a flower with drastically different colours, will immediately highlights those mistakes.

Solution:

@Rabbid76's suggestion to use not-semi-transparent textures with multi-sampling & anti-aliasing: I believe this is the way to go for best visual effect with reasonably low cost on performance.

Alternative solution:

I found an intermediary solution which is probably the cheapest in performance to the expense of quality. I still use textures with gradient transparent edges, but instead of discarding fully transparent pixels, I introduced a degree of tolerance, for example any pixel with alpha < 0.6 is discarded - the value is found experimentally to find the right balance.

With this approach:

• I still perform depth tests, so output is correct
• Textures quality is degraded/look less smooth - but reasonably so
• The glitches on semi-transparent pixels still appear - but are nearly not noticeable
• See capture below

So to conclude:

• My solution is a cheap and simple approximation giving less smooth visual result
• Best result can be obtained by rendering all the billboards to a multi-sampled texture resolve with anti-aliasing and finally output the result in a full screen quad. There are probably to ways to do this:
  • Either rendering the map first and use the resulting depth buffer when rendering the billboards
  • Or render both the map and billboards on the multi-sampled texture
• Note that the above approaches are both meant to avoid having to distance-base sort a large number of billboards - but this remains a valid option and I have read about storing billboard locations in a quad tree for quick access.