I'm trying to implement transparency in OpenGL and from what I've read it's necessary to first render all opaque objects and then render the transparent objects in the correct order.
My issue is how do I go about separating opaque from transparent objects in my scene graph so that I can render the opaque objects first. My scene graph consists of a bunch of nodes which can have entities attached to them, and each entity can be composed of several meshes with different materials, each with one or more textures.
If I load a scene into my graph I need to know which materials are partially or completely transparent which means I need to check if the loaded textures have any alpha values smaller than 1. I'm currently using Assimp to handle loading the models/scenes and SOiL to read the textures and I haven't found any simple way to separate transparent materials from opaque ones.
This probably has a really simple solution because I haven't found anyone else with the same question, but I'm still starting with OpenGL and I have been stuck on this matter for the past hours.
How is transparency normlly done and how are opaque objects separated from partially or fully transparent ones so that they may be rendered first?
For most rendering method, you don't strictly have to separate the opaque from the transparent objects. If you think about it, transparency (or opacity) is a continuous quality. In OpenGL, the alpha component is typically used to define opacity. Opaque objects have an alpha value of 1.0, but this is just one value in a continuous spectrum. Methods that can correctly handle all alpha values will not suddenly fail just because the alpha value happens to be 1.0.
Putting it differently: Is an object with alpha value 0.9 opaque? What if the alpha value is 0.99, can you justify treating it differently from alpha value 1.0? It is really all continuous, and not a binary decision.
With that said, there are reasons why it's common to treat opaque objects differently. The main ones I can think of:
Since the non-opaque objects have to be sorted for common simple transparency rendering methods, you can save work by sorting only the non-opaque objects. Sorting is not cheap, and you reduce processing time this way. For most of these methods, you get perfectly fine results by sorting all objects, at the price of being less efficient.
Often times, objects cannot be sorted perfectly, or doing so is at least not easy. An obvious problem is when objects overlap, but the challenges go beyond that case (see my answer here for some more in depth illustration of problematic cases: Some questions about OpenGL transparency). In these cases, you get artifacts from incorrect sorting. By drawing opaque objects with depth testing enabled, you avoid the possibility of artifacts for those objects, and reduce the overall occurrence of noticeable artifacts.
Your situation of not knowing which objects contain transparency seems somewhat unusual. In most cases, you know which objects are opaque because you're in control of the rendering, and the content. So having an attribute that specifies if an object is opaque normally comes pretty much for free.
If you really have no way to define which objects are opaque, a couple of options come to mind. The first one is that you sort all objects, and render them in order. Based on the explanation above, you could encounter performance or quality degradation, but it's worth trying.
There are methods that can render with transparency without any need for sorting, or separating opaque and transparent objects. A simple one that comes to mind is alpha-to-coverage. Particularly if you render with MSAA anyway, it results in no overhead. The downside is that the quality can be mediocre depending on the nature of your scene. But again, it's worth trying.
You can find a basic explanation of alpha-to-coverage, as well as some other simple transparency rendering methods, in my answer to this question: OpenGL ES2 Alpha test problems.
There are more advanced transparency rendering methods that partly rely on recent hardware features. Covering them goes beyond the scope of a post here (and also largely beyond my knowledge...), but you should be able to find material by searching for "order independent transparency".