Search code examples
shaderhlsldirect3ddxc

HLSL 6+ uniform variables and compilation

I'm coming from a DX9 background, using the now outdated FX framework, where uniform variables can be used as a sort of preprocessor value that changes the compilation of a shader. This allowed the developer to easily and effortlessly compile a single shader into many versions of itself, where each accepted varying values of each uniform variable. Then these shaders were automatically selected at runtime based on data sent to the API. This was extremely useful when writing shaders that dramatically change based on the number of lights or bones, for example.

I'm now trying to build a non-FX system, using straight HLSL 6 with a custom D3D12 game engine, and I'm having trouble understanding what the point of uniform variables are, if not for the purpose of generating multiple shaders from one source code? All of the information I've been able to find makes it sound as though a uniform variable is just something that needs supplied at runtime, which indicates that it would not modify the compiled code.

I'm currently using the most up to date dxc compiler with Visual Studio 2022, and cannot see how it would be possible to define any sort of multi-shader-output system that relies on uniform variables, or even macros for that matter. It seems like one project file equals one shader output, period. Is there something I may be missing here? It doesn't seem like this would be useful for anything other than the most trivial shader, or perhaps shaders that never make use of uniform values.

I'm considering writing my own runtime compiler that uses dxc at runtime, which could programmatically substitute uniform variables with each possible value right before compiling, then output that as a separate *.cso. However, coming up with an efficient way to lookup which shader is needed based on non-int values, like floating point values, seems like it would be a nightmare, requiring some type of hash table. Add to that the fact that there can be multiple uniform values for one shader, and it seems like it would become very complicated. To quickly connect a specific set of inputs to a specific compiled shader at runtime, at first thought, would require something like..

csoObject FindCompiledShader(name,arg1,arg2,arg3,...)
{
    auto hash_index = Hash(arg1*S1) + Hash(arg2*S2) + Hash(arg3*S3) + ...;
    int table_index = hash_index % size_of_table;
    return Table[table_index].SearchFor(name,arg1,arg2,arg3,...);
}

And that would need to happen any time a dynamic value changes per rendered element. This seems like it could get sticky pretty quickly. Especially if there are many elements that change uniform values every frame, which seems pretty common.

Could anyone help me understand how this sort of thing is handled these days? Am I just not understanding how uniform variables work? Or is there some easier way to manage it? There doesn't seem to be much information out there about compiling HLSL in general. I'm assuming because everyone uses a popular game engine these days.

Solution

  • Generally the same shader source file is compiled many times to generate various permutations of the HLSL shader. For example, in the DirectX Tool Kit for DX12, the EnvironmentMapEffect.fx shader file uses some uniform parameters for a function:

    VSOutputTxEnvMap ComputeEnvMapVSOutput(VSInputNmTx vin, float3 normal, uniform bool useFresnel, uniform int numLights)
{
...
}

    That is invoked four different ways:

    // Vertex shader: basic.
[RootSignature(DualTextureRS)]
VSOutputTxEnvMap VSEnvMap(VSInputNmTx vin)
{
    return ComputeEnvMapVSOutput(vin, vin.Normal, false, 3);
}

[RootSignature(DualTextureRS)]
VSOutputTxEnvMap VSEnvMapBn(VSInputNmTx vin)
{
    float3 normal = BiasX2(vin.Normal);

    return ComputeEnvMapVSOutput(vin, normal, false, 3);
}

// Vertex shader: fresnel.
[RootSignature(DualTextureRS)]
VSOutputTxEnvMap VSEnvMapFresnel(VSInputNmTx vin)
{
    return ComputeEnvMapVSOutput(vin, vin.Normal, true, 3);
}

[RootSignature(DualTextureRS)]
VSOutputTxEnvMap VSEnvMapFresnelBn(VSInputNmTx vin)
{
    float3 normal = BiasX2(vin.Normal);

    return ComputeEnvMapVSOutput(vin, normal, true, 3);
}

    The compilation is then done with this batch file:

    call :CompileShader%1 EnvironmentMapEffect vs VSEnvMap
call :CompileShader%1 EnvironmentMapEffect vs VSEnvMapBn
call :CompileShader%1 EnvironmentMapEffect vs VSEnvMapFresnel
call :CompileShader%1 EnvironmentMapEffect vs VSEnvMapFresnelBn
call :CompileShader%1 EnvironmentMapEffect vs VSEnvMapPixelLighting
call :CompileShader%1 EnvironmentMapEffect vs VSEnvMapPixelLightingBn

call :CompileShader%1 EnvironmentMapEffect ps PSEnvMap
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapNoFog
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpecular
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpecularNoFog
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapPixelLighting
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapPixelLightingNoFog
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapPixelLightingFresnel
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapPixelLightingFresnelNoFog

call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpherePixelLighting
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpherePixelLightingNoFog
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpherePixelLightingFresnel
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapSpherePixelLightingFresnelNoFog

call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapDualParabolaPixelLighting
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapDualParabolaPixelLightingNoFog
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapDualParabolaPixelLightingFresnel
call :CompileShader%1 EnvironmentMapEffect ps PSEnvMapDualParabolaPixelLightingFresnelNoFog

...

:CompileShaderdxil
set dxc=%PCDXC% "%1.fx" %FXCOPTS% /T%2_6_0 /E%3 "/Fh%CompileShadersOutput%\%1_%3.inc" "/Fd%CompileShadersOutput%\%1_%3.pdb" /Vn%1_%3
echo.
echo %dxc%
%dxc% || set error=1
exit /b

    For the full shader source and compile script, see GitHub.

    The legacy Effects system did all these permutations for you automatically. With DirectX 12, the root signature management makes it hard to make a fully general solution that's also efficient. The need to match root signatures also often results in additional permutations of the same shaders being compiled for different contexts.