I'm trying to create a WebGL shader that can both output solid rectangles as well as hollow rectangles (with a fixed border width) within the same draw call, and so far, the best way I've thought of how to do it is as follows:
In the vertex shader, send in a uniform value uniform float borderWidth
and then inside the fragment shader, I need a coordinate space that is x = [ 0, 1] and y = [0, 1] where x=0 when we are the leftmost, and y=0 when we are at the topmost of the shape's borders, or something like that. After I have that then drawing the lines is straightforward and I can figure it out from there, I can use something like:
1a - Have a smooth step from the fragment's x=0 coordinate to x=borderWidth for the vertical lines and x=1-borderWidth to x=1 for the vertical lines
1b - Something similar for the horizontal lines and the y coordinate
The problem I'm facing is I can't create that coordinate space. I tried using gl_FragCoord but I think it's undefined for shapes rendering in TRIANGLES mode. So I'm kinda lost. Anyone have any suggestions?
gl_FragCoord is never undefined, it is the position of the fragment in the output buffer (like your screen), if you're rendering to the center of a FullHD screen gl_FragCoord would be vec3(940,540,depth), however this data is of no use for what you're trying to do.
What you describe sounds like you need barycentric coordinates that you have to define as additional attributes next to your vertex positions, then pass through to the fragment shader as varyings so they're linearly interpolated across the vertices. If you render non-indexed geometry and use webgl 2 you can derive the barycentrics using gl_VertexID % 3 instead.