I'm trying to draw a dialog box with sfml and I'm using a VertexArray with quads. I want it to have a corner with 2 points like in this image:
Could someone help me with the math on how to get the length of A? Or maybe someone has a suggestion to do it in a different way in sfml?
What you are seeking is called the miter angle. And the miter angle is just half of the corner angle.
See this interactive diagram (at Geogebra)
By trigonometry, the ratio A/Th = TAN(angle/2).
A = Th * TAN(22.5)