I would like to generate diagrams similar to this, but I don't have a clue what to look for.
That's a Syntax Diagram.
Syntax diagrams (or railroad diagrams) are a way to represent a context-free grammar.
Links: