How can I make a 3D visual programming language

Question

This is not a joke. and I want a tutorial.

I want to make a 3D (node editor / programming language / graph editor) like this picture:

How can I do that?

Answer 1

These are just some thoughts on the matter, I think some planning will be required. I'm unaware of any tutorials explaining how to build node editors, but since so many node editors exist, perhaps your best source for inspiration can be other people's codebases.

I think you can look at prior art:

https://architecture.mit.edu/computation/project/visual-programming-three-dimensions-visual-representations-computational-mental

Also, I think you can plan it like this:

1. figure out what it will look like (maybe you can make a mockup on blender first)
2. think about what kind of interactions will be allowed, how will the mouse work?
3. Consider using an existing 3d framework like openframeworks or unity
4. design a command palette or list of keyboard shortcuts of things that will be required
5. study existing 2d note editors, consider evaluating the work of John Paul Morrison (Flow Based Programming) and things like NoFlo or PyFlow among many others, you can study the ones that have actual users like the Unreal Node Editor, or more recently I think Unity also has one.
6. design an execution engine (how will the code actually run, input ports, output ports, etc).
7. consider the format that you'll use for saving/loading the networks, is it sqlite? json? will you need to store the coordinates and perhaps the dimensions of the nodes, and the connections between the nodes?
8. if a new node is to be located, where will it be put? do you need to indicate an arbitrary point in 3d space, or is it better to have the new node positions to be relative to existing objects?
9. component property editors may be important, each node will probably have it's own set of properties, will each node be custom, with a custom ui for editing? or will there be a general property editor for all the nodes? these decisions should be made.

custom node properties can be used to make the ui easier for the user, however the global property editor will allow you to create many components relatively inexpensively.

10. what operations need to be supported? do you need auto-layouting algorithms, like graphviz's dot?

11. will the runtime allow for step by step execution? node statistics? real time visualization?

12. consider who is your target market for this tool, and why this tool will be useful outside the existing tools, is it music artists using the next vvvvv ? or is it graphic artists making the next shader? will be useful for web development? or for a specific niche?

I hope you'll find these thoughts and ideas useful in your process, and I look forward to the things you will build.