Requirements and Design
So you want to learn how to create a node based scripting language? You know, one of those languages that have become extremely popular in today’s game engines.
This post series will help you along the way.
We’ll cover the entire process from the design of your language to implementing the runtime side and a simple editing frontend for it.
By the end of the series, you will hopefully be armed with the skills required to create your own visual scripting language. This is why you’re here right?
So let’s get started.
But first, you must understand why these scripting languages exist in the first place.
Why Node-based Scripting?
Why not just use code to deal with everything? Or a text-based scripting language like Python or Lua? Why go through the trouble of creating this whole new system when a text editor would have done fine?
There are several answers to this depending on what type of users you have.
Generally, your artists and designers will have a much easier time understanding and using a visual node-based scripting language than they would a text-based one.
Why’s that you might ask?
Well, the text-based languages require correct syntax. And that’s precisely what makes them hard. Syntax errors are the most common mistakes beginners make when learning how to program.
Also, a visual node-based language does the job of automatically explaining the flow of execution or data. But only if the building blocks (aka the nodes) are straightforward and without hidden side effects.
Another result of a well-designed node based scripting language is that it creates a sandbox. A sandbox is an environment where very few errors might occur. This, in turn, encourages experimentation in a way that text-based languages fail to do.
But node based scripting languages are not just beneficial for your artists and designers. They will usually result in simple and well-designed code.
The nodes enforce a decoupled architecture for complex game logic.
Sure, integrating a language like Python or Lua can accomplish this too. But they don’t have the other advantages that have been brought up.
So now that you know some reasons for the existence of these languages, you should be better armed when determining if it’s the right solution for you.
Even if you’re on a really small project you might consider it.
As an example, I am creating this course on how to create an interactive children’s book for iOS using SpriteKit.
The book needs to run cinematic actions like animations and sound interleaved with touch and drag interactions.
Now if you’re not familiar with SpriteKit, I might tell you that it is possible to run these actions on a timeline for the objects they apply to.
But the timeline plays automatically when a scene is loaded and in order to run an animation or play a sound as a result of a touch interaction you need to use code.
Now my course is targeted at non-programmers, otherwise, I might not have considered a visual scripting language for this. But as it happens, I did.
Once I came up with the idea, I was happy with the fact that it would make things a lot easier to explain. Little did I suspect that I would end up using this myself to experiment with and polish my book with an excellent turn-around time.
It was liberating.
And I was surprised how much cleaner the code got.
And so can yours. It really isn’t that hard.
The biggest danger when creating a node based language is to create too thin of an abstraction. This tends to yield huge node graphs for even simple tasks.
Instead, keep the language simple and cut out for the project at hand.
My interactive book authoring node based language is excellent for that type of project but might do a poorer job implementing a full game such as a platformer. I haven’t tried it yet though.
If you don’t need to, don’t try to be too general. This defeats the purpose of simplicity and will make it a lot harder to implement the language.
In fact, the only time you should consider a general node based language is if you are creating a product with the requirement to be generic, like a general-purpose game engine.
Types of Languages
So basically there are a couple of types of languages.
The Fully Featured Programming Language
A scripting language that has all the standard programming language features like control flow, loops, functions and so on. Unreal Engine 4 is a perfect example here. The blueprint scripting language is almost able to replace coding in C++. It is very granular and scripts easily get big.
The Data Processing Language
A type of language that is used to run data through different transforming functions. This could be generating content as in Houdini’s node networks, applying filters to images etc. There are usually very little or even no control flow structures involved here.
The Event Communication Language
A node-based graph to hook up event flows between different objects or functionality. My interactive book language is an example of this. All it does is sequence events in a scripted manner. In languages like this, the nodes rarely represent programming constructs, rather objects. It is useful for scripting scenarios where the logic itself is embedded inside the nodes.
The Expression Language
A language mostly focused on performing calculations using mathematical operations and functions. In general, calculations are not very well suited to be written in graph form as they tend to generate very large graphs if one is not careful. But there are some use-cases that are still valid. One commonly used example is that of a shader graph enabling content creators to experiment with the material look of their assets. Here, every step of the calculation is essential and can even be visually displayed.
Of course, there is everything in-between as well. Your language might end up being a mix between these types.
Types of Execution
When it comes to executing the functionality of a node-based script there are a couple of different variants.
The node-based script might end up being compiled into something like byte-code. A virtual machine can then execute this. In the case of Unreal’s blueprints, a compile step translates them into UnrealScript VM bytecode that is then executed inside the runtime.
This is almost like compiled, except that it consists of code generation into a language that is then compiled further. Shader graphs are for instance usually generating something like HLSL or some other shading language.
In this case, the nodes usually exist as objects in the runtime, and the runtime implementation handles the communication between them. If this is used for very granular languages, it might make turn-around for authoring faster, but runtime execution slower. It is better suited for less granular languages like the node networks in Houdini. Here, the nodes themselves perform heavy operations and the node language runtime overhead is negligible.
Of course, again, a mix is possible. For example, my interactive book language outputs Swift code that constructs the graph, but the nodes themselves exist in the runtime. I simply didn’t want or need the complexity of writing and parsing a data format here.
Your Language Requirements
You may now sit down and think about what type of language and execution model best fits your needs.
Again, don’t try to make it too generic. Instead, think about what the true scope of the language is. Most of the implementation will be different between different types of projects so there is very little room for reuse. What should be general and reusable is the framework used to create the nodes. If you can reuse the framework, you can reuse generic nodes like control flow.
That said, you should make sure to have a few different use cases in mind before trying to design your language. This way you’ll have something to test your level of abstraction against to ensure that it is usable.
Now, begin to sketch out a set of nodes that can be used together to accomplish the tasks in your use cases.
- What common tasks exist?
- Do you need control flow nodes?
- Is there a need to perform calculations?
- Will you have nodes representing objects or functionality or both?
- Do you need subgraphs/functions?
One tip here: Start at a high level and sketch out your use cases. If you find yourself dealing with a combinatorial explosion of similar nodes with different variations, you should probably split up the nodes and separate concerns.
Some other things to think about is if you need to control the order of events, if data changes are pushed or pulled from one node to another, and what the entry points of your scripts are.
Once you have a satisfying set of nodes with their own responsibilities, make sure to think up more of your use cases for a final test.
Check back next week when we’ll be talking about how you can go about implementing the runtime side of your language.