NewGoofy Meta Language

I've been writing NewGRFs for a while. My first GRF released on BaNaNaS was a China Set GRF – despite the fact that I don't really like the project's artistic style. It's been a bit more than a year since the release, and learning the specifications as I developed more GRFs, NML showed its shortcomings and awkward design to me.

1. What is a GRF?

The two terms "NewGRF" and "GRF" are interchangeable at most times. "GRF" stands for "Graphics Resource Files", and the "New" in "NewGRF" adds another layer of newness to it.^[1]

GRFs are the de-facto "mods" of OpenTTD and TTDPatch. OpenTTD never developed an alternative type of file for content-wise game modifications (GameScripts and AIs are not in the scope of this article). Instead, it used the legacy of TTDPatch and TTD – the .grf format.

A GRF is composed of real sprite blocks and pseudo-sprite blocks. A real sprite block contains a sprite. Think of a sprite as an image of a car, or a house. The landscape, bridges, road and rail infrastructure graphics are all real sprites. In contrast, a pseudo-sprite block does not contain visual elements. It is a piece of code, and the game interprets it to define or redefine vehicles, industries, houses, etc., skip a specific portion of the game, and modify global variables.

GRFs are unlike lua scripts in Transport Fever or Factorio. It's also not the same as zipped xml definitions in RimWorld. ^[2] They are more like self-contained executable files that contain assembly instructions. GRFs are assembly like, and writing a GRF from scratch, without the help of NML, or a macro processor, is hard.

2. Writing a GRF

Early GRF developers relied on the grfcodec utility, a GRF compiler and decompiler. At this stage of GRF development, everything is at the assembly level. A bit later, some developers (namely Michael Blunck) developed pre-configured macros for NFO using GNU M4.^[3] After fighting with spriteblocks for a while, NML was born.

Modern GRF developers would start using NML. NML abstracts away the low-level spriteblocks when writing a GRF, and provides semantic codepoints for runtime variables, object properties, and control flows. You can see the code examples at NMLTutorial for details.

Things are regulated with NML. NML also provids a solid foundation for writing NewGRFs. It ensured that all spriteblocks generated on the lower level are in the correct orders and positions. An example would be the if control flow. Writing a structure that is equivalent to the if-else control flow in other languages is quite hard. Basically, there are three steps to do:

• Count how many spriteblocks to skip
• Write a action 7/9 block at the start of the spriteblocks you would like to skip.
• Write down the counted number. If the number exceeds 255, you would then need to

write an action 10 "label" block at the end of the skipped spriteblocks, assign an index to it, and write the index in the action 7/9 block.

While in NML it is this: if (...) {...} else {...}.

The example above should show how helpful NML is. ^[4] Instead of writing action 8 and action 14 for GRF information declaring, NML uses the grf block that handles them simultaneously. Instead of using an action 0 and composing multiple action 2 together, NML does the item block. NML has helped numerous developers to focus on the actual GRF logic without wasting time studying GRFSpecs, and everything seems to be in a harmony. OpenTTD GRF devs use the latest NML, and produce wonderful contents for players.^[5]

3. The Twist

We've talked about the good of NML, now it is time to talk about its bad.

NML is wonderful for simple operations such as adding a vehicle or simple house. Its quirks starts showing up when you decided to implement some advanced logical mechanism.

I've worked on a Station GRF called WINS Is Not (only) Stations. I tried to implement advanced logic, such as querying nearby tiles to determine the current tile's graphics in the GRF – and implementing the stuff is a pain in the arse. Station tiles may use multiple pieces of graphics, and graphics are organized using a structure called a spritelayout. The Sprite layout block keeps in track of a list of bounding boxes ^[6] and the graphics associated with it. To make graphics change or display conditionally, I must write conditions in the spritelayout block. Even if I write those variables, because a spritelayout block have limited registers, I cannot use more than six variables, and that is for all components on a single tile.

4. The Solution – External Tools and Macros

NML, by default, does not support merging together multiple files. ^[7] A popular way to address that is to use gcc -E -x c <input-file> > <output-file>, with an input file that looks like this:

#include "trucks/torakku-hi"
#include "trucks/torakku-fu"
#include "trucks/torakku-mi"
...
#include "buses/basu-one"

Other methods include:

• Using a python script that reads and concats all files, and output the result to another file, which is then passed to nmlc, the NML compiler
• cat src/**/* > /tmp/nmltmp; nmlc /tmp/nmltmp.
• StarRaid's NML pypatcher – this is basically the same as the gcc approach.

Using GCC comes with another side affect, which I consider as a benefit. GCC supports expanding C-style macros^[8], and macros help reducing boilerplate.