Big Milestone checked off! Symbol is ready!

It's one of those huge-to-us milestones that will only later in the game's life have exciting impacts for you the player, but it's vital none the less.

Underlying all of our game's objects, be it swords or armor or eyebrows, is an XML system.  All of our items are named, described, defined, and their art linked in those XML files. We do this so future modders can make mods of our game, and we can proceed the way we always have on similar moddable games like StarSector, RimWorld, even Fallout to a lesser extent. 

The Symbol system is unique to us. It's got a lot of familiar styles with the Def system of RimWorld but is implemented in significantly different ways, such as our use of Tokens and the implementation of MessagePack to serialize all these loose XML files to one big Binary as the game starts -- which as long as no files are changed before the next game starts, the game doesn't need to do that again and starts up almost instantly.

And that's what's so important today.

Symbol is now done, documented, commented, and ready to go.

This is a huge pre-production milestone, as every feature we want to add to the Full Production version of the game relies on it.

It's also extremely fast.

I'll let Kent Explain:

Kentington — Yesterday at 16:49

Today we ran the final stress-test on version 1.0 of Symbol, the system we'll be using to load data into the game. The short version is, if you've ever been frustrated by long load times in games like RimWorld - especially when heavily modded - we've come up with a silver bullet for your troubles. 

Nerd details below.  

With a sample size of 10,000,000 procedurally generated blocks of data spread across 100,000 XML files, a parser using C#'s native System.Xml  took 3321 seconds to read the files and set up the Symbols in memory. Our production Symbol code reduced this to 25 seconds. Spot-checking the hashes of 10,000 randomly selected data fields in each run showed them to be identical, making it a virtual certainty that this is not an artifact of testing.  

It should go without saying that our game won't have ten million Symbols to load! Although this won't directly impact some aspects of loading, such as reading textures or applying Harmony patches from mods, a reduction in Symbol load times of more than a hundredfold - while keeping the definitions of game objects as human-readable (and human-modifiable) XML - is monumental. 

We plan to push this accomplishment even further by leveraging some of Unity's cutting-edge asset management technologies to quickly stream art assets on demand, rather than loading more than the bare-bones at game startup.  

There's a little "secret sauce" in here that we can't talk about yet, but rest assured that the Project Morningstar code revolution has only just begun. Later this year I'll have more posts like this to show off just how dedicated we are to blazing new trails in fast, fun, flexible gameplay experiences.

---

So with that, we're ready to kick off the next major phase of leaving Pre-Production on our roadmap, which is converting over all the old legacy prototypes to our new system, and bringing James and Curtis aboard to begin their work within this new framework. 

With Symbol done and two new coders pouring gas in the tank, we are ready to launch a sprint into a new roadmap milestone. :)

We can continue with the confidence that not only is Morningstar super flexible and extensible at its core, but future modders of Morningstar can produce flabbergastingly huge mods like SOS2 or New California, and not be trapped with an aged and decrepit architecture which can't handle it well. If Morningstar has 300 mods, it should still load in under 25 seconds under all that strain the first time that mod list is run, and after that it should start up in just a few seconds.

We can't account for modders using bad code practices, but we can have a rock solid system to support the effort put in.