Understanding the Unity Serialization System

Unity's Serialization System is the backbone of various core features, and it's essential for developers to grasp its working principles. It's the process of converting data structures or object states into a format that's easier to store and reconstruct later. In this guide, we'll discuss the intricacies of Unity's serialization system, including what it can serialize, its limitations, and custom serialization.

What Is Serialization?

💡

Serialization in Unity is the process of converting the state of an object into a format that can be stored or transmitted.

Basically Unity will take all the data that is scattered all over the memory and organized in a linear sequence one after another.

Example: the data of an array of strings will be scattered in various place in the memory, but when it’s serialized for example into a JSON file like you will se in a but, that data it’s organized in a linear sequence one after another.

Where does

A Deep Dive Into Unity's Serialization

Unity's serialization backend takes data scattered over memory and lays it out sequentially. Although memory layout is often overlooked in C# development, it's crucial to be aware of several built-in Unity features that use serialization:

Saving and Loading: If you open a .unity scene file with a text editor and have set Unity to “force text serialization,” the serializer is run with a YAML backend.
The Inspector Window: This interface doesn't directly communicate with the C# API to determine the values of whatever it's inspecting. Instead, it requests the object to serialize itself and displays the serialized data.
Prefabs: Internally, a Prefab is a serialized data stream of GameObjects and components. A Prefab instance is a list of modifications applied to the serialized data.
Instantiation: When you instantiate a Prefab (or a GameObject living in the scene), you serialize the object, create a new object, and then deserialize the data onto the new object.

The reorganized data stream can be stored in a database, a file, or a memory, and “deserialization” is the reverse process.

Source: Unity 3d

How does Unity serialize his stuff

Source: Unity3d

From the Unity Blog: “The serialization system is written in C++, we use it for all our internal object types (Textures, AnimationClip, Camera, etc). Serialization happens at the UnityEngine.Object level, each UnityEngine.Object is always serialized as a whole. They can contain references to other UnityEngine.Objects and those references get serialized properly.”

The c++ components of the serialization system

In Unity, serialization is handled by a combination of two systems: the serialization backend and the transfer system.

The serialization backend is responsible for the process of converting an object's state into a format that can be stored or transmitted.

This could involve turning a complex object into a byte stream or a text representation, like JSON or XML.

Unity's serialization backend works by serializing all public fields in your scripts and all non-public fields that are marked with the [SerializeField] attribute.

This serialized data can then be used for several tasks, like saving it to disk, copying and pasting objects in the editor, undo and redo in the editor, instantiating prefabs, etc.

The serialization backend also understands references, meaning that if you have multiple references to the same object, it will not duplicate data when serializing that object. Instead, it will keep a single copy of the object and references to it. This can help you save memory and avoid issues with data consistency.

On the other hand, the transfer system is responsible for the task of moving serialized data around. This includes tasks like:

Loading serialized data from disk into memory when you open a scene and viceversa.
Saving serialized data from memory to disk when you save a scene.
Sending serialized data across the network in multiplayer games.
Copying serialized data from one place in memory to another when you copy and paste objects in the editor, duplicate objects in a scene, or instantiate prefabs.

So the transfer system will go trought all the data that we are interested in serialize and the serialization backend will converto that data into a “readable” format.

These two systems work closely together in Unity to provide a robust and efficient serialization system.

The transfer system is built on top of the serialization backend, using the serialized data it produces to move object states around as needed.

They ensure that your game's data is accurately and consistently maintained across different game sessions, builds, and platforms.

Serialization Backend In-depth

Disclaimer: Inside the serialization backend the it a function called Transform which it’s different from the Transform function that we have seen in the previous chapter.

SerializeTraits

SerializeTraits in Unity is a c++ class part of the serialization backend that steps in when we need to decide how specific types should be serialized and deserialized. 📚 The main aim of these traits is to fine-tune serialization behavior for complex or unique data types that require a bit of special handling.

For each type, a corresponding SerializeTraits specialization can be created that instructs how that type is serialized, deserialized, and how its memory layout is arranged. 🧠 These traits provide functions for creating, copying, and demolishing instances of the type, as well as peeking into and modifying the type's data.

Imagine you have a custom collection type like a linked list or a hash map. The default serialization might not be ideal or might not work at all for these guys.

By pulling SerializeTraits into play for such a type, you have precise control over how the type is serialized, ensuring its journey to and from the serialized state is just as you want it. 🕹️

Here's a sample of what a SerializeTraits specialization might look like for a hypothetical CustomType:

namespace UnityEngine.Serialization
{
    template<>
    struct SerializeTraits<CustomType>
    {
        typedef CustomType Type;

        static bool CanSerialize(Type& val)
        {
            // Return true if the value can be serialized, false otherwise.
        }

        static void Write(SerializeOutStream& stream, Type& val)
        {
            // Write the value to the stream.
        }

        static void Read(SerializeInStream& stream, Type& val)
        {
            // Read the value from the stream.
        }
    };
}

As you can see, this opens up a high level of control over the serialization process. 👏

But, remember! SerializeTraits is a complex and advanced feature that should only be used when absolutely necessary.

Most of the time, Unity's default serialization behavior will handle the workload just fine and more efficiently for typical use cases.

Ho the serialization backend works:

The Transfer function of the serialization backend will take the data from the outside (passed from the other Transfer function) and will instantiate a SerializeTraits based on the type of the date been passed. For example, if the Backend has to serialize an int, the Transfer function will generate a SerializeTrait<int>.
The specialized SerializeTraits will be passed to another Transfer function that will write the data to the output.

What Can Unity Serialize?

Is public, or has a SerializeField attribute
isn’t static
isn’t const
isn’t readonly
Has a field type that can be serialized:

Primitive data types (int, float, double, bool, string, etc.)
Enum types (32 bites or smaller)
Fixed-size buffers
Unity built-in types, for example, Vector2, Vector3, Rect, Matrix4x4, Color, AnimationCurve
Custom structs with the Serializable attribute
References to objects that derive from UnityEngine.Object
Custom classes with the Serializable attribute. (See Serialization of custom classes).
An array of a field type mentioned above
A List<T> of a field type mentioned above

Serialize custom classes

To enable serialization, apply the[Serializable]attribute

//Create a custom struct and apply [Serializable] attribute to it
    [Serializable]
    public struct PlayerStats
    {
        public int movementSpeed;
        public int hitPoints;
        public bool hasHealthPotion;
    }

How to serialize properties in Unity

Let's take a look:

Example A: Serialization of a Private Field

[SerializeField]
private List<Crossword> _crossword = new List<CrossWord>();

public List<Crossword> Crossword { get { return _crossword; }

In this example, a private field _crossword is created, which is directly serialized by Unity. This field can't be accessed or modified directly outside the class. Instead, we use a public property Crossword to expose this data to the outside world. The property only provides a getter, which means the data can be read but not modified externally.

Example B: Serialization of a Property

[field: SerializeField]
public List<Crossword> Crossword { get; private set; }

In this case, a public property Crossword is serialized directly. It has a public getter, which makes the data readable from outside, and a private setter, which restricts modifications to within the class only. The [field: SerializeField] attribute allows Unity to serialize the backing field of the property, rather than the property itself.

In a nutshell, the primary difference lies in the level of encapsulation and direct serialization.

Example A utilizes a private field for data storage, which is directly serialized, and a public property for data access. It offers a greater degree of encapsulation as the underlying data is completely hidden, and the access is regulated through the property.

On the other hand, Example B offers a more streamlined approach where the property itself is serialized. The access to the data is still controlled, but the encapsulation is slightly less strict as we're serializing the property's backing field directly.

What Unity Can't Serialize

Unity has limitations when it comes to serialization, and it can't serialize anything beyond basic collections. For example, Dictionary and multidimensional arrays can’t be serialized.

🙈 Custom serialization

Sometimes you might want to serialize something that Unity’s serializer doesn’t support (for example, a C# Dictionary). The best approach is to implement the ISerializationCallbackReceiver interface in your class. This allows you to implement callbacks that are invoked at key points during serialization and deserialization:

When an object is about to be serialized, Unity invokes the OnBeforeSerialize() callback. Inside this callback is where you can transform your data into something Unity understands. For example, to serialize a C# Dictionary, copy the data from the Dictionary into an array of keys and an array of values.
Later, when the object is deserialized, Unity invokes the OnAfterDeserialize() callback. Inside this callback is where you can transform the data back into a form that’s convenient for the object in memory. For example, use the key and value arrays to repopulate the C# Dictionary.

‣

Unity custom Dictionary Serialization Example

using UnityEngine;
usingSystem;
using System.Collections.Generic;

public class SerializationCallbackScript :MonoBehaviour,ISerializationCallbackReceiver
{
    public List<int> _keys = new List<int> { 3, 4, 5 };
    public List<string> _values = new List<string> { "I", "Love", "Unity" };

    //Unity doesn't know how to serialize a Dictionary
    public Dictionary<int, string>  _myDictionary = new Dictionary<int, string>();

    public void OnBeforeSerialize()
    {
        _keys.Clear();
        _values.Clear();

        foreach (var kvp in _myDictionary)
        {
            _keys.Add(kvp.Key);
            _values.Add(kvp.Value);
        }
    }

    public void OnAfterDeserialize()
    {
        _myDictionary = new Dictionary<int, string>();

        for (int i = 0; i != Math.Min(_keys.Count, _values.Count); i++)
            _myDictionary.Add(_keys[i], _values[i]);
    }

    void OnGUI()
    {
        foreach (var kvp in _myDictionary)
GUILayout.Label("Key: " + kvp.Key + " value: " + kvp.Value);
    }
}

Differences between Editor and runtime serialization

Unity serializes some features only in the Editor, while it can serialize other features in both the Editor and at runtime:

As you can see, if you want to serialize and deserialize objects at runtime outside the editor you are forced to use the Unity JsonUtility (or other tools as we will have a look later)

🏧 How to serialize assets in Unity to Load and Save them

If you are in the editor or on a device, you can use JsonUtility which can convert any MonoBehaviour, ScriptableObject, or plain class/struct with the Serializable attribute to a JSON string.

For example, you can create an instance of a ScriptableObject at runtime using ScriptableObject.CreateInstance and edit its properties via script.

You can use JsonUtility to convert the ScriptableObject instance to a JSON string, which can be saved and loaded from the disk.

Basically JsonUtility will serialize the class and save the content into a JSON file.

‣

Load Example

using UnityEngine;

public class PlayerState : MonoBehaviour
{
    public string playerName;
    public int lives;
    public float health;

    public void Load(string savedData)
    {
        JsonUtility.FromJsonOverwrite(savedData, this);
    }

    // Given JSON input:
    // {"lives":3, "health":0.8}
    // the Load function will change the object on which it is called such that
    // lives == 3 and health == 0.8
    // the 'playerName' field will be left unchanged
}

‣

Save Example

using UnityEngine;

public class PlayerState : MonoBehaviour
{
    public string playerName;
    public int lives;
    public float health;

    public string SaveToString()
    {
        return JsonUtility.ToJson(this);
    }

    // Given:
    // playerName = "Dr Charles"
    // lives = 3
    // health = 0.8f
    // SaveToString returns:
    // {"playerName":"Dr Charles","lives":3,"health":0.8}
}

Serialize Static Reference

Serializing static references directly in Unity is not possible due to the very nature of static variables.

The static variables are not tied to an instance of a class, but rather to the class itself, so they persist across multiple instances of that class.

As such, they are not serialized because the serialization process in Unity involves converting instance-specific data into a format that can be stored and later used to recreate the object.

Unity Editor Extensions: Understanding Data Persistence Across Assembly Reloads

If you've been developing editor extensions in Unity, you've likely been taken aback when your painstakingly entered data disappears after entering and exiting play mode. It feels as if your robust tool has reverted to its default state. Before you blame the gremlins in your computer, let me shed some light on what's happening - it all has to do with how Unity's managed (Mono) layer works.

What Exactly Happens During an Assembly Reload? 😬

Entering/exiting play mode or tweaking a script prompts Unity to reload the mono assemblies - the DLLs associated with Unity. From the user's perspective, this is a three-step process:

Serialization: All the serializable data gets extracted from the managed side, creating an internal data representation in the C++ side of Unity.
Memory Purge and Assembly Reload: All memory or information linked with the managed side of Unity is destroyed. Then, the assemblies are reloaded.
Deserialization: The saved data from the C++ side gets re-serialized back into the managed side.

The key takeaway here is that your data structures or information need to be serializable to survive an assembly reload. If your data can be serialized into and out of C++ memory efficiently, it means you can also save this data structure to an asset file and reload it when needed.

The Benefit of Understanding Serialization in Unity

While this might sound like a complex process, it's actually a fundamental part of working with Unity - particularly when creating editor extensions.

Understanding how this serialization and deserialization process works not only helps prevent data loss during assembly reloads, but it can also open new possibilities like saving your data structures to an asset file and reloading them as needed. This can be a powerful feature when developing complex tools or games in Unity.

Serialization in Unity: Advantages of Using Scriptable Objects and "SerializeReference" Attribute

So far, we've delved into serialization within Unity by using standard classes. However, serialization with regular classes in Unity can have its limitations. Let's illustrate this with an example.

Case Study: Serialization with Standard Classes

Here's a scenario where we have two fields of type NestedClass. When we draw the window for the first time, it shows both fields. As m_Class1 and m_Class2 point to the same reference, altering one would affect the other.

Consider this code in C#:

using System;
using UnityEditor;
using UnityEngine;

[Serializable]
public class NestedClass
{
    [SerializeField]
    private float m_StructFloat;
    public void OnGUI()
    {
        m_StructFloat = EditorGUILayout.FloatField("Float", m_StructFloat);
    }
}

[Serializable]
public class SerializeMe
{
    [SerializeField]
    private NestedClass m_Class1;

    [SerializeField]
    private NestedClass m_Class2;

    public void OnGUI ()
    {
        if (m_Class1 == null)
            m_Class1 = new NestedClass ();
        if (m_Class2 == null)
            m_Class2 = m_Class1;

        m_Class1.OnGUI();
        m_Class2.OnGUI();
    }
}

But, here's the catch! Try reloading the assembly by entering and exiting play mode. What you'll notice is that the references have been decoupled. The reason for this disconnect is the method Unity employs to serialize a class marked simply as [Serializable].

When Unity serializes standard classes, it traverses the fields of the class, serializing each one independently, even if multiple fields share the same reference. This method means that the same object can be serialized multiple times.

When deserializing, the system won't recognize them as the same object. This constraint can be particularly frustrating when designing complex systems, as it hinders the accurate capture of intricate class interactions.

Example 02: Serialize reference in a list

Custom classes behave like structs

[Serializable]
class Animal
{
    public string name;
}

class MyScript : MonoBehaviour
{
    public Animal[] animals;
}

To provide some context, let's start with a simple scenario. Imagine you have an array of Animal objects, and you populate this array with three references to a single Animal instance. When you serialize this configuration, you might naturally expect to find just one object in the serialized stream since you only created a single Animal instance. However, that's not the case.

In Unity, each reference is treated as a separate entity in the serialization process, even though they all point to the same object. This means you will find three objects in the serialized stream, not one. And, more importantly, when you deserialize this data, you'll get three different Animal objects, not three references to a single object.

This might lead to unexpected behavior if you're not aware of it. This peculiar characteristic of Unity's serialization system means that if you need to serialize a complex object graph with references, you can't just rely on Unity to handle it all for you. You will need to implement additional serialization steps yourself.

It's crucial to note that this serialization behavior only applies to custom classes. These classes are serialized "inline," meaning their data becomes part of the complete serialization data for the MonoBehaviour they're used in. On the other hand, if you have fields that reference a UnityEngine.Object-derived class (like a public Camera myCamera), the data from that Camera isn't serialized inline. Instead, Unity serializes an actual reference to the Camera UnityEngine.Object.

To navigate these complexities, you may need to customize your serialization strategy, depending on the specific needs of your game. Understanding these subtleties will allow you to make the most of Unity's powerful features while avoiding potential pitfalls along the way.

Introducing ScriptableObjects and "SerializeReference"

That's where ScriptableObjects come to the rescue! They're a type of class that serializes correctly as references, ensuring they only get serialized once. This unique attribute allows complex class interactions to be stored in a way that aligns more with our expectations.

Just like MonoBehaviours, ScriptableObjects are internally identical within Unity.

However, a key difference lies in the fact that a ScriptableObject doesn't need to be attached to a GameObject, unlike a MonoBehaviour. This makes them exceptionally useful for general data structure serialization.

But there's more! Unity also provides us with another tool for our serialization needs: the "SerializeReference" attribute.

You can use this attribute to maintain references between objects during serialization. By marking a field with [SerializeReference], you enable Unity to keep track of relationships between classes or structs, even when serialized. It's an excellent tool to overcome limitations faced during the serialization of standard classes.

[Serializable]
public class SerializeMe
{
    [SerializeReference]
    private NestedClass m_Class1;

    [SerializeReference]
    private NestedClass m_Class2;

    // rest of the class
}

With these tools in our Unity toolkit, we are now equipped to create more complex systems while ensuring data integrity and interactions are preserved as expected. Don't let the serialization woes stop you from achieving your development dreams in Unity!

📕 To know more about SerializeReference, read the following article.

Serialize nested ScriptableObjects or GameObjects references

The problem

Assuming you have a scriptable objects that hold a reference to another scriptable like in the image below:

Now if you try to serialize the object, will get the following json:

{
  "Id": "fb90e07d-2507-4f19-84b4-37f84d75d483",
  "sectionDefaultData": {
    "instanceID": 43610
  },
 //rest of the class

As you can see unity saves the object, and serialize its instanceID.

The problem is that the InstanceID of a Unity object is not guaranteed to be persistent across different sessions.

It is used by Unity to uniquely identify different objects during a single session and can change between sessions.

So if you close the editor and reopen it and try to load the data from the saved JSON you will have a random object loaded instead of the original one:

Solution

So far the best way to solve this problem is to create a database with a key-value pair that contains a unique id of the object and a reference to the object itself, then when it’s time to serialize the object, just use the unique ID from the database, then when it’s time to load the object, you can retrieve if from the database.

But isn’t there any easier way? Yes, as you can see in a bit, there are some plugins that can help you solve this problem with just a single line of code.

Dealing with Non-serializable Types in Unity 🖼

Unity's built-in serializer has some limitations compared to the .NET Framework's serialization capabilities. For instance, it can't handle complex types such as Action, Func, dictionaries, and polymorphism effectively. If you've tried to serialize these types without specifying otherwise, you might have noticed these fields end up being null after deserialization.

Take a look at this example:

public Action OnSectionUnlocked;

If this code is within a class marked as [Serializable], Unity will attempt to serialize the Action OnSectionUnlocked. However, since Action is not a serializable type, it ends up being null when deserialized, which might look like it's being "overwritten" with null.

To prevent Unity's serializer from attempting to serialize these non-serializable types, use the [NonSerialized] attribute, like so:

[NonSerialized]
public Action OnSectionUnlocked;

With this attribute in place, Unity's serializer ignores this field, leaving it as is.

As a result, OnSectionUnlocked won't be overwritten with null after deserialization. It's a crucial detail to keep in mind when working with non-serializable types in Unity.

However, bear in mind that non-serialized fields won't retain their values when you stop playing in the Unity Editor.

The behaviour described above can lead to some nasty bugs 🐞 , in my case I had some classes subscribing to an Event, then when my load system tried to load the class, the Action was overwritten with a null value, hence cancel all the events that had previously subscribed to the Action😬. So using NonSerialized its a best practice that can be very helpful to avoid bugs like this one.

External serializations system

So far I’ve been trying all the serialization systems that there are out there and every one has its pro and cons, but here are the most used:

These solutions work well enough if you have a non-complex system to save, let’s say that you have to serialize and save complex objects like nested scriptable, dictionary of gameObjects etc, then these assets won’t be enough.

Easy Save plugin

Easy Save lets you save almost anything with ease across platforms, along with features such as encryption, compression, cloud storage, spreadsheets, backups, and much more.

If you want to save and load basically everything inside Unity without having to worry about it, then I strongly recommend trying Easy Save, it’s a bit expensive, but it will solve all of your problems.

Conclusion 🚂

In conclusion, Unity's serialization system is a powerful tool that enables developers to save and load data efficiently. The system can serialize a wide variety of data types, including custom classes and structs, and built-in Unity types such as Vector2 and Color.

However, there are limitations to the system, such as the inability to serialize static references and certain collections such as dictionaries.

To overcome these limitations, developers can use custom serialization methods such as implementing the ISerializationCallbackReceiver interface or using third-party solutions like Easy Save. Additionally, the use of ScriptableObjects and the SerializeReference attribute can help to maintain reference integrity during serialization.

Understanding Unity's serialization system and its limitations is crucial for developers looking to save and load data in their projects. By utilizing the system effectively, developers can create more complex systems and achieve their development goals in Unity.

I personally spend so many hours to get a grasp of how the Unity serialization systems work, and it’s such an important part of game development that must be studied and understood (hopefully with the help of this article)

Resources

Unity Documentation
Serialization in Unity (Unity blog)
Understanding Unity’s serialization language, YAML (Unity blog)
Serialization best practice megapost (Unity Forum)
Unite Europe 2017 - How Unity's Serialization system works (video)
Serialization in-depth with Tim Cooper (video)

✍️ Author

Marco Mignano 👋 ➡️ Passionate Unity Game Developer Marco - coding aficionado, video game enthusiast, and self-proclaimed piazza addict. Constantly conquering new challenges, one line of code at a time. Got an exciting project? Let's make your game next game together!

Contact: marco@gladiogames.com

Unity Serialization System