Design philosophy

Note

This is part of the Low Level Transport for advanced users only.

UDP vs TCP

Most data transmitted over the internet uses either UDP or TCP.

UDP simply encapsulates a byte array into a packet, and sends it to a destination IP address and port. It doesn't check if the packet is delivered, it doesn't check if the packet was corrupted in transit, it doesn't even check if there is someone listening at the destination IP address. It just sends the packet into the internet and forgets about it. There is also a limit to the amount of data that can be sent in a single UDP packet called the MTU (Maximum Transmittion Unit) which is usually around 1500 bytes. This makes the delivery of the packets as fast as possible.

TCP on the other hand establishes a connection by first checking if someone is actually listening on the other end via a handshake and then periodically sending packets to make sure they are still there. It also retransmits all packets that have been lost, checks if the packets have been corrupted in transit, and makes sure they are received in the same order as they are sent. This makes sure all the data is delivered reliably and there are no limits to the amount of data that can be sent.

UDP is better

Fast paced multiplayer games need positional data of players to be transmitted as fast as possible and as often as possible. It doesn't matter if some of the packets get lost between because newer positions override the old ones. This makes UDP is the preferred choice for games.

UDP is not enough

In some cases however, you might want to send packets that can't be lost or corrupted such as notifying when an object spawns or despawns, chat messages or login information when you have accounts. You usually also want to keep some sort of a "connection" open to know when a player connects or disconnects. This is where the Low Level Transport comes in.

Custom protocol on top of UDP

The Low Level Transport builds a custom protocol on top of UDP that allows you to establish a connection, send reliable and unreliable messages and automatically fragment and reassemble messages larger than MTU. It also compresses all data to save on bandwidth, provides accurate timings and can ensure messages get delivered in proper order. If you're sending sensitive data such as passwords, it also allows you to encrypt it.

Connections

To establish a connection, the initiator sends a handshake to the receiver, and then waits for a response. If the response arrives, a connection is considered established. The netcode can initiate a connection or accept a connection. If two sides initiate a connection to each other at the same time, a connection is also established.

The netcode then periodically sends reliable ping packets to check if the other side is still listening.

Reliability

To ensure reliability, the netcode marks certain messages as reliable which makes the receiver send an acknowledgment back. If the acknowledgment has not been received after a period of time, the message is retransmitted. If the message is retransmitted enough times without an acknowledgment, the connection is considered dropped.

Duplicates

Packets sent to the internet bounce around different routers before arriving to their destination, so they can get duplicated sometimes. To prevent this the netcode optionally appends a unique number called a sequence to each message. The sequence is incremeneted by one for each message sent. When a message arrives it saves its sequence for a short amount of time so if the same message arrives in the future, it can discard it.

Ordering

Since packets can take different routes to arrive to their destination, they can get delivered in a different order that they are sent. To ensure order, some messages can be marked as ordered.

Ordered messages check the sequence number of the last received message and the message currently being received to see if it arrived in order. If the sequence is lower, the message is dropped. If the sequence is higher, then the message is processed.

For reliable ordered messages, if there are missing messages between the arrived message and the latest received message, the arrived message is delayed until the missing messages arrive.

Timings

The netcode internally has a clock which keeps track the number of milliseconds that have elapsed since the netcode initialized. This clock is used to optionally add timestamps to messages to let remote receivers know how long ago the message was created and the duration of the transit. There might be a difference between the local clock and the remote clock, so the netcode automatically resolves that by calculating and taking it into account.

Consolidation

If you are sending a lot of small messages really fast, it is useful to combine the messages into one bigger packet and send that. The netcode can optionally delay the sending of messages for a few milliseconds to wait if any new messages are released and during this delay consolidate all messages together into a single packet to be sent.

Compression

After the messages are consolidated together into a packet, the packet is compressed with a DEFLATE based compressor. This dramatically reduces the amount of bandwidth required.

Encryption

After compression, the packet is encrypted if the encryption is turned on. This provides security for sensitive data.

Fragmentation

After encryption, if the final packet is too big, it is split into smaller packets which are then sent individually into the internet.

Allocations & pooling

Allocation of resources such as arrays and data structures takes a while. And when you stop using them, the resources need to be cleaned up via garbage collection which can cause micro stutters. This is why the Low Level Transport pools all its resources. When a resource isn't used anymore, the netcode keeps it around until it is used again. This reduces the amount of allocations needed and speeds up the netcode.

Asyncronous & multithreaded

The netcode was designed to always run in the background. Recieved messages are processed as soon as they arrive. When you send a message, you don't have to wait for it to be sent, it is sent in the background. Most of the events are invoked in a separate thread.

Internally it uses Task Parallel Library (TPL) which uses async/await keywords instead of creating threads.