Retrofitting Multiplayer - HELLFRONT: HONEYMOON
Retrofitting multiplayer with coherence. An elegant netcode for a more civilized age.
There is a sentiment among game developers that multiplayer needs to be planned out from day 1. And there sure are lots of games where you can tell that multiplayer was tacked on, without much care, at the end of the development process. So waiting with multiplayer until the last minute before release clearly is not ideal - and might not always be advisable.
But what if your game project is not multiplayer on day 1? What if it is day 500? What if your game is already released? Is it too late then?
We originally wanted it to support online multiplayer, but being the only coder on the team, I didn’t have enough time to pull it off. So we settled for couch multiplayer instead.
Now it is 2024, and I have dusted off our old Unity game project again, with the goal of adding online multiplayer using coherence.
Here's is how it went.
Playtesting
I completed the first iteration of the online mode over a weekend. This work mainly consisted of installing the coherence SDK and adding CoherenceSync components to a bunch of Prefabs. I used the default connect dialogs, which didn’t match the style of the game at all, but it was fast and it worked.
Having an actual playable build ready for playtesting this early was a great morale boost to me, and it helped kick-start the testing process. I reached out to a bunch of our most dedicated players and some gamedev friends and (through persistent DMing!) I managed to form a small group of die-hard beta testers.
Play sessions were scheduled for Tuesdays, Thursdays and Sundays and took place on the public voice channel of our Discord server. New game builds were distributed ahead of each session, using Steam Playtest. Eventually, curious players started finding their way into the play sessions, without me having to DM them first.
A play session would usually last between one and two hours, with us taking turns to play PvP and PvE matches in different combinations of players and maps. I captured video recordings and log files to track down bugs while playing, and then I tried to fix as many of those bugs as possible in time for the next play session.
We’ve had a ton of fun playing together like this, especially the 4-player PvP matches, and it turned out to be a great way to build momentum ahead of the multiplayer update. The scheduled play sessions also formed a natural feedback loop for iterative improvements throughout the development process.
Code. Build. Play. Rinse and repeat.
Turning Up The Awesome
The online mode was pretty much feature-complete at this point, but performance wasn’t great and players kept complaining about lag, especially during the most intense matches. Hellfront is all about fast-paced action so the netcode needs to handle replication of hundreds of units - and there shouldn't be any degradation of performance compared to playing offline.
This sets a high bar in terms of network throughput and latency. The good news is that coherence supports extremely fine-grained netcode optimizations and network LODing. It allows you to fine-tune exactly what data to sync over the network, at what frequency and at what compression. All down to the last bit.
Position samples, in particular, tend to consume a lot of bandwidth because they are continuously synced for all objects that are in motion.
By enabling fixed point compression, and tuning the value range to match the size of my game world, I was able to reduce the bandwidth cost for each position sample from 96 to 33 bits. This way, I could fit almost three times as many samples in each packet. By reducing the sample rate for marines from 20Hz to just 2Hz, their bandwidth was effectively reduced by another factor of 10x.
The optimization window is not pretty - but it is where a lot of the magic happens.
Rotation also tends to consume a lot of bandwidth, but I could make even bigger savings there.
Aliens, for example, always face in the forward direction of the path that they are moving, so a client can simply infer an alien’s rotation by tracking its movement.
Turrets, by contrast, always rotate towards their current target, so syncing a reference to the target, instead of the actual rotation, saves bandwidth and improves visual responsiveness.
Bullets are not networked at all in Hellfront. They exist independently on each client, but due to the high rate of fire, high projectile speeds and low per-bullet damage, it doesn’t really matter that individual bullets are not accurately replicated. What matters is that damage is perceived the same across all clients.
Player characters are the most crucial game objects to sync, because they respond directly to user input. They require higher frequency sampling and client-side prediction to prevent input latency. Always apply client-side prediction in server-authoritative games.
As I kept tweaking the synced prefabs, the lag and network artifacts gradually began disappearing, and our play session matches became more about skill and less about ping.
When it no longer made any difference who was hosting a game, and when multiplayer performance was indistinguishable from single player, I was finally happy with the results.
Server Hosting and P2P
Network traffic is normally routed via coherence Cloud Replication Servers. Replication Servers are high-performance relays designed specifically for the coherence protocol. They provide low latency replication, area-of-interest management, out-of-the-box host migration, server meshing and a bunch of high-level features used to scale huge and persistent worlds with massive player counts.
Another important feature that coherence offers is switching between dedicated servers and client-hosted servers - without rewriting your netcode.
This gives you the option to scale up to dedicated servers at any point during the game’s life cycle, and - just as important - scale back down to client-hosting, if you no longer want to pay for those dedicated servers. That means that you never risk bricking your game in case it underperforms, and you will never have to slap players with ridiculous fees just to afford online play.
It is really important to consider the platform and operational costs when planning out a multiplayer game. The coherence Cloud free tier includes 30k virtual credits, that can be converted into 30k CCU hours / 150GB of egress bandwidth (or any combination of the two) per month, depending on the game's needs. From there on, server costs scale automatically with usage.
In the case of Hellfront, the free tier is more than enough. However, coherence recently added support for routing traffic via 3rd party P2P relays, like Steam Networking, Epic Online Services, and Microsoft PlayFab Party. P2P relays offer a limited feature set compared to coherence Replication Servers, with higher latency, reduced bandwidth, and limited crossplay support. On the plus side, they potentially allow you to operate your game’s online multiplayer at no cost and with no CCU limitations.
Since I was deeply involved in implementing Steam Networking support for coherence, I chose to give my own work a proper real-world test and try out Steam Networking for Hellfront.
Night Of The ONLINEAGEDDON
It was getting close to launch. I had thrown out and replaced the ugly placeholder UIs, updated the in-game menus and implemented a ton of small quality-of-life improvements.
Last minute, I added a skill-based ranking system and some new online Steam achievements.
Everything was wired up and ready to go.
We released the online update to the public on 18 April 2024 and broadcast it live on Steam.
The event - dubbed “The Onlineageddon” - saw developers pitted against the best PvP players from the community - and holy cow did we play some epic matches! The release went off pretty much without a hitch, no bugs and no lag, even when playing across the Atlantic.
I've made a couple more minor updates since the initial release, fixing small glitches and annoyances that players have reported. Player feedback, in general, has been super positive, and now we get requests for more features, like new weapons, turret upgrades and a 2v2 game mode.
So while the game has not become a viral hit yet, we definitely see increased engagement from both new and old players who all really enjoy the online mode (especially 4 player deathmatch, it is INTENSE!).
And in the end, it is the players who make this all possible and meaningful. To me, it is all about the connections we make along the way. This project has reconnected me with friends and fans who used to play old Skygoblin's games from over 10 years ago. It almost felt like a reunion of sorts. This sense of community is what I love about making games.
Final Thoughts
This was a successful project. And fun!
I guess my main takeaway is that multiplayer doesn't have to be painful, if you use the right tools.
Adding multiplayer late in the development process is really only hard if it requires you to restructure your game and assets. While most netcode solutions use specialized classes (NetworkVariable, NetworkBehaviour, NetworkTransform etc.) that force you to replace code and Prefabs, coherence works as an additive framework that extends existing functionality with minimal interference to your workflow.
Here are my best networking tips, in no particular order:
- Parrelsync + local Replication Server = short iteration times
- Go for the low-hanging fruit when optimizing netcode: position and rotation.
- Use cheap broadcast RPCs to trigger local instantiation of short-lived game objects (e.g., particles and vfx).
- Use [Sync]ed properties to network private data. Anything that can't be synced out-of-the-box can be accessed using simple getters and setters.
- Start testing ASAP.
- Always strive to keep iteration times down - every extra second spent compiling/building/uploading will accumulate over time.
Don’t be afraid to pursue your dream project.
Good Luck, Have Fun!
Written By