12:00am EST - Another week in Rust development and we’re seeing more work on the upcoming Minicoptor, some new electrical components, a fresh camera model, and more.
Let's talk about performance
It’s no secret, performance on our servers hasn’t been the greatest recently. If you play on or host a popular server, you know we’re not alone in this. Performance across the board has been slipping for months now, and it’s reached critical mass.
What took so long to say something?
We constantly monitor the performance of our servers and are in close communication with the Facepunch developers when we experience poor performance. Although I’m ever present to this, I don’t make it a habit of harping on it. It is not my intention to bust Facepunch’s balls - I greatly appreciate what they do and fucking love the game they have created. They are always very responsive when we bring issues up, and I give them the benefit of the doubt that they will resolve issues in a reasonable amount of time.
I don’t like to leave the community in the dark as to what is going on though. Therefore, I’ve decided to fully flesh out where things are at, what we’re doing about it, what you can do about it, and where we go from here. This mainly touches on server performance, although we all know client performance could also use some work.
What’s causing this?
A server will run decently for a period of time, then at some point, performance will take a dip, and gameplay becomes either sluggish or completely unplayable. Very rarely does a server recover fully without a restart - and even then, restarts just kick the can down the proverbial road.
There doesn’t appear to be one single cause of these slowdowns. Rather, it’s likely a culmination of various changes and additions to the game over time has gotten us to this point. Historically, AI has been a main culprit. Between the cargo ship, bandit town, compound, junkpile scientists, animals, and everything else, there is more AI in the game than ever before. The presence of the cargo ship in particular appears to be a catalyst in these dips. It’s not clear if this event is the main cause of the server slowing or simply the straw that's breaking the camel's back.
We also have electricity as a new part of the equation. Although Helk specifically throttled the amount of electrical requests which can be processed by the server at a specific time, there is definitely some overhead involved with this system. All this doesn’t even touch on the shortcomings of the underlying engine of the game, Unity.
In all fairness, Rust is really pushing the boundaries farther than pretty much any other game out there. Each server has potentially hundreds of people at a time gathering, hunting, building, raiding, and the like. It’s really incredible when you think about it, and the sheer massiveness of the scope is one of the things that makes Rust so special.
By the same token, when every action is delayed by seconds, or you end up dying and losing hours of work due to lag, the magic of the game is quickly replaced by anger and frustration. The very things which make the game amazing actually exacerbate the need for optimal performance.
What are we doing to combat this?
As a server owner, this is disheartening to say the least. When complaints of lag flood our Discord, I feel it in the pit of my gut; I cringe when I see posts referencing the poor performance on our servers.
Our Systems Team and I have been doing everything in our power to remedy the situation. We just purchased two new servers with Coffee Lake processors in our US location. This is not the first time we’ve thrown more hardware at the situation and surely won’t be the last - we are always on the lookout for better hardware to be available from our host. Along those lines, we also rented new hardware in the AU last month so each server in that location has its own dedicated box.
In fact, we currently run 21 dedicated boxes to host our 31 Rust servers. All of our larger servers have their own dedicated box to run the single instance of Rust. If you have any experience with hosting servers for other games, you know this is fucking insane.
In addition to this, we are running server restarts much more frequently and have set up special alerts to inform our Systems Team when a server’s FPS drops to single digits consistently. We also, as always, are working with the developers at Facepunch to provide them with any information we can to help identify the biggest culprits of low performance.
What should you do?
If you’re playing in our servers and performance drops consistently for 5 minutes or more, please let us know in Discord and our Systems Team will look into it ASAP. Please keep in mind that a server restart isn’t a silver bullet and may not always be our immediate course of action.
If you’re looking for a Rust experience that is as smooth as possible, I recommend finding a very low population community server (of which there are numerous great ones). If you only like playing on high pop servers, you may want to consider taking a break from Rust all together till the performance pendulum swings in the other direction.
If you’re a server owner, I suggestion playing with disabling or limiting the amount of AI present on your server as this tends to be one of the largest performance hogs. Unfortunately, this is not an option for Rustafied servers as it fails to qualify as an official vanilla experience.
Where do we go from here?
The ball is in the developer’s court at this point. Thankfully, both Helk and Alistair have told me they are looking into it this month. Will we see some relief in the February update? That remains to be seen, but I sure as fuck hope so.
Thank you for your patience in this matter. I am a strong believer that the best days of Rust are still ahead of us and I am excited to be here, reporting on development and providing the best server experience we possibly can.
Work on the newest vehicle to take flight in Rust continues this week. We’ve seen progress on animations and poses, along with work on the core functionality of the Minicoptor. Testing in game isn’t quite possible yet as this work is still being done on a separate branch. That said, it does appear as though this new vehicle may make its way into the next update. Stay tuned over the coming weeks for more as this progresses.
New electrical components
Helk is working on a couple new items for electricity. Specifically, a new type of switch is on the way. The rand switch, although still on a separate branch, will likely be just what it sounds like - a switch that flows current through a random direction. More on this as we get closer to the update.
New camera model in game
You’ll be a bit more stylish when snapping shots in game of your friends after the next update. The camera model has received a facelift. Gone is the old Unity asset store model, a new custom design from Tom has made its way in game.
Shifts to water being worked on (including the addition of a wake for rowboats)
More progress on the Oil Rig monument
Lighting tweaks to Military Tunnels
No longer force full snapshots when spawning / respawning
Loading screen status tweaks when spawning / respawning
Add WorkshopSkin and MaterialReplacement components to skinned entities in prefab preprocessing to avoid runtime stutter (performance improvement?! Looks like it!)