Good Morning Friends. I think we need to take a moment really quickly to appreciate just how damned good the atmosphere in New World is. In the above shot, I am sitting in Weavers Fen, which is on the Eastern edge of the landmass and is largely represented by bogs, mires, and other colorful terms for swampland. Were this Magic the Gathering, the Black Mana would be flowing freely… and to be honest given that the zone is heavily populated with ghosts and gravediggers… who is to say it already isn’t? What would a town look like in this region? A ramshackle series of buildings on stilts and boardwalks that are trying to sink into the ground of course. This looks like the town you would expect in that region and I greatly appreciate the little touches the wooden planks already half-submerged and buildings that have been lost to the swamp on the outskirts.
I am in Weavers Fen because quests sent me there, most specifically the level 30-40ish faction upgrade mission. Every so often you cap out your faction experience and have to perform a trial. The first of these sent me to fight through a town of ghosts that were level 27 and included fighting an elite one at the very end. This time around it sent me far to the north of the zone… to what I thought was a level 40 gold border champion area designed for five players. So after running Amrine my solemn crew agreed to head forth to help Omali and I complete this faction quest. I fully expect to be called upon to do the same when Vernie and Waren get to that level. The thing is… turns out the mobs that we actually needed to kill were not champions but were on the far side of that outdoor dungeon… with no clear path to them except for through.
We took it slow and steady as we crossed the zone and had to hunt drops at three different locations before finally being able to turn back in at the settlement in Weavers Fen. Originally I wanted to get this through because I noticed there were some level 35 equippable items on the faction vendor, and in Amrine I had just dinged that level. Unfortunately, I was not paying super close attention and it appears that almost all of the armor is level 40. However, that does give me time to do faction quests and earn the currency to pick up a set of gear waiting for me for that level range. I have mostly hit level 35 through crafting and completing quests in the starter zones, which means I have the vast majority of content in Brightwood, Cutlass Keys, and Weavers Fen to explore, and as a result I expect that hitting 40 won’t be terribly challenging.
My core focus of late has been being able to get up to level 100 in Engineering. This will ultimately allow me to craft Starmetal Tools, which is something that I want to accomplish before considering a jump to a lower population server. Essentially, I know that I would likely be losing access to a number of tier 4 crafting machinery, and instead of allowing that to delay my progress, I want to be able to take with me the resources that would ultimately give me the edge in the new environment. In theory, Starmetal tools should come with three traits on them, and I have enough of the material bank that I should be able to craft a few different versions of the key tools like Pickaxe and Logging Axe. So my goal is before the end of the day to have crafted a few of these. While I also cannot equip them until level 40, I can at least bank them for future use like I am going to start doing with my set of armor.
One of the things that drive me up a wall in New World is the fact that once you block someone… they will forever show up on your social list at the bottom. When you block or mute someone you want them to be whisked away from your sight, never to be noticed again. However, for some reason, Amazon devs thought it was a grand ole time to have these names constantly show up in your social feed. This is ultimately stopping me from going even more mute/block happy because I don’t want to keep clogging my screen and instead find it better just to completely ignore global chat. One thing that I did notice last night is that the queue times never really did get that insanely high. When I logged out around 9:30 CST the queue was reported to be only about 10 minutes. Not sure if players are giving up or simply rerolling like some of our crew has on a lower pop server. I think ultimately it is because of this that I want to see how the server transfers tilt the balance for a number of servers.
This weekend I was not certain how much time I would get to play, because the weekend tends to be when we do things. For the most part, I got to hang out and play New World all Saturday because it was raining buckets outside, and neither of us had the desire to leave the house. Sunday however it became rapidly clear just how untenable our current server situation is. Because of just the nature of how our Sundays go, I spent time in queues three different times for a grand total of four and a half hours spent watching this box tick down. Essentially if you cannot log in before noon, you are not going to have what I would call a reasonable experience. Thankfully my machine is capable of running multiple games at the same time so on the longest of these queues… 2.5 hours, I spent time over in Final Fantasy XIV doing roulette.
One thing that I can attest to with a fairly high degree of confidence is that the tool we have been using to estimate how long a queue is… is pretty freaking accurate. The three queues that I entered yesterday were pretty much dead on for what the queue estimates were at the time of me logging in. These are some sample queues from the site at the time of writing this, and I sorted them by the most players in a given queue. This more or less tells the tale of New World right now. Notice that a few servers have seemingly been bumped up to 2250 players as a test, but I am starting to doubt that they plan on scaling these up very much higher. In fact 2000 players seems to be an important number for the design and stability of this game. I provided some server data on Friday when I wrote about the game and since then 131 more servers have been added… only further increasing the fragmentation of the player base. The current data centers look something like this:
Frankfurt, Germany – 230 Servers
Arlington Virginia, United States – 200 Servers
Sydney, Australia – 70 Servers
Umatilla Oregon, United States – 64 Servers
Sao Paulo, Brazil – 44 Servers
Amazon has made some attempts to flag players who are trying to get around the AFK timeout and force them out of the game. Cities right now are rife with players running in place against obstacles or auto-attacking while standing still via a macro. So while I am happy to see them making strides in keeping this from happening, I also don’t think it is going to really go very far in solving the problem. The core issue is that the day one, day two, and probably even day three servers are way too populated.
Amazon has officially locked many of these overly populated servers so that new characters cannot be rolled on them. However, this action probably came a little too late for it to make a difference either. If you have a fixed server size in mind as they did… maybe stop character creation at 2.5 times that number? On opening day there were servers with queues that were upwards of four times the size of the total number of players allowed on a server. Ultimately I am just not certain that staying where we are is going to be tenable much longer. There has been a discussion that Amazon is trying to rush a solution to allow players to migrate elsewhere, and right now I am thinking that is our next best hope. Once this opens up, find a smaller server and then coordinate a move to that location so that we can recreate the company again on the other side.
There is a lot of really fun group content available in this game. I spent some time over the weekend closing “Rifts” aka corruption breaches and they were great. I also got pulled into a random group with players from various factions and did Amrine Expedition the first dungeon. All of this is pretty awesome content, but also things that would have been so much more enjoyable had I been able to do it with my friends. The core problem with the game right now is actually getting players online. It is making all of the core systems function a little worse as a result. For example, I watched a video from a YouTuber that talked about how hard it is to make the War system work right now because they just can’t reliably get players online at a fixed time in order to meet up and defend a territory.
There are times I am committed and connected to the community on a server. For example, I would refuse to ever move away from Cactuar in Final Fantasy XIV, because that server is just phenomenal. In New World, however… I think pretty much every server is a dumpster fire right now and Minda is not really a place I care about significantly. I am deeply connected to my character and would not want to re-roll elsewhere. I am deeply connected with the people I am playing with… whenever we can actually play together. I am not however connected to any given server, so far as I am concerned our best bet is to move. This is going to still have some pain associated with it because I am in a place where I need those tier 4 crafting machines, but I can deal with being delayed if it means we can all play the game in a reasonable manner.
I think the plan going forward is to wait for the server transfer tool to open up, and see what it does to server populations. Then find a relatively low to medium population server and transplant our little group of players. The players that rerolled on low pop servers are seeing a lot more of these sweet sweet loading screens than those who have stuck by Minda. At level 33, I am too deep to contemplate starting over, so my last hope is for a server transfer.
Featuring: Belghast, Grace, Kodra, Tamrielo, and Thalen
Tonight we are back after taking last week off for Tam’s wedding. We also record an exceptionally long show because we got into the topic… and were not watching the clock terribly closely. So it clocks in at around an hour and forty minutes of talking about Amazon’s New World release and all of the good and bad that comes with that. While this makes for an interesting show… it makes for some pretty boring show notes.
Good Morning Friends. I find myself still here and still playing New World. I did however take a break and play some Final Fantasy XIV… while the New World client was up and idling in the background. Yesterday I logged in over lunch to about a thirty-minute queue, and that was enough anxiety to have me just leave the client open all day long. After yesterday’s blog post, I took to “the twitters” and threaded up some commentary for anyone who might not read my blog. My advice this morning is still pretty much the same as that thread, if you are not already bought into the game then I would highly suggest just waiting to pick it up until the server congestion situation is more reasonable.
The New World team took to their Twitter account and posted this message, trying to put a positive spin on the current situation. The tweet talks about how many servers they created on the fly and how they are working to increase capacity on each server. It also goes on to say that if people just pick any available server now they are working on a way to get folks transferred onto a server with their friends. The problem here is this fundamentally is not how an MMORPG works. You need those friends the most in that early rush as you are attempting to do dungeons and collaborate on getting folks geared. Even if you are not actively playing with other folks, there is something necessary about just knowing that friendly voices are out there waiting on the other end of guild chat. There is something deeply flawed in the design of this game and I will go into detail a bit but first, we need to talk through some numbers.
Right now at this very moment, we have the following server capacity:
United States East Region – 158 Servers available
South America East Region – 35 Servers available
Europe Central Region – 195 Servers available
Asia/Pacific Southeast Region – 40 Servers available
United States West Region – 49 Servers available
Looking through this you notice some strange data points. First off the EU region has significantly more servers than I would have expected at this point based on the sizing data from other MMORPGs. The United States West Coast region seems grossly undersized to what I would normally have expected based on my experience with other games as well. In grand total, you have 477 servers and if my math is correct a maximum capacity of 954,000 players could be playing the game at any time. There is a maximum single server capacity of 2000 players based on the metrics we have available to us, and there are many realms that have more players in the queue than actively playing on the server.
In yet another tweet thread, I surmise that there is something about their design model that is fundamentally easier to scale horizontally than to scale vertically. This 2000 player limit seems to be important, as does the 100 players in a single company. That tells me this game was more or less designed for 20 companies to be duking it out for the resources of Aeternum, which makes me wonder if New World more or less started its design cycle as a slightly larger scale Battle Royale game, and morphed into an MMORPG game as players violently rejected its PVPcentric design. We know without a shadow of a doubt that AWS can scale upwards extremely well, and you can very well keep throwing resources at a problem by adding more processing power, memory, and disk space ad infinitum. So that tells me there is something on a software level that makes it a challenge to scale above 2000 players in a single server.
Let’s talk about that 2000 player number a bit. Once upon a time both Everquest and Dark Age of Camelot showed the current player population of each server on the login screen. Quite frankly this was something they were both proud of because you have to understand… coming from games with a maximum player population of 16 in a single match, having thousands of players was quite the feat. The average peak for those servers generally speaking was in the 2000-3000 player range. As we move forward to the modern era of MMORPGs, I have heard that 6000 players are the point at which the login queue starts to kick in on a World of Warcraft server.
Final Fantasy XIV requires a bit more factoring to come up with some sort of estimate. Based on the player knowledge I have… most people do not use the Steam client, and I would guesstimate that at most 1/5rd of the player base connects that way because you have a huge group that plays through their console. There are 68 total servers in FFXIV, and Steam recently reached a peak concurrency with the game of about 67,000 so for sake of making things simple let’s call that 1000 steam players per server which again is going to be a deeply flawed number. If we are taking as an assumption that the number represents a fifth of the player base then my guess would be that would give you around 5000 concurrent players per FFXIV server.
So you might be asking yourself… Bel, why did you go through all of that contorted math that you know is going to be completely wrong? Just trying to paint you a picture of what the modern MMO landscape looks like as far as player population caps, and how 2000 per server seems extremely small based on the demand already existing in mature games. In an immature game, there is going to be even more of a burst of players that needs to be dealt with as folks are currently hungry for any new games coming out regardless of genre. New World also is a special game, because it represents the first completely new “Western” MMORPG in years. Western players have developed a pretty sour taste in their mouth for the microtransaction hellscape that are Korean and Chinese localizations. Based on the recent flurry of activity in Final Fantasy XIV, there should have been signs that the community was primed and ready to latch onto something new and fresh given that the entire World of Warcraft player base seems freshly unmoored.
So once again I fall back on, that there is something specific to their design model that indicates 2000 players as a specific unreachable boundary. Otherwise knowing what I know about AWS, it would have been far simpler to add additional processing power to the servers that were already provisioned than to keep spinning up new nodes. There has to be some reason on the software side that they are not doing this, however, and it makes me question if they are confident that they can scale upwards. Alpha and Beta testing didn’t really give them an idea of what a real-world player demand looks like. We went into those tests just happy to be playing, and not much caring if we were playing with our core group of friends. However, an MMORPG launched is a totally different beast, and players organize into dense clusters as everyone tries to make landfall on the beach that their friends already arrived on. Having more servers doesn’t really do much to dilute the demand placed on those first few servers.
Now we are going to swing back around to why the servers per region numbers look fucked. Amazon shot themselves in the foot with this launch and rolled the game out in a manner as to encourage players to dogpile a limited set of servers. Amazon rolled the servers out by region in a staggered manner… causing the EU and NA East servers to fill up completely because once players are established somewhere, there is entirely too much inertia to get them to move elsewhere when more appropriate servers are opened. The schedule looked a little something like this:
European Servers Opened at 11 pm PDT on 9/27
South American Servers Opened at 4 am PDT on 9/28
Asia/Pacific Servers Opened at 4 am PDT on 9/28
North America East Servers Opened at 5 am PDT on 9/28
North America West Servers Opened at 8 am PDT on 9/28
The first servers to come online were in the European Data Center, and there was a mad rush to get in and reserve your name. This is another flaw that added to the problems we are dealing with, but the fact that character names are globally locked and not tied to a specific server. As result, there were a lot of folks that popped into those servers to create a character reservation, and likely a handful of people who just wanted to play the game period regardless of server environment stuck around and started playing the game. The next big avalanche came when the East Coast servers opened, and since there is a relatively insignificant difference for most players between the two environments… every single major guild chose to start the game at this point.
Regardless of how painful the launch might have been for some players, the staggered launch and the fact that there was a land run on virtual real estate… aka player and company names made everything that much more important that you got there first. The problems that we are dealing with in this New World launch are absolutely problems of Amazon’s making, in part due to sheer lack of experience in dealing with the MMORPG player base. There is only about a 100 ms ping difference between me playing on my NA East server and playing on a European server, which means for most people the choice of where they landed was largely meaningless. New World was treated like a highly localized experience when time and time again the players have proven that they are a global audience.
So here we are with this botched launch and questionable game design… that we could easily walk away from were it not for the fact that when you ARE able to get into a server the game is damned addicting. New World is a great gameplay experience when it is operating under optimal conditions. My hope is that they have engineers working on how best to scale the servers, and as one of my guild members commented… if they doubled the capacity per node most of the queues would go away instantly. I think they need to do precisely that and at the same time offer free server moves at a company level. Let entire companies transplant themselves on greener pastures, because quite frankly unless they specifically are holding territory there isn’t a lot connecting people to a specific node at this point.
I hope they can make this work because the game legitimately is good. We haven’t had an MMORPG launch that was this solid in a very long time. My advice stills stand for anyone waiting and watching from the wings. Keep waiting. If they manage to pull out of this tailspin I will be the first to raise notice that it is time to start paying attention again. Until then, however, all that waits for you is a whole lot of frustration and anxiety buffered only by just how good the game actually is.
