Why Do We Ship Buggy Games? – A Look Behind the Scenes – Extra Credits

Every once in a little while Some big-budget game is released in a horribly buggy state and the gaming public at large asks: “Why with a hundred million dollars at their disposal Couldn’t the studio launch something that worked?” and “Why wasn’t the game delayed so that it could be fixed?” Today I will attempt to answer those questions. First, let me start off by saying that I am not defending the practice of launching buggy games. It’s bad business and it means that something got done wrong. But exactly where and when that something went wrong isn’t always as obvious as it first appears. On a macro level, it often boils down to two fairly straightforward categories: scheduling issues and production issues. Let’s start with the production side. Here, there are three possible causes for a bug-filled game getting released. One: we can’t fix it; two: it’ll cost more to fix than we will make fixing it; or much less frequently, three: we just didn’t catch it in testing. What players sometimes fail to realize, though, is that the “we can’t fix it” or the “it’ll cost more than we’ll make to fix it” scenarios aren’t a case of mere laziness and certainly not malice. Bugs like these are often the result of decisions made super early in the development process, and by the time the consequences of those decisions are visible there’s very little that can be done about them without a complete overhaul. For example, since we just recently had a case of this: Buggy animations. If a game ships with a load of obvious animation problems it could be the fault of incompetent animators but it probably isn’t. Often, issues like these are technical – a result of how the game engine handles animations; how the animation system itself works, and that system is built early in development – way before a single asset gets put into the game So you’re not likely to start running into the edge cases where that animation system starts breaking in unexpected ways until a lot later – once the game is mostly put together – and by that point the flawed animation system you’ve built is such an integral part of the game engine that removing or even altering it could break tons of other systems, setting development back months and leaving tons of expensive employees sitting on their hands for weeks. Now, truly great studios have all sorts of practices for testing these things early and lots of fail-safes to fall back on, but problems like these are really tricky and even the best studios sometimes get thrown when they hit a really fundamental or sometimes undiagnosable bug in some core part of their engine two-thirds of the way through production. And there’s another side to this problem: sometimes these unforeseen bugs aren’t even the studio’s fault – at least not directly. See, many studios don’t actually build a lot of their core engine anymore. Modern game engines are such a huge investment to build that it often makes much better business sense to license one of the many quality third-party options out there, like Unity or Unreal, and then flesh those out even further with middleware tools like FMOD or Havoc. It’s often the best option, but it does mean relying on a series of third parties. Every developer I know has a story of that time where they were waiting on a crucial feature which the engine company promised but never delivered, or of having to integrate a patch from a middleware developer that suddenly broke everything. And, by its very nature, the decision to use such software happens very early in the project cycle – well before any of those potential problems could possibly be foreseen. By the time bugs start surfacing or some critical feature never gets added to the engine, it’s too late to go back and that means that it’s often too late to ship anything but a buggy game. But perhaps even more frequently, buggy games get shipped due to scheduling issues Players will often ask things like, “Why don’t they just push back the release date?” But a lot of people don’t realize just how much is tied up in that release date. Marketing spend can begin as much as a year in advance of release. And things like magazine covers and TV ad buys?
Those are often locked-in months and months before launch. These aren’t things which you can just reschedule at will – at least not without huge expense. Then there’s launch windows in general. Everybody can probably think of some game that they love which didn’t get the attention it really deserved because its launch week was dominated by something like a Call of Duty or any given Blizzard game. Publishers really want to avoid those scenarios, too so they carefully coordinate all of their launches and try with varying degrees of success to make sure that all of their games get at least a week where they don’t have to compete with some other major title for attention. Pushing back a game’s launch even just a fortnight can put it right in the shadow of a competitor and screw all of that up. Then, there’s the need to move on to other projects. Major AAA developers, especially ones owned by a publisher, move – or at least are supposed to move – like clockwork. When a title is done, that team is rolled onto a new project or sometimes they’re temporarily brought in to help on other titles that the publisher studios are working on, while a portion of the team spins up the next project. This means that delaying a game isn’t just delaying that game, it’s also delaying that team’s next game or delaying projects that team was meant to help with after launching their own. And this can be an even greater issue when working with outsourced developers, because today you’ll often find that some assets, levels, or parts of the game weren’t built by the main studio, but by some outsourced team elsewhere in the world. And these external studios? They’ve got to move from one project to the next too. They’ve got their own production schedule to keep. If some unexpected bug bubbles to the surface and that outsourced content, many of these external studios are actually great about going above and beyond to try and help fix it. But if they’ve already moved on to another project by the time that bug is encountered, they may not be able to help, and if so the main studio on the game either has to run with what they’ve got or remake all of that work themselves, putting things further and further behind. And I know it’s easy for us as players to brush off factors like increased development costs, because the publisher or the studio having to spend more money isn’t our problem. But this is a business. AAA games represent highly calculated risks – a precision investment of millions of dollars based on a carefully calculated prediction of how well that game might sell. And sometimes, not always, but sometimes, as much as the artist in me hurts to say it, sometimes the objectively better business decision is to not sink more millions into that investment. So should we be shipping buggy games? Obviously not. We have to keep looking for ways to test things early and to leave ourselves some fall-back options if things go wrong. And sometimes this stuff is just a matter of studio discipline. But, when you’re building software as complex as this, bugs are gonna sneak up on the best of us, and it’s often not as simple as just delaying the game a few weeks to iron those bugs out. Sometimes, due to decisions made months or even years before launch, those bugs are embedded into some part of the engine that can’t be replaced or sometimes those bugs are out of the studio’s hands entirely, still waiting on an update from a middleware developer that never happened. And sometimes, even when it is theoretically possible for the team to iron those bugs out, the potential cost of doing so is simply so great, with so many irreversible decisions about marketing spend and release schedules already made, that the publisher chooses to move forward anyway. And I don’t say any of this to excuse releasing broken buggy titles nor am I saying that it’s wrong for you to be upset about being sold a broken buggy title. I’m just saying to be aware that the bugs in that broken buggy title – they might be more than a delay could ever fix. See you next week.

100 thoughts on “Why Do We Ship Buggy Games? – A Look Behind the Scenes – Extra Credits

  1. I think that the the broken game with a first week full game patch is a way to fight pirates.

    Pirates race with each other for whom will get the first crack of a new game. After the first crack there is not that much reason to make a secon one.

    After that devs will make a version 1.1 patch with a bug fix they already got on there hands even before the launch plus a small think that will change the files just enough so that the crack would't work.

    After the second week. The buyers have a good fixed game and the pirates have a broken mess.

    Thats only my spiculation

  2. so don't spend more so people won't want to buy the game, hire other people to do it so we don't have to, and if it is broke just don't fix it. Yeah that makes sense.

  3. This is another video I don't fully buy (I'm binge-watching them at the mo, and I have just subscribed!). I'd like some more specific examples. I think Bethesda titles are a prime example of this – they're so buggy upon release that it's become expected, and even they joke about it. It's been widely noted that bugs in Fallout 76 existed in earlier iterations of the game (something which modders have been able to fix). How can this be excused as anything other than 'lazy developers'? This is a studio that's using the same game engine they did a decade ago – they know what it can and can't do inside out, or at least they should. It may seem like cherry picking with Bethesda, but let's face it they're a big player.

    I work in books publishing. Much of what I publish is technical (e.g. university level textbooks) and you publish it with mistakes you'll get your head ripped off consumers. You simply CANNOT release a maths textbook if it has mistakes in the maths formatting. In scientific texts, publishing an errata (the equivalent of a patch – hey, equation 4 was wrong, here's the correct version) is a big deal – there are entire websites dedicated to keeping track of them. If something is wrong in a peer-reviewed (alpha-tested) scientific text gets published then someone has massively screwed up, and heads roll. I often work on medical texts, and without any exaggeration you make a mistake and people die (literally). There have been cases where pharmaceutical lab books have had the wrong dosages (sometimes as extreme as grams instead of milligrams being administered) published resulting in patient deaths (and costly litigation).

    What I do isn't on the same complexity as a AAA video game, which have many more variables to piece together, but I think the principle is the same. We have lots and lots of safe guards and rounds of checking by multiple pairs of eyes to ensure that it doesn't happen. A buggy game release, and I'm not talking about the occasional animation glitch, but stuff that crashes my computer that wastes hours of my time is unacceptable. Ethically, there's no way consumers should be sold these games. No other industry gets away with it like video games do, and I blame gamers for a lot of this by buying into gimmicks like pre-ordering because it has 'Fallout' written on it. (I didn't buy FO76 because it looked crap from the start.)

    In books publishing you cannot patch a text – you've printed thousands of copies of the book at great expense and it's shipped out around the world. Making a mistake isn't an option, especially a critical one (i.e. beyond a little typo, although it's soul-destroying when you spot one after a text has been printed). You couldn't do it easily before the internet with games either. I even had a phone call from my car's garage recently saying that my car needed to go in for essential and urgent work because of a fault – you take that seriously when your car has a potentially serious safety problem. It didn't cost me anything, but it cost the manufacturer (I imagine) a lot! I'm sure Volkswagen understand this with their emissions problems last year. Product recalls because of a fault are serious, especially if there's people's health and safety at risk (all food products, medicines, vehicles, all manner of consumer goods, etc.) and national standards/laws/guidelines are not fulfilled.

    Bottom line is we all have jobs and we all offer a service or product of some description. Video game developers (and politicians) get away with murder compared to most businesses.

  4. But can't you have legal contracts with these third parties that obligate them to addressing bugs, if not before release, at least with patches?

    Back to that marketing issue. I commented on another video of yours about the cost of games, I don't think such marketing is a good idea. Especially TV ads hyping the games over a year before its scheduled to release IMO is a horrible idea. The money is better spent on the game with reviewers, general release announcements and word of mouth being your best advertisement. And it makes it so you aren't locked into a specific date.

    And I think lately developers have gotten too dependent on selling games before they are finished. With PC they got this category called early access. People get to pay for incomplete games and to be unpaid(pay themselves to be) bug testers. All the while risking the game never being properly finished. And with console systems they release games and you can download "patches" that are 50gb big right away. Having to download the game from disk onto the hard drive and then patch from internet down onto the hard drive, before they can even play and that STILL might not address all bugs. Compare to a company like Nintendo that have a quality control department that makes sure their releases are bug free. You rarely found Super Nintendo games that were substantially buggy because the games could not be patched. That means you had to get it right the first time, and that means budgeting in a proper amount of quality control. Shift that wasted money on advertisement onto quality control where you are thorough the whole way through, and walla, a more reliable product.

    And lets say you got a buggy game you essentially have to start over from scratch on because the bug is buried deep in. You said advertisement can be the cost of the game again, well then do the cost of the game again, again. But it shouldn't have to be the full cost of the game all over again. Even if if the base code has to be rewritten, the ideas about how to make the game play, level design etc still exist. So it wouldn't be the cost of the game again. Even if it means delaying a game for years. If you release something really bad in bugs and don't have a patch quickly lined up, you are tanking all future games of that line. It comes down to branding. You make a good game, you are trading on that quality and concept next game you release under that title. If you release a stinker, the whole series usually turns into a dead end (very rare exceptions, but the stinker game will still haunt the series even with successful games after) That's alot of potential profit down the drain.

  5. The main issue is that once the game is released full of bugs, after many years of patches the game is still unstable, not optimized at all, in fact is still a garbage. What's the excuse then? We should receive money back if we don't receive the quality stuff.

  6. Since you eluded to it…

    Every problem with Mass Effect Andromeda was either a direct or an indirect result of forcing them to use Frostbite instead of Unreal. It meant that the team had to learn a whole new engine (instead of the one they already made 4 successful games in) and it also meant that certain sections of the engine – mostly the RPG systems – could not be updated from previous games, but instead had to be rewritten from scratch. It also mandated a change in graphics programs (I forget whether it was from 3DS to Maya or the other way around) which meant the same issues plagued the art teams.

    In short, EA's decision to use Frostbite caused, either directly or indirectly, EVERY other issue in Andromeda, save maybe some of the writing which really wasn't up to the original trilogy's standards. And why did EA do this? Because they could license Frostbite for free from themselves (err DICE) instead of paying Epic for another Unreal license.

    In the end, it probably cost them far more, both in trust and money, than just licensing Unreal Engine 4 would have, which would've looked just as pretty and ran better (Unreal has always been better optimized than Frostbite, due to offloading a ton of work onto the CPU that Frostbite, CryEngine, and others always stick on the GPU). In the end, this one decision is why Andromeda was a bug-ridden mess, and it was ENTIRELY the publisher's fault because they wanted to save a tiny pittance of money (comparatively speaking) up front without understanding the costs to the development team, and thus, the game itself.

    But it's EA so I'm sure they convinced their shareholders this was a win, somehow, and that's all that matters to EA.

  7. Skyrim comes to mind with the specific release date being integral to marketing. 11-11-11 is not a very repeatable date.

  8. Watching this in 2019 having Battlefield V really enlightens me because when EA/DICE delayed BFV I was super angry about but now I can kind of see where they came from.

  9. Creating a game is a lot more difficult and stressfull then we realize, espescially if companies have already established themselves as competent and qualified.

  10. Total War: Three Kingdoms versus Imperator:Rome. TW already delayed their release, ensuring that I will not buy that game for a long time.

  11. Battlefleet Gothic Armada ll was one buggy mess of a game and even one year after release it was 't all fixed not to mention it had way less mp content han the first title. And then they had the audacity to put a chaos campaign dlc out there for 15 bucks while I still couldn't continue my tyranid campaign due to game breaking bugs.
    The truly infuriating thing is that they tread players who paid the full 60€ amount as not only free but paying beta testers.

  12. anthem didnt even exist in a core concept until the manufactured trailer shown at the E3 show. some points you did in the videos are valid, but not on anthem, that awful excuse for a game.

  13. And know you wonder why Bethesda hasn't fixed their obvious flaw as they continually released buggy games and go back on promises.

  14. I long ago stopped buying games until after reviews are out. buggy game? I won't buy it at all. I dodged the SimCity DRM debacle, even though I was dying for that game to come out at the time.

  15. I get this. However I think the issue comes when it seems like some developers seem to do this consistently. Like ubisofts bugs have become a meme at some point where you would be expected to fall through the earth forever, have your skin disappear leaving just your mouth and eyes behind, etc. It seems ton always be the same or similar bugs happening over and over. Can't catch/fix it the first time, I get it. It's expensive, like you said. But when it happens to your next game, and the next one, and the next one…I'm gonna raise an eyebrow.

  16. What these game companies don’t seem to understand is that releasing a buggy game does actually cost a lot. They’re typically mostly concerned with short term sales. This quarter or this year. So they don’t want to mess up their production schedule by fixing the bugs. But releasing a buggy game might actually end up being more expensive than fixing the bugs when you consider long-term sales. A buggy game damages the company’s reputation, making players less likely to recommend that game to their friends, and less likely to buy from that company again. If you go ahead and release a buggy game, your first week sales might still be great, but your yearlong sales will be far lower than they otherwise would’ve been, and it might be the first step in completely destroying your company.

  17. 2017: "Every once in a while, a big budget game is released in a buggy state."
    2019: "Every once in a while, a big budget game is released in an at least mostly playable state."

  18. I know that is the classical way how software projects often work, but in an ideal software project you could replace any module with a new one without breaking anything. It's all about interfaces and modularity (high cohesion within the module and loose coupling between the modules)… Sometimes you see projects where this is done really nicely and it is really fun to work with such code… Actually, it's embarrassing how often this goes wrong. This is also why very early I introduce a lot of structure in my projects (if I have the authority to do so) and always try to keep the exposed interface as slim and generic as possible. At the beginning this feels pretty overengineered. When you just have a few ten thousand lines of code, but already multiple modules and namespaces, etc.. But later on, when the code base grows, this turns out to be a big plus… The worst is actually when you work on the project for a year already and then you notice that you're missing modularity and start losing the overview over the code. At that point it's already quite some work to fix it, but it's still possible. I've also done this on a project 1.5 years in already. But if it's a big project that already runs for some years, there is no chance to fix this with a reasonable effort… Do it right from the beginning! Bad code structure is something you can see early on and this will help you "in the late game", when the product is heading towards the first release and you have to straighten out bugs, etc.. These nasty bitches always occur in bulk more and more towards the end of the project, as more and more people are doing explorative testing on the prototypes. And if your code structure helps you to track that shit down, you get a lot more room to breathe at the end. (Which doesn't mean that a release is relaxing ever – but the final product is just better if you could find and eliminate more bugs)

  19. Here’s a bug that was intentional: a call of duty game, (a top down strategy) was released on mobile, (no, not call of duty mobile) and I begged for it, and bought for 10 dollars….

  20. You're not taking into consideration one other major important factor: consumers accept it. If the majority of consumers didnt accept it, game devs wouldnt release a broken game.

  21. cough cough Lego Star Wars 2 for DS Cough Cough
    Seriously, it was so bad LucasArts had to offer refunds/exchanges for other versions.

  22. I work in steel. If I make a mistake, I am forced to fix it. If I have a "bug" in a buildings structure, I can't just say "Nope. Too expensive to fix. I'm not going to do it."

    I feel like everyone should be held to a similar standard. But that's just me. I'm kind of a jerk. 🙂

  23. Once in a while heh? Even in 2017 this trend has started to become common. Nowadays the absolute majority of games, especially triple A games, are released buggy, broken and unfinished. The whole games as a service model is reinforcing this attitude of fixing it later, if at all. All these excuses about development costs and release windows are just that, excuses. At the end of the day it's about standards and acceptance. If you were to buy a new car and than realize it wasn't completely finished or there was an issue with the brakes would you accept that and just move on? Hell no, you'll be up in arms demanding a refund and suing the car company for endangering you for driving without functional brakes on a brand new car. I mean sure brakes are an extreme example but even something smaller like the windows failing to roll down would lead to a similar result(barring the lawsuit, which would be altered or outright forsaken). So why is it that when it comes to games we're so much more accepting and making excuses for the developers/publishers rather than call them out for this phenomenon and demanding for it to stop?

  24. Don't support companies that just want to make money! We can patiently wait as long as the make the game good enough. On the other hand a bugged game is unacceptable. We're the customers. We're always right!

  25. I don't care about their excuses or reasons. A bug is a bug. It must NOT be there. I pay for a good time, not for a semi good time with the occasional cockroach trying to ruin it. It's like presenting a point that because some moron in a bread baking factory that supplies a city, made some early days fuck up when he/she was setting up the production line, then a month down the line they figure out they'll need to tear down the entire factory and redo it otherwise it'll keep puking out half baked bread, and the project manager trying to tell the people to "live with it, it is what it is". That's not going to fly. And it's not going to fly here either. If they aren't willing to take the blow for their fuck up, they should stop making games. The users must NOT be the collateral.

  26. If the average gamer stops being a bottom-feeder gratefully eating all the sh*t greedy publishers and lazy developers throw at them for a premium and the industry stops treating QA testers as expendable slaves, this problem would vanish overnight. Game creation is an art, a science and a business, yet the AAA industry treats it exclusively as business, and that's the saddest part.

Leave a Reply

Your email address will not be published. Required fields are marked *