Dark Theme | Category: Educational, Gaming
Cyberpunk 2077: A QA Engineer’s Perspective
[Spoilers ahead! I’m not that far along yet, but if you want every aspect of the game to remain a mystery until you’ve had a chance to encounter it for yourself, don’t read this until you’ve completed The Heist]
I stood on the street corner and waited for the car I hailed to drive itself to my location. As it approached, I watched it swerve slightly to the left, hit a guard rail, shoot up into the air, and flip three times before landing in front of me. Fortunately, my precious beater still drove like a charm – or would have, if I had mastered the delicate controls. As it was, I managed to swerve down the street without hitting any pedestrians – this is Cyberpunk 2077, after all, not GTA V.
>> Necessary Disclaimer << I don’t think the Cyberpunk 2077 player base or users in general should necessarily just grin and bear it whenever they encounter a legitimate issue that genuinely has a negative impact on their experience. By all means, report bugs. Just remember when you do that studios, no matter how big or small, are ultimately comprised of people – human beings with feelings like you and me – doing their best. Have some compassion and patience. Report bugs from a place of wanting to help improve the experience for everyone, rather than just flaming the devs because you’re frustrated. And please, for the love of Gaming, keep it all in perspective! Like my former boss used to say: “We’re not doctors or lawyers; no one’s dying or going to jail.” Alright, let’s dive in.
Since it was released on December 9th, Cyberpunk 2077 has been, well… whined about. If you’re not familiar with the saga, the game was announced in 2018 for console and originally slated for release in April 2020. It was then pushed back three times, to September, November, and finally December. Each time, eager fans clamored for CD Projekt Red to just release the game already. Now that they finally have, players are complaining about every minor imperfection, and frankly, I’m freakin’ over it.
Admittedly, that may have something to do with my job. Although my title is officially Tech Support Engineer / Junior Developer I, the actual function of my job is QA. Finding and fixing bugs is my bread and butter. I know how many clients use our platform each day without incident, blissfully unaware of any issues at all, and on the other side, I know how many bug tickets are in our queue. I also understand that not all bugs are equal: some are minor irritants, and some stop prevent people from doing their jobs. So when I encounter a bug in the wild, my first response isn’t to take to Reddit and spew my outrage impotently across a screen – it’s to find a workaround. With that in mind, let’s talk about some of the bugs I’ve encountered in Cyberpunk 2077 so far.
First, there are some minor physics glitches – my car pirouetting through the sky, for example, or being blasted into a field when attempting to climb through an open window (my partner found that one). Honestly, I find those amusing. Then there was a glitch at the end of the mission The Pickup, when you’re fighting your way out of Maelstrom’s lair. I opted to hack the chip before giving it to them, so the gang was on my side against the corporate goons. Once those suckers were defeated, I had an objective to follow Dum Dum out of the building – except, Dum Dum wasn’t budging. I popped online, did a little searching, and found a source that said it was because my accomplice, Jackie Welles, gets stuck somewhere and the fix is to reload the game, get the corpo goons to shoot at you at a certain point, and make sure Jackie follows you through a particular doorway. Except for me, Jackie was right there, just chilling. Even so, I reloaded from my last auto save and this time the mission wrapped up without a hitch.
I won’t lie, I was a bit disappointed at losing out on a rare something-or-other I got off one of the corpo robots the first time around that wasn’t there on the second. However, on the second go-round I figured out how to track optional objectives and was able to find and free Royce, and now he owes me a favor. I’m not sure if that’ll come in handy later (something tells me it will), but regardless, I feel better about having rescued an ally. Plus, the game autosaves frequently on top of allowing manual saves, so as long as you follow the age-old principle of S.O.S. (Save Often, Superstar) you won’t lose much progress if you do have to backtrack.
One final glitch I’ve encountered so far occurred during the braindance in preparation for The Heist. If you’re not there yet, a braindance is part memory, part simulation. You can fast forward, rewind, pause, play, and scan the environment throughout for visual, audio, and thermal information. The three tracks are color-coded and light up to indicate points of interest. There are two points of interest for audio: Yorinobu’s phone conversation, and the screen behind him. Despite scanning both, my audio strip stayed lit up as if there was something more to discover. Other than spending an inordinate amount of time searching the room because I was convinced I missed something, it didn’t impact gameplay.
The game also freezes and closes every so often, but between the frequent autosaves and my own frequent manual saves, I haven’t lost any progress because of it. I guess if you wanted to get super granular with it, you could say that it takes you out of the world (in the literal sense of getting booted and the metaphorical “interrupting the flow of your experience” sense), but since I can hop right back in where I left off, I personally find it to be a very minor inconvenience.
Now, quickly take a peek at the other side of the coin: the game wasn’t cheap, the release was pushed back three times already, and it’s still having problems. Problems that do sometimes interfere with gameplay. Why weren’t these bugs caught? I’m so glad you asked. Here are a few reasons you can apply to this game, other games, and honestly to a wide variety of software scenarios:
Physical Constraints on Testing
You’ve heard it before, but I’m going to say it again: two hundred people testing functionality of software are just not going to be capable of running as many scenarios as two million people playing with the software, point blank period. There aren’t enough hands on deck with hours in the day. Next time you’re fighting some goons, think of all the potential variations on the situation: which weapons you’re using, to kill whom, in what order, what armor you’re wearing, what your inventory looks like. If you know anything about software engineering you know that the goal is decoupling as much as possible – whether or not you have a granola bar in your backpack shouldn’t affect the DPS on your pistol, for example – but sometimes weird things end up coupled without devs even realizing it. One studio simply isn’t capable of testing every possible edge case; no game would ever be released if that was a requirement.
Edge Cases
A close cousin to physical constraints on testing is the user using the program in an unintended way, either purposefully (“I wonder what would happen if I…”) or accidentally. Sometimes one of our clients will remember that they can perform a certain action, but not remember the steps they were instructed to take. So they’ll do what makes the most sense to them, and then when their wacky ways result in an outcome they didn’t expect, they’ll report it as a bug. Because we didn’t intend for the sequence of actions they took to ever be taken by anyone, let alone have the outcome they were expecting, it’s not really a bug: it’s user error. We didn’t test for that thing they did because we couldn’t have dreamed that someone would do it that way when they have the correct way available to them. Of course, we test for as many edge cases as we can think of, and we include warnings and error messages to gently shepherd the client down the correct path, but developers simply cannot fathom all the myriad ways a user will find to break a program. Sometimes, the best we can do is wait until it’s brought to our attention by the user and then build more fences along the path to keep the user from wandering off into the woods.
Highway Hypnosis: Testing Edition
There’s automated testing and then there’s human testing. And when you’re relying on humans to test the same functionality over and over again, you have to accept that sometimes those humans will miss things for human reasons, repetition being one of them. Twice each week my team performs smoke testing on the platform, testing out the major functionality by going through the motions the way our clients would. One week we were tasked with taking some new hires through our testing as a way of exposing them to many different areas of the platform, and their questions opened my eyes to nuances I had stopped being aware of after going through the same motion week after week for over a year. Now, this may not be a reason for missing something major, but overlooking a minor detail because you’re just so used to staring at the same thing can happen when you’ve been testing extensively for a long period of time.
The Odd Coupl(ing)
As I mentioned before, decoupled software is the goal, but every now and then you’ll come across a “That shouldn’t affect that, but…” scenario. When this happens, it’s easy to break some piece of functionality by fixing another. Ideally, automated testing coverage (that is, tests you write to check for expected output given particular input) should account for that – but the term “edge case” exists for a reason. Things fall through the cracks. I’ve said “There’s no way this could possibly be related to that” and then had to eat my words enough times that I will never again be surprised when I’m told that one seemingly-unrelated line of code broke another.
Something Borrowed
You may remember plagiarism being a big deal in high school. In programming, not so much. There’s open source, there’s frameworks, there’s that snippet you saved from Codepen three years ago that you’re still dying to use. Programmers use other programmers’ code because it’s faster, it’s cheaper, and it’s just downright more efficient than reinventing the wheel. So, where’s the bug? Well, let’s say you borrow your friend’s car to pick up a large piece of furniture. For the most part, their car is a car, so you already know some things about it: the gas is here, the brake is there, if you turn the wheel like so the car will go in that direction. Maybe you take a quick peek at the manual to figure out how to adjust the seat and mirrors. You’re using the car for a particular job, so your friend tells you there’s a button that opens and closes the hatchback, and you’re not supposed to just slam it closed with your hands. For the most part, you have all the information you need to accomplish your goal with this car. You pick up your furniture and you decide to put gas in the car to thank your friend. Every car you’ve ever driven takes regular 87 gas, so you don’t even think to check what kind of gas this car takes. Your friend didn’t tell you it actually takes diesel because they didn’t know you’d be getting gas. Your furniture is fine, the hatchback is fine, but now you have an engine problem. Using code with which you don’t have comprehensive, intimate familiarity can trip you up about the same way.
Priorities
Before I was a software engineer I was a project manager, and a very common question on job applications for project managers is, which is most important: being on time, on budget, or in scope? I always answer the same way: on time. If you run out of money, you can have a bake sale. If you’re experiencing scope creep, you can sit down with the client to more deeply interrogate their needs and find a workable solution. But even if you move deadlines around and work overtime, you will only ever have 24 hours in a day. When it comes to bugs, you have to prioritize. In fact, we have a weekly meeting to discuss ticket priority with other departments. When you encounter a bug in a game you may be encountering a bug the devs didn’t know about, but you may also be encountering a bug they’re aware of but didn’t fix because they had to prioritize fixing a bug that had a more serious impact, or that would have affected a greater number of players. Maybe the devs didn’t even determine the priorities – that could’ve been some higher-up who actually didn’t understand the problem that well but has the ultimate say because it’s their signature on the paychecks. Hopefully that’s not how any business is running, but hey – we can’t know, can we?
Okay, that’s me and my soapbox. If you got this far, thank you for reading, and hopefully this perspective was helpful. If you take only one thing from this post, please let it be to express your dissatisfaction with grace and compassion in whatever circumstance you feel the need to express it, be it buggy software or otherwise. I think we can all agree that no one, least of all the devs, are out to ruin your experience of Cyberpunk 2077 (or any game) on purpose, if for no other reason than that would be a horrible financial decision. So be kind and happy gaming, everyone!
</ XOXO>
[Photo credit: Kamil S. via Unsplash]