Hello there, my name is Will Bonner. And today, I'd like to pick up where my previous talk left off in describing the development and operation of games as a service. When I refer to games as a service, I'm describing a sub class of games that you as a player do not directly own. Instead, a developer owns the game and they provide access to that game via some service, that allows them to connect to and download their game content. Over the course of this talk, I am going to define exactly what a game service is and why building one can be desirable for the right game. Then, I'll share some advice in how to develop and operate a game as a service effectively. And I'll also sprinkle in a few tips for game development with the Unity engine as appropriate, since I'm still using the engine quite a bit myself. So let's dive in. Firstly, I'd like to discuss the motivations behind developing a game service in the first place. I mean why are we even doing this? The games industry is grown from the work of hobbyists to a multi-billion dollar industry primarily on the model of games as a product. Whether that's with games played on arcade cabinets or in home consoles and PCs. This model clearly isn't broken. So why are we so hell bent on fixing it? Those motivations become clear if you consider some of the most successful game service titles out there. League of Legends, Overwatch, Destiny, Heroes of the Storm, Clash of Clans, World of Warcraft and HearthStone are all entirely service based games. Under normal circumstances, you can't play these games in any way without an Internet connection that is logged in to their respective services. Similarly, Defense of the Ancients 2, Counterstrike Go, Diablo, Farmville 2, Starcraft II, Call of Duty, Battlefield, Team Fortress 2, and Halo are all largely service based games. While there is some part of them that can be played standalone, a great deal of their content is intended for online use on servers that are owned and operated by the developers of these games. So, that's a ton of games. That's many of the most popular games in the world, in fact. And they're targeting players of all kinds on platforms and demographics. So, what are some of the similarities between them? Well, firstly, they're all big. They have millions of players. Multiplayer is a very important feature for each of them as well. Many of them are free to play and by extension many of them support microtransactions as a primary means of generating revenue. Lastly, a lot of these games tend to have long life spans over months and years as players develop mastery and continue to learn about these games. This is in stark contrast to a lot of games that are intended for a single or limited play through. Now I don't think that's a coincidence. Let's try and connect the dots on how those features can be supported by a service based game. Firstly, if multiplayer is important to your game, your success hinges on ensuring your players can reliably connect to and play with each other. That is probably the first reason that you would choose to develop your game as a service. Given current industry practices in technology, a game service is probably the best way to give your players a stable multiplayer experience. Consistent multiplayer is a difficult issue to solve when you consider that your players can connect to your game from anywhere in the world at any time. But fortunately, there's flexible and affordable Cloud based solutions available nowadays such as Amazon's AWS. That means you don't have to buy a zillion servers around the world and house them all yourselves. In addition, you can force update some maintenance to your servers, and therefore your players, at your leisure. You can also keep conditions, such as latency, consistent from game to game and patch to patch, since your servers are always located in the same geographic region. You can also guarantee that your servers are sufficiently powerful and configured to handle expected player traffic and monitor the status of those servers. And perhaps most importantly, you can guarantee the security of servers that you own and operate in a way that you can't with servers that are operating on anybody else's hardware. That's probably the second main reason that you would develop from developing a game as a service, if post-installation security is critical to the success of your business model. This feature affects every example title that I listed in a very powerful way. Farmville, League of Legends, Clash of Clans and Hearthstone all cannot make any money if hackers make it easy to unlock all of their content for free. That's true of every game on Steam as well. World of Warcraft and Destiny could be ruined if item duplication was easily possible. Starcraft 2, Counterstrike and Halo are all horrible multiplayer experiences if your opponent has even a minor unfair advantage. Games are intrinsically one of the hardest types of software to secure as well, because it's like trying to stop a burglar that's already inside your house. It's because your hackers are indistinguishable from legitimate players until the moment they do something harmful. Plus literally every byte on a personal computer can be inspected and modified if you've got the patience, interest, and tools to do so. In fact, for games made with Unity it's particularly easy. I'll get into why later. So, the last reason that you might consider building a game service, is if your players would benefit from being able to interact with portions of your game while not logged in. There's all kinds of features that fit this bill, but a few of the more popular ones include a chat service that would allow your players to talk to each other when they're offline. An auction house, a guild chat or bank, a competitive ladder or tournament system. For any system like that to work, you'll need a mechanism for your players to connect to you that's perpetually operating. If you don't, all of the chats or items involving offline players could disappear or become inaccessible if everybody in the game logged off at the same time. Or at least everybody involved in that transaction. So, in summary, game services might be for you if you want to build a big, long lasting, multiplayer game. Or if you can't afford to let people easily hack your game. And lastly, if you want people to engage with your game even if they aren't currently logged in. So, for the last half of this talk, I'm going to focus on how you'd actually accomplish a game as a service. Now, as I've previously alluded to, this is an incredibly broad and complex topic. I can't give you a step by step formula for success or silver bullet. But I can cover some of the big must do, should do, and should avoid items, and that's how I'm going to structure the rest of the talk. Keep in mind that while these are specifically focused on game service styled games, they're frequently great general advice for developing any sort of game software. So first onto the must do items. The first must do, is using version control systems. If there's one game development practice that I can't stress the value of enough, it's VCS, using something like Unity's Asset server, gitLFS, SVN, Perforce or Mercurial. Even if you're working by yourself on any type of game, it is literally more valuable then your game engine. Personally I would recommend using Git or Per Force as industry standards since both are free of it and easy for you to use. In addition to VCS however, game services benefit tremendously from a continuously integrated pipeline. A CI program like Jenkins can be installed on a computer and configured to watch your VCS repository for updates. Whenever somebody pushes new content, Jenkins can be set up to try and make a build of your game client and server. Why is that valuable? Well, it turns out that if the newest version of your game can't compile for whatever reason, Jenkins can send you an email or a text to let you know about it immediately. And finding about that problem 30 seconds after it happened instead of three days later, can save you hours of effort in figuring out what went wrong. Jenkins can also be configured to run automated tests on completed builds. These can range from simple unit tests that exercise a bit of your code in isolation to full-blown acceptance tests that simulate starting up a server, connecting users to it, pushing buttons, and playing your game. Which can be immensely convenient when you're targeting multiple platforms. For games that run in services for many years, tools like Jenkins can amount to millions of dollars saved in developer efficiency and prevention of bugs. But even if you're an Indie developer, these tools are useful. Jenkins doesn't care what type of game you're building, and once you've got it set up, you can use it for every game project you ever make. And, of course, Jenkins is free in MIT license to, so go out there and try it out. The second must do item is pulling out security sensitive code from what you ship to your players. Hacking unsecured games isn't terribly difficult, but with Unity it's even easier. To understand why, you need to have some idea about how this engine works under the hood. Unity allows you to make games in it by defining scripts called mono behaviors. Those scripts that make up the logic of your game are executed in what's called the mono virtual machine. Now from a security standpoint, all you need to know is that those mono behaviors are not directly compiled into ones and zeros. Instead, that code is bluffed in an intermediate form called the common intermediate language, that the Mono VM can process when somebody who plays the game using a just in time compiler. Now, it a lot of ways, the JIT compiler is a great feature. It makes your game logic more easily portable, and by operating in monos VM, there's some cool optimizations and features, like Garbage Collection, that you get for free. But it also allows the bad guys to reconstruct your source code almost verbatim with free tools like IL Spy, pictured here on the right. Since intermediate language that is spit out by Unity's compiler has a lot of information about how your program runs so clearly visible, you can reconstruct almost the entire thing and modify it as you see fit. By contrast, statically compiled code like C++ at least mangles the original source enough that reconstructing it is much, much more difficult without knowing the specific details about how it was compiled in the first place. Now, of course, regardless of the language you pick, once it's installed on somebody's computer they can hack it. No amount of obfuscation or security checks can prevent someone from monitoring and modifying anything on their machine as they see fit. The only way you can really stop this kind of cheating is by hosting your own security sensitive content on servers that you own and monitor. And restrict the ways that end-users can interact with them, via firewalls and well-defined APIs, that double-check everything the player is trying to do before letting them do it. Otherwise, you can end up with a game like Jet Pack Joyride, where all of your content can be instantly unlocked with a single click and download. Those are the must do items that I would emphasize the most. So now I'll move on to things that are just very good suggestions. The first one I'll recommend is having some kind of load to performance test on your services before you go live. Understanding why your stuff will break and when it will can help you understand what your operating costs are and mitigate powerful, painful crashes after you've launched. You're also going to want to have some type of tools and reporting in place so you can see what the current running status of your game is. A few free options exist, but writing a simple script just to monitor the memory and CPU usage on your servers isn't terribly hard for you to do for simple stuff. In addition, it would also be great to have an idea ahead of time how often you'd like to push updates to your game and message that to your players. Nobody likes to get booted in the middle of a raid for unexpected server maintenance. So if you can build those expectations in from the start, your players will be happier. And lastly, if you have the resources, consider building your services with zero downtime deploys in mind. The idea is essentially that [COUGH] servers with new versions of your game are spun up before the old versions of your game are decommissioned. And then the next time your players connect they are redirected to the new version. As a result, people playing on old versions don't have to get booted off before your updates are delivered. This isn't always simple or easy depending on the constraints of your game, but it can be a huge benefit for your players if you do it right. And lastly, I'll cap this off with some items that you should definitely avoid in game service development. Firstly, don't ignore your players. Even outside of game service development, player feedback and sentiment is incredibly important for your success. It's especially important for game services though, as downtime translates directly into angry people and less money. Plus, balance and mechanics changes are far more likely to stir controversy with players that experience the content together and take it seriously. And secondly, don't forget about your garbage collector. This is a utility that's built into Unity courtesy of the model run time that it operates on. But it can be used in other languages too. The garbage collector's job is to monitor your running game and figure out when it starts to run low on memory. And then it will look for things that aren't in use anymore and delete them, and that's a cool and useful feature. However, Unity's garbage collector requires that the entire main thread halt while the garbage collector is running. That means while this memory recycling is going on, you have no UI, you have no game logic. Nothing is going on. And that's a particularly big problem for games that might be running for a longer period of time, and individual performance hiccups can be a big deal. You're more likely to encounter these spikes the longer that your play sessions go for. So service based titles are frequently at risk. You can avoid this by intelligently constructing and pooling your objects. Avoiding unnecessary allocations and calling the GC.Collect method at intervals when you know the heap is small, and your GC Collect will complete quickly and safely. This concludes my overview in development and operation of games as a service. I hope that this inspires you to think about the development of games differently, and helps you solve difficult problems in delivering awesome experiences to your players.
