[MUSIC] Hi everyone, I'm Jeremy Gibson Bond, and welcome back to the Unity Certified Programmer Exam Preparation Course. In this video we're going to have a conversation about augmented reality. It's going to be similar to the virtual reality video in that I'm just going to be talking about kind of best practices that work across all of the spectrum of augmented reality. And the reason that we're not talking about tech here is because this tech is moving so very, very, very quickly. So, let's jump on into this. So augmented reality is another aspect of XR or extended reality that Unity enables. And this fits in with things like, virtual reality of course, and things like cave environments, and even things like projection mapping. One of the things that I wanted to talk about in this video is the difference between terms like, virtual reality, augmented reality, and mixed reality. All of these fit under the Unity XR umbrella, but they mean different things, and those meanings have actually changed over time. So VR is the one that is the most clear and the most static in its meaning. Virtual reality is a completely virtual world that you inhabit through a head mounted device or something else similar to that. Augmented reality is anytime we put virtual things into a real world environment. Now the most common version of this is cellphone or mobile AR, right? So this is, you have a cellphone and you point it at a surface, and you see a virtual thing projected onto that surface. That's the most common version, but there are also things like the Microsoft HoloLens or the Magic Leap. Which provide stereo versions of this where you look through stereo goggles and they put a 3D virtual object into your world. Microsoft in particular has started using the term mixed reality a lot, and they use it to describe everyone of their headsets, including both the HoloLens, and their fully enclosed HMDs. And when they say mixed reality, they're talking about everything up to and beyond normal augmented reality. So in normal augmented reality, you see some of the real world. And then there was this thing for a while called augmented virtuality, that nobody likes using that term. Where you would see your real hands in the virtual world. This is something you saw in the Intel merged reality demo that they showed a lot of places in 2017. Microsoft's term for mixed reality, also involves bringing virtual representations of real world objects, into a virtual space. And that's why their fully enclosed HMDs are still under the mixed reality umbrella. So that means, for instance, I would put on a Microsoft HMD and I could see not my actual desk but a virtual desk that was in the same location as my actual desk. And a virtual computer that was in the same location as my actual computer. That's their concept of mixed reality. And you've seen this also in things like the Leap Motion, where Leap Motion for a while would put their device on the front of an HMD. And allow you to see your hands, virtual representations of your hands, projected into that virtual space. So that's what augmented reality and mixed reality, and virtual reality, and extended reality with the XR all kind of meet. There's an article by Patrick Catanzariti called What is Mixed Reality? It's part of devdiner.com. That you can look at to get a little bit more detail on all this information. So there's three more topics we're going to talk about in our discussion of AR today. What is optimization, which is just critically important for any kind of AR, particularly mobile AR. The next is designing AR apps, things you want to keep in mind while doing that. And then the third is really some best practices to play to the strength of AR and minimize the weaknesses. First off, let's talk about optimization. Now, all of the things we mentioned for optimization for VR are pretty much the same for AR, especially because you're looking at a mobile platform a lot of the time. One of the benefits of you get with AR is that usually you're only dealing with a single viewpoint, a single camera, rather than having to have the two eyes of virtual reality. However, there are devices I mentioned, like the HoloLens and the Magic Leap that do have stereo vision, and do still have to render two viewpoints. However, you get the benefit of only have one viewpoint most of the time. But, you also have the performance hit of tracking. And that tracking takes a lot of time and a lot of processing. There are cheap ways to do AR tracking, where you just really pay attention to the orientation of the device, and don't pay attention much to the actual environment. What most AR now is going to a place where it's tracking optically the environment around it and trying to pay attention to that. So trying to pay attention to planes and surfaces and point clouds in that kind of thing. And that takes a lot of processing power. So let's talk about some good features of how to design AR well. The first thing is, you really want to focus on stickiness rather than novelty. A novel AR app that's just like hey, look, it's an AR thing. That's actually really cool initially but people fall off very quickly. And in particular, when you're dealing with mobile apps, stickiness is very, very helpful if you want to make any money off of it. So, a good example of this is Pokemon GO. Pokemon GO had AR. The AR was one of the big selling points for it. But honestly, the Pokemon experience was the real selling point and is the real stickiness factor that kept people playing. In fact, many, many people eventually turned off the AR aspect of the game because it saved battery power and allowed them play for longer. Another thing to think about, a really critical question to think about is, how does AR make your app better? If you're going to add it to an app, what is it doing in the AR that you couldn't do otherwise? If it's just taking more battery power to run an AR scene, but it's basically the same experience. That's not really adding much. You want to have experiences where AR makes your app much better. When adding AR to an app, you need to think about how it's, both, an enticement and a barrier to entry. So, AR's a cool new thing and people are going to want to check it out. But also, it's something that people lack experience with. And so, new users might find it fiddly or difficult to understand. That being the case, you need to make sure that the AR aspect of it is pretty simple in terms of what you're asking them to understand at any given point. So if you do have a complex experience that's going to happen in AR, give them something simple to start with. And sort of scaffold that or ladder it so that you're adding concepts on top of each other over time, rather than dumping everything on somebody all at the same time. Speaking of timing, it takes time for the AR to establish tracking of the space. It takes time for it to understand where the planes and point clouds, and things like that are. So, you need to give the player an understanding of that. A couple of ways to do this are, one, if it's still establishing tracking. And it helps to move the device around as you're establishing tracking. Tell the player this. Tell them to kind of move the device around. Give them a little animation, kind of showing them how you want them to move the device. And that will give them something to do, and some feedback while it's establishing tracking. Another thing to think about is, can you give them the opportunity to do something before this tracking is really established well? Can you allow them to kind of place something and then refine that placement later once the tracking is established. In terms of helping players understand AR, there's a few more things to think about. The first is just this idea of privacy, you're asking players for access to their camera on their device. And so it's worthwhile to reassure them that you're not going to use that for nefarious purposes. Just make them feel a little more comfortable, and a little more likely to use your app. Another thing I strongly encourage you to do is to use animations to describe interaction that you want players to use. So if, for example, I'm going to encourage someone to rotate this cup in AR. I could put up text to describe that, but then I have to translate that text into multiple languages, and it might be confusing. On the other hand, if I show an animation of an object and a finger swiping across the object, and that object's rotating. Then that will be much clearer and I don't have to translate it later. Similarly, if placing an object don't expect the player to just sort of tap on screen where they want the object to go. Allow them to tap and hold, and place a ghost of the object that they can drag around. Because often, that first tap is not going to be terribly accurate. But refinement with a ghost is actually a really easy thing for players to do. Finally, let's talk about what to do when things go wrong, right? When you lose tracking, for instance. So definitely tell your player that you lost tracking, don't just leave them hanging, right? But it helps to tell them why, rather than just say that tracking was lost. So, if it's because there's too little light, or there's too much movement in the scene, or something like that, express that to the player. However, in these descriptions, try to not use standard AR jargon, like plane and point cloud, and feature. Try to describe it in ways that are more natural English that people would understand a little bit better. If you do lose tracking, one thing you want to make sure to do is hide the virtual objects until the tracking comes back. Because as the tracking is reestablishing itself, your objects might jitter or blink in and out of existence. And you really don't want that. Anytime you're asking for the player to interact, you want to have a timer on the situation. Because the longer they don't interact, the more likely it is that they need you to tell them what to do. And so, having a timer, and if a certain amount of time has expired, giving them a prompt, or giving them a video showing them the kind of thing that you want them to do could be really helpful. Finally, go ahead and always give the player a way to reset the experience. If somehow maybe they've established a bad tracking location or they want to reset it in someway, just give them the ability to do that. And don't lock them into the first established tracking that they receive. Now let's talk about the strength and weaknesses of AR. AR's greatest strength is this idea of freedom of movement within 3D space. So the idea that I can hold up my phone and I can walk around the thing. Or I can look around in a virtual environment is really very powerful. It's this magic window into another world. The ability to zoom in on something and see the details, or go through the skin of something to see the innards. Those are really cool things that can only really be done with AR. And so that's the greatest strength. Because this is the greatest strength of AR, it means that physically moving the device is by far the best control scheme that you have. Buttons on screen, particularly if there's more than just one or two, can get really distracting and pull the player out of the experience. You want to have as little as possible interaction by touching buttons on the screen, and much more sort of movement through a space. However, that being said, all this movement and holding your arms out for an extended period of time can be fatiguing for players. Things like holding a device up and looking up, that can really cause a lot of fatigue. So be aware of that, and don't design your application to require your users to hold their arms out or up for an extended period of time. This is another place that Pokemon Go worked really well. Players could just kind of hold it close to their body and usually point it down because most of Pokemon where near the ground. And then have a single swipe with their thumb or with a finger to throw the Pokeball. Now worked really, really well in AR and was an easy gesture that was non-fatiguing. So lets talk about some weaknesses. AR doesn't work perfectly everywhere. And in fact it doesn't even work well everywhere. There are a lots of environments that will not work well for AR. These include environments that don't have a large flat space. Environments where things are moving a lot. Environments where you're actually moving like being in a moving vehicle. It's a terrible environment for AR. And because of that, you need to be aware that player is going to try and use your app in different locations. And you need to test yourself in all those different locations to see which ones are going to be a big problem for you, and which ones might be a problem that you can deal with in a software. Another somewhat technical weakness or issue to be aware of is that the edges of surfaces tend to drift some. So for example, you wouldn't want to have like this cool mountain here with some water features. And then a waterfall that came and dropped off the edge of the desk because that edge is going to drift some. So in general, try and just kind avoid the edges of planes. And just focus on the large flat surface in the middle. So that's it for our discussion of AR. There are lots of augmented reality tutorials and demos on the Unity website. And I encourage you to check them out, and look to Unity for more information about this. Thank you very much for watching this video, and I'll see you in the next one. [MUSIC]
