[MUSIC] Now you've learned a lot about virtual characters, the psychology behind it, and a few applications using virtual characters, you are probably quite excited to start designing your first social VR application where you can interact with the avatar for another person or a computer generated agent in VR in realtime. But because of the complexity and the subconscious nature of social interaction in real life, there are a few hurdles that could be getting in the way. In the next couple videos, we'll discuss a few challenges you might be confronted with. First, we can divide social VR interaction into two different categories. We can be interacting VR with avatars or agents. Each identity is used for a different types of applications. Let's start with the first one, the user/avatar interaction. This form of interaction's useful when two or more users want to work together in a shared virtual environment. Engineers could use VR to discuss the structural design and technical details of a or a oil trading rig. Neurosurgeons could use VR to work together on the surgical plan. Chemistry teachers could use VR to teach students how molecules work in 3D and gamers could collaborate in an online mission to plan their attacking strategy. In all these applications, it is important that not only we all share the same space where we can see these 3D objects of interest, but also we can be together with our colleagues, students or collaborators. We should be able to see each other, hear what they say, and read their body language, only in real time. The user/avatar interaction should be very similar to our everyday face to face interaction. The difference being you will be in a shared virtual space without being physically together. And each one of you is represented by your avatar. Which can look exactly like yourself or look very different. In this case, the challenge is mainly about capturing each user's face and body movements and apply them on their avatars in real time. The user should be using a VR display and holding a VR controller in each hand, all 6 degree of freedom tracked. Ideally, we should also track their torso, by having an additional tracker on their chest. If we have 6 degree of freedom tracking for someone's head, chest, and two hands, we can have a not perfect but reasonably good reconstruction of their body language. Facial tracking, on the other hand, is trickier in VR especially if the user's wearing VR headsets. Of course, we can have both external cameras track their lower face and trackers inside the headsets for the upper face. But, generally speaking, facial mocap is still more challenging than body mocap. This is mostly because we are more sensitive to any subtle discrepancies on the face between the real and tracked data. In order to produce a choose for representation of someone's facial expressions, ideally, you can capture as much detail on the face as possible. Including any tiny muscle movements on the face, eye movements, and subtle facial color changes, such as blushing. But in reality this is also limited by the network capacity to transmit all the data in real time to the user you're interacting with. As timing in real time interaction is crucial, there's no point in delivering a super realistic link with a two second delay, it just loses its meaning. So there are always social signals you won't be able to track and deliver in real time. Another problem in this type of interaction is the mismatch between the performance of different technologies each user has access to. In an ideal world, each user should have exactly the same capacity for tracking, networking, and realtime rendering. But in reality, they're likely to be interacting with different VR set hubs and networks. This could potentially introduce bias in the interaction where the users who have access to more advanced technologies would find it easier to be more expressive and dominant in the conversation. [MUSIC]
