Hello and welcome to a round-table discussion on the future of AR.

So far on the Teach-Out you have learned what AR,

MR and VR is.

And we have seen various examples throughout.

Now, we have brought together a team of domain experts working at

the University of Michigan and they work in very promising domains for

AR and MR. We want to have a discussion with them and why

not start with saying who you are and what you do here at Michigan.

Okay great. So my name's Michelle Aebersold and I work at the School of Nursing.

I'm a Clinical Social Professor

there and I'm also the Director of Educational Innovation.

So part of my role is to do research using

these various methods of virtual reality and augmented reality, mixed reality.

And then I also use them in my teaching.

Part of my role as the director of simulation is I work with our nursing students in

using virtual methods as

well as our mannequin based simulation in our clinical learning lab.

My name is Ted Hall.

I actually went to school to be an architect,

entered the college of architecture as an undergrad in 1977,

started writing computer graphics software in 1978 and graduated

with a Masters of Architecture in '81 as

a defacto computer programmer and that's what I've been doing ever since.

So I'm now a virtual reality visualization specialist

in the Duderstadt center here at Michigan.

I'm Mark Lindquist, I'm an Assistant Professor

in the School for Environment and Sustainability.

I teach in the Landscape Architecture program.

My research involves virtual reality and

visualization primarily for public engagement around design decision making.

Okay so we have structured this discussion around

three big questions and let's get started with the first question.

What is the impact of AR,

MR on our daily lives?

So we have experts from various domains sitting here.

In your personal experience,

what is the impact of these technologies? Michelle.

So I think that these things are really going to impact in

the world of healthcare both from a consumer as well as a provider.

So let me talk a little bit about the consumer first.

So as we are consumers of healthcare so all of us are certainly consumers of healthcare,

so we can go to see our physicians,

or our nurse practitioners,

and sometimes we need a procedure.

And sometimes those procedures aren't always very comfortable.

Let's talk about children so let's say you take your child and we've all taken,

if we have children, we've taken them for vaccines, right?

They don't like to get shots.

And it's very traumatic for them.

Well, there's some really interesting research out there that's showing if you can engage

these children in a virtual game

while they might be getting an injection or a blood draw,

they get so immersed in the game they don't even realize that that's happening.

What a great experience for them.

So it's some really promising technology that we can use as a distraction.

They're using it for cancer pain.

They're even using it for burn victims.

And if anybody has ever known anybody who's had to go through all of

the really traumatic scrubbing and things that

goes on tremendous amount of pain for burn victims,

they have this game now out there and they've done some really good research.

It's called Snow Village or Snowglobe.

And so it shows them immersed in

this virtual reality environment with all this snow and it's cooling.

And so it's just really quite amazing but what they're

finding in the research is that activates certain areas of the brain.

So it's not just about distraction but it's also about

activating those regions of the brain and it literally blocks the pain.

And they've shown this on MRI.

So that's one area.

As a consumer that we're seeing for immersive virtual reality.

Another area is the provider side.

So physicians, cardiac surgeons are now, if they have to go in,

and do a really delicate heart surgery so really complicated,

they can now take scans of the heart.

And we thought 3D printing was cool,

and we could print out the heart.

Well, now we can actually write in the OR,

they can put on their head mounted display and right before they go to do their surgery,

they can spin that heart up and really look at it in 3D and see where they're going.

So before they actually do the surgery they can sort of narrow in,

the whole team can see that.

So they really have precision into what they're doing.

So not only can we use it from a consumer end of those of us who are getting healthcare,

but from a provider end and from a training end.

So we're using it to train teams of healthcare in virtual environments.

So I think it really has some exciting potential for us

and I think we're just seeing the very tip of what we can do.

So Mark in your area,

environmental design and landscape architecture,

how do you see it the impact of AR,

MR on our daily lives?

I think the current consumer focus is largely driven by

the gaming industry and that really does provide

an opportunity to visualize change in a variety of spaces.

For example, Ikea Place app allows you to use

your phone to visualize a piece of furniture in your apartment.

But I think on a more broad scale,

we're seeing it more used in the built environment for Architectural Visualization,

putting people in the actual spaces that are going to be

designed where in the landscapes that are going to be designed.

So it really lets people understand

the future potential of those spaces in a more realistic and accurate way.

So and now Ted working in the Duderstadt center and

in the UM 3D lab you too touch various different domains.

How do you see it with the impact on our daily lives?

Well, in general terms,

it affects the way we access, view,

understand, and share information and that increasingly we're all information workers.

As more manufacturing jobs and other sort of manual labor is automated increasingly,

careers are information, medicine auto repair is information.

Cars are still computerized and so it's going to affect the way people work.

We have to deal with this information.

We have to understand how it's generated and how we access it, and how to understand it.

It also affects the way we play,

Pokémon Go is just the tip of the iceberg.

Go to Gallup park now and you'll see these crowds walking around with their phones you

know hunting for Pokémon and that was just a proof of concept.

So we haven't seen where that's going yet.

There's a lot of specific examples I could give but I think I'll

save them for later in the conversation.

Okay, good.

So, this is very nice.

And it's good to see that there is

so much potential for AR and VR in these different domains.

Now, I want to be a little bit more specific.

Digging deeper into your domains in which you are experts.

So, in your specific domain Michelle,

medicine and nursing, what is an important problem in your field?

And how do you think AR,

MR or even Virtual Reality,

how can these technologies and tools help to address this problem?

So, I really think that there are two specific areas that I'm familiar with,

or that I work in that we could use these technologies to help with.

And one of the first areas that I think about is patient safety,

something that we all need to be concerned about.

And with patient safety,

we know that one of the biggest problems with patient safety is communication.

So, the joint commission who accredits all of our hospitals and healthcare centers,

looks at major safety events that occur in hospitals.

And when they do this root cause analysis,

one of the things that they find,

is communication is at the root cause of 70 to 80 percent of these,

you know, major errors that occur that can oftentimes lead to patient death.

And people don't realize how often that that can happen.

So, one of the things that we know is that training teams of physicians,

nurses and other healthcare providers can improve communication and teamwork,

and could reduce these errors.

But it's very costly to train teams.

You have to bring them together,

you have to put them in simulation centers.

We know simulation is an effective way to train teams.

And right now, the best way that we have to do

simulation is in some centers using these mannequins.

But, what if we could put them in a virtual environment?

And that's the project that we're working on now,

is looking at creating an immersive environment that we could do multiplayer.

I mean we know it works in gaming, right?

We've got people playing from around the world in these immersive gaming environments.

Well, what if we could turn those into

an immersive training environment using virtual methods?

So, I think that that is one of the things that immersive VR can help us do.

We can create these realistic environments where we can train teams of physicians,

nurses, we can teach them how to communicate better.

Because right now, the way we educate them,

they don't learn how to communicate well.

And that's what we're seeing at our healthcare environment.

So, I think that's one area that immersive VR can help us.

The second area is in the world of AR.

And to me, and if you look at Harvard Business Review just

put out this great article about "Why your business needs an AR strategy".

And so, I think when AR first came,

people had these like ideas about doing all this really cool sophisticated stuff with AR.

And it's like, yeah that's great.

But I think AR is going to have more practical applications.

We're going to use AR to communicate with somebody on how to fix an electrical plug.

Or, you know, you see it now in factories where you can pull down a little screen and it

kind of walks you through how to fix or how do I use a 360 camera or something.

And so what we need to use AR for,

where I think it has some really practical applications is in this whole area telehealth.

Telehealth is huge.

We have people in these rural areas,

in rural ERs, in rural clinics that don't have access to good healthcare,

and we know that.

But how can we link them up to the specialist who sits

down here at the University of Michigan in the pediatric ER?

Well, we can do that through AR.

You put them in an AR device,

let's just say a HoloLens or somebody else's AR device.

And now, our specialist in the ER can see what they see,

whether it's a nurse practitioner or a physician or whomever is the specialist.

They can see what that doc or

that nurse practitioner in this clinic or this little ER sees.

And now we can clearly examine that patient.

And we can help them make a good diagnosis.

So we can bring world class healthcare to this rural area in the Yupi of Michigan.

So, that's where I think AR can help us.

So, I think that there's just exciting potential.

And again, I think we've only as you say, ''Tip of the iceberg.''

You know this is definitely a really powerful example.

So Mark, again, coming from landscape architecture.

What do you think is an important problem in

your domain similar to how Michelle just described it for medicine,

where AR, VR makes reality?

Any of these technologies can impact positively and help?

Well, I think there's been a few decades of research showing how important it

is to have public engagement in any design or planning decision making process.

And currently, despite all that evidence,

there's not proven effective ways to really do that.

Visualizations are one way,

and they have been studied,

and they've been shown to be effective for a lot of different contexts.

But I think that AR and VR in particular have

the opportunity to really put people in that environment.

And that's kind of a game changer because I think,

the accuracy and the realism once again,

the difference between looking at a picture,

looking at something on screen versus being in that environment,

being able to walk around in it,

and interact with it, is a completely different experience.

So, we're able to give people

that experience of a future landscape that's yet to be designed,

and they can more accurately respond to that.

I think it has impacts in terms of how things are going to be built,

the decision making process,

and also giving people

a little bit more power themselves to generate some of that content.

So right now with tools like SketchUp,

you can take a relatively simple model using plug ins,

get it into a VR headset,

and walk around in it.

Ten years ago, that was completely unheard of.

So I think some of the empowerment of individuals is going to

be something moving forward that really helps in these decision making context.

Great. So Ted, again as

the person working across disciplines at the University of Michigan,

you've probably seen lots of compelling examples,

but also had problems.

So if you could pick some of them for us and,

share with us how you think AR and VR could help.

Well, you're right. I get to see a great cross-section of

applications from across the university which we've had a lot of interest from them.

The health professions touching on,

reiterating some things that Michelle and Mark said.

The ability to take an AR device,

and look at the real world but see

some other information projected on that that we wouldn't normally see.

Whether it's looking at a patient and seeing what's going on inside their body,

or looking through a wall and seeing the utilities

behind that wall before we start ripping things down.

An auto mechanic again looking at a car,

some mechanic looking at some complex piece of machinery,

and seeing it in an AR display, you know,

some sort of diagnosis of what that is,

what's going wrong with it, how to repair it,

what bolt do I have to loosen first before the procedure for repairing that thing.

A sort of a crossover between

design and medicine is a real project we've done with the hospital facilities.

They are moving into renovated space.

The pathology department, particular is moving to renovated space.

And we have played middleman between the pathology department,

the university facilities people, and the architect.

So the architect sends us their CAD model of this renovated space,

we project it at full scale in Virtual Reality,

and they bring pathology staff into

inhabit that renovated space before anything has been built.

And they fill the white boards with design feedback.

They can inhabit their labs,

their offices at full scale.

They can practice walking from A to B and see how long that takes.

And they can look at adjacencies and spaces.

And then they're using that as an actual design tool.

They're so happy with it that they've done it three or

four times now and we're already talking about the next one coming up this spring,

this exercise that we've done.

In screen arts and cultures,

it's being used to reconstruct 3D movie sets,

for movies that some of them have never actually been filmed.

Some have been filmed but lost,

all they have is stills.

Some for example, Citizen Kane,

we are planning with our Matthew Solomon and Screen Arts and

Cultures to reconstruct the rage scene from Citizen Kane,

in 3D with motion capture,

shot in virtual reality and then let students, film students,

play with different camera angles,

and see how they might have done things differently than Orson Welles.

We have done a lot with museums,

scanning physical artifacts using photogrammetry and then able to

display those in VR or MR augmented reality.

Go into a museum and,

you know, maybe there's just an empty pedestal there.

But you can sort of pick what display you want to see

because 80 percent of the museum collection is in storage.

So you can sort of pick your own tour through the museum.

Biochemistry, we're using it to look at proteins,

crystal structures that are very complicated 3D geometries,

that are very difficult to understand on a 2D piece of paper or 2D computer screen.

But in mixed reality,

augmented reality, virtual reality,

you can surround yourself with this complicated structure.

And I've been in the cave with

a guy who says he's been studying this one protein for 10 years.

One protein, trying to figure out how this thing folds on itself.

And it's fascinating to see this guy in the cave in this 10-foot cubic space,

and he's ducking all around and looking at different corners and he said,

''We know this thing gets over here somehow and we don't know how that happens.''

And so just being able to immerse yourself in that kind of

complex 3D geometry and think in

a way that you have not been able to think about it before.

Panel - Impact on Our Daily Lives (Part 1)

Augmented Reality, Virtual Reality, and Mixed Reality: Opportunities and Issues Teach-Out

Meet the Instructors