2 - Inpainting in Nature - Duration 05:01

Loading...

From the course by Duke University

Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital

525 ratings

Duke University

Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital

525 ratings

In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.

From the lesson

Image and video inpainting

Students will get involved with a very exciting topic, since image and video inpainting is one of the most used tools in the movie industry. They will learn the problem, and also how they can approach it from multiple directions. This will also help to illustrate how the same problem can be approached from multiple mathematical angles. We will connect this with Shanon’s work providing yet another angle. If you watched the lectures on PDEs you will have more mathematical background, but you will enjoy this unit and learn without it as well.

1 - Introduction to Image Inpainting - Duration 08:168:16

2 - Inpainting in Nature - Duration 05:015:01

3 - PDEs and Inpainting - Duration 21:59 - Optional break at 13:3121:59

4 - Inpainting via Calculus of Variations - Duration 15:32 - Optional break at 10:0615:32

5 - Smart Cut and Paste - Duration 07:517:51

6 - Demo - Photoshop Inpainting Healing Brush - Duration 02:312:31

7 - Video Inpainting and Conclusions - Duration 05:135:13

Meet the Instructors

Guillermo Sapiro
Professor
Electrical and Computer Engineering

Hello, and welcome back. We actually see inpainting in nature all

the time. Let me show you some examples.

What do you see here? You'll probably see a rock, and nothing

else than a rock. But there's actually something hiding in

that rock. Let me just show that to you, or at least

help you to see it. There is actually a fish hiding in that

rock. You might be able to see it. I hope you

do, but I'm going to help you.

You can see the eyes here, you can see the tailfin here. The whole fish is in

here. Pretty amazing isn't it?

That's camouflage, but that's also inpainting.

The fish has inpainted itself in order to get camouflage.

It has basically, it's kind of object removal in nature.

Pretty amazing. We also see that with octopuses.

I'm going to give you another example of how amazing this fish is.

This is not a very high quality image, but still it's going to illustrate how

amazing this fish is. The fish was now, not in nature as

before, but basically in a very nice fish tank, was basically put, an experiment

done by Ramachandran and collaborators, on top of polka dots.

So it's basically a piece of paper or basically something that is colored with

dots, as we see here. It's very interesting that if you have

started here, for example, the fish has turned around, has created two polka

dots. In its own body.

And once again I apologize for the relatively low quality of this image that

was scanned from a paper. It's just kind of a long process to get

it into the computer. And it's also turning around to put

basically its eyes on top of one of the polkadots.

So basically the fish is trying to ("inpaint) himself,

trying to camouflage itself so it cannot be seen by predators.

So we see inpainting in nature, fish, octopuses and others.

We actually are doing inpainting all the time.

When you are watching this video, you're actually doing in painting.

Why is that? When we have, when we remember that we

learn about the human eye, in the retina, in the back of the eye there's a region

with no sensors, no sensors at all. No cones, no rods.

That is kind of the region where the sensors basically send the cables back to

the brain. So there's actually no sensors.

So what should happen if you have no sensors?

We should actually see a hole in the visual world, where basically you would

be watching this video and there would kind of a region of missing information

because there are no sensors. But the humans are all the time filling

in that region by different techniques that the visual system does and actually

you can do those exercises. I took these images from the website that

is provided here. And the basic idea is that if you manage

to fixate. Basically if you manage.

You stand like a meter away from the screen and you manage to fixate the

basically a point. Like this point here.

So you concentrate on fixating your visual system into that point.

After a while you will see that the sun disappears.

So you're basically forcing your system to not in- paint the region that

basically there is no receptors. If you don't do that, if you don't

fixate, you basically see the sun with no

problem. The exercise is to try to stop the visual

system from doing in painting. Here is another exercise you could fixate

here. You might be able to basically pause

right now, or after we finish this video,

you might be able to go back, pause here, and then play with your distance to the

screen. Fixate here, concentrate on fixating

that, and you might be able to see, for example the ear of Van Gogh disappears,

or basically, the sun disappears. So you're forcing the visual system not

to inpaint. Otherwise, we're in painting all the

times. We don't see those problems that, we

don't see holes when we are looking at a scene.

So these are just examples that we actually have image and video inpainting

when we're watching at the regular scene that's basically our video coming into

our eyes. Examples of that in nature.

So now we're ready to see how the computer does in painting.

Thank you. I will see you in the next video for

that.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play