0:00

Hello and welcome back. We have been talking so much about active

contours during this week as one of the examples of the use of partial

differential equations in image processing that I want to conclude the

week with a bit more of details now that we know all the background material that

we need. We know curve evolution, level sets,

calculus of variations, differentia; geometry.

We have all those concepts that we learned this week.

Let's put them together in one of the active contours type of techniques.

So, let's just discuss that a bit more. We normally talk first about edge

detection. Now, edge detection, basically only talks

about local churches. So it's basically giving us places where

there's a big change in the image. And, because sometimes we compute edges

with radiant, for example, computing the radiant of the image.

Then at every location we might have the strength of the edge, which is the

absolute value of these or the magnitude of these.

We might have the basic direction of the edges.

Remember that the normal to the level sets is in this direction.

So basically if we have an object. This is the level line, and then the

gradient is perpendicular to it, so we also get the direction of the edges which

are in this direction. So we get a lot of information, but all

that information is local. As we see in this picture.

The goal of active counters as well as other techniques for example, was also

the goal of graph guides, is to integrate those local measurements.

If we look at this picture for example. That we see local representation of

edges. There are some basically spread around

edges, but if we look carefully, it looks like there is an object here and probably

another object here. So there is some relationship between the

local edges and that's what active counters is going to try, try to capture.

Let's just see that. That's what we want to do.

We, I want to run that again. We basically want to integrate this local

measurements of image changes. And that's what active converse does,

exactly in the way I just showed it. It's just curves that move with the

equations that we have seen and we are going to discuss again, curves that move

towards those local edges. So, active converse basically starts from

an image. The first thing you do is you have to

compute edges. You have to do these local computations.

There's a lot of art in how to compute those edges.

But that's always the first step. So you end up with an image that has,

kind of, information about the local characteristics of the image, and often,

when we talk about active contours, we talk about this function.

So we smooth a bit the image, and that, we know, by this time, we're in the sixth

week, we know images are noisy. And very often we have to just clean the

noise a bit. We take the gradients as indicator of the

local edges, this is just to make sure that we don't divide by zero.

So we basically take one over a regularized gradient or one over the

gradients of our regularized image. And that gives us count on indicator

function of where should edges be. If there are, if there are edges, this is

very high so one over is very small. Once we have that, then the active

contours have to go and integrate that. Now, where is the active contours

equations that I showed you before and I'm going to show you again coming from.

One of the things is that, what is a curve?

What is an ice curve? An ice curve is a curve that goes around

areas where this image that is basically this one is very low.

Remember, once again, gradients very high, one other gradients very small.

So I want to find curves. That travel around regions that this

image is very small and this is represented here.

I basically am integrating over the image G and I want that integral to be very

small. That will be my curve for the active

contour. So, this is a function of g.

And I'm looking for the curve that minimizes that function.

We know what, how to do that. That's calculus of variations.

We're looking for a curve. That's a function that minimizes this.

So, we plug. G in here and we do calculus of

variations and we get an equation for the active contours.

That's basically whole trick. So, once again, we do the local edge

indicator. We are looking for curves that are what

are called curves of minimal effort or shortest path in this space.

We do calculus of variation. And we get the curve deformation that

moves the curve towards the minima of this function.

Again, sometimes, just a local minima. We move it towards the minima of that

function. And those are curves that are traveling

around low values of g. Which means, traveling around edges.

And that's basically how we get the equation for active contours.

Let's see that equation again. Before that I want to show you just one

example, think it's a nice example in ultrasound.

Those green curves are original curves and they deform, according to the

equation that we obtained from the Euler-Lagrange, towards the red curve.

The red curve is sitting in a place where if I integrate, if I sum G.

All around these is low certainly much lower than the initial one.

And that's what the calculus of variations gave me that deformation.

So let's talk about this function g. The function g, I'm going to just draw

some graphs here, is basically, this is an image.

It's a very simple binary image. These are the edges of the image.

Let's look at a profile. This is almost binary.

It looks in the screen as binary. The profile, when I travel here, it goes

up and down. So it's about this.

Now when you compute edges in this image, you are going to get basically high

values around these regions. And then if you do this, those are going

to be low values, and we know that. Remember, and we're going to see the

equation again. Remember first of all we want to travel

in regions of low G, because the, we're, we're doing the integral of G.

So we want to go around regions with low G, as we see here.

That's one thing and, remember, we want this G to be very small because we want

the curve, when it's evolving, to stop at the boundaries.

We don't want it to continue evolving. So that's one very important concept and

that's what we get from taking this G function, this potential function, to be

a function of the gradient and inverse function of the gradient.

Now, when we do the Euler, the, basically the Euler-Lagrange of the calculus of

variations of this. We got this equation.

That we see here. This is equation that I show you a number

of times to be basically the equation of the calculus of variation.

So this equation. That is minimizing.

This is that. Curve evolution.

The curve is evolving. We have seen this a number of times

already. And of course, this is implemented with

the level set, and I showed you in one of the previous videos, at the end, a slide

with this equation in the level set's framework with an implicit function fit.

So, this is the equation that is basically the gradient descent or the

equation that basically minimizes this, which is also called a Geodesic curve.

It's a curve of minimal weight and they're also sometimes called Geodesic

curves. Now we already saw in the previous slide

this function, and this is the function that gives us basically this, what we see

here. Now it comes out from the minimization of

these also, these term. What these term is doing, look, there's a

gradient. Of the potential function, not a gradient

of the image, a gradient of the potential function.

And those are represented here by these green lines.

So it's not only that G gets very close to zero, and is going to try to stop the

evolution. Remember, if we don't have G, this is

curvature motion and it shrinks it to a point.

This would try to stop the evolution but if we don't have an ideal age, this will

never be zero. Then it will slow down but it won't stop.

It will come here and it will kind of move because G is not zero yet.

It will maybe just move around or maybe even jump over, if G is not, not really,

really a very low and the curvature wins. So, this other term, that once again

comes natural from the calculus of variations, is.

Basically pushing the curve. Towards the gradient of G, this green

arrow. So it's saying, okay, but I'm going to

trap you. Because I have this term that is not

going to let you go away. So this is kind of an extra velocity.

And once again this part is because, remember, when we have velocities that

are not perpendicular to the curve, all what we care is that perpendicular are

the normal component, and this is what this operation does.

It projects it to the normal component. So we have a stopping term that says.

Come to me, come very smoothly to me because you have curvature here.

Now we know that curvature motion smoothes the curve, stop when you get

close to edges or at least slow down and this time traps you is pushing you

because is at gradient of this function so is pushing you so you get stuck there.

So that's basically what the active contours or in particular the geodesic

active contours. Sometimes they are also called the

geometric active contours because it has this curvature and a lot of geometric

components of the curve. This is what these geodesic active

contours are doing, are minimizing this energy, this geodesic energy, are trying

to find curves of minimal distance, minimal weighted distance with waves G.

And they do that by doing calculus of variations getting to this curve

evolution. And implemented in, in the level size

frame work. So everything together that we have

learned during this week is in this Geodesic Active Contours.

12:02

Now that was just an example basically in 1D, what happens in 2D is basically the

same. It was an illustration in 1D, and what

happens here is we take the image, we smooth, you compute edges.

Let me just show you that again. You compute edges,

local edges based on this function and then you basically evolve a curve, you

can start from a curve for example that surrounds the object, you just do some

initial condition. You can start with multiple curves,

remember we implement this with levis sense, we don't care about topology.

So you can start with multiple curves or you can start with a curve around curves

inside. Just let them evolve and when they

evolve, boom. They get into the integration and if you sit at and I'm

going to just show you that once again. The initial curve has much more energy.

So if I go and integrate. G.

For the initial curve. It's much larger than if I integrate G

for the curve that we obtain at the end of the process, and that's a goal of

this. Curve evolution, once again, implemented

in the level sets framework. So this Geodesic active contours are used

a lot. This is another interesting example.

I'm going to run it a couple of times. That we start basically from multiple

curves. Let them evolve with a function of G and

basically we see that it attaches itself to the boundaries.

So look at the number five starts only from these two counters.

And it evolves to detect. This red line is detecting the numbers.

And we see this in here, basically. We start from a bunch of contours, and

they evolve, and they get integrated to find the boundaries.

We went from the image to a G function. From it, we did the curve evolution using

level sets, and we get these solutions. You can basically observe a number of

different things here. And get, you know, as I said before, this

is level sets you can play very freely with the initial conditions.

So this is one example. And active counters have been extended to

surfaces, so they're called active surfaces or minimal surfaces.

And here we see examples from grey matter segmentation in MRI.

So it's used a lot as we're going to discuss during the last week of this

class. It's used a lot in medical imaging.

This is one example in 3D, this is another example in 3D of finding very,

very tubular type of structures. All these different examples use

different G functions that are basically adapted to the problem.

We talk about one over the gradient, sometimes we have much more information

about the type of images that we are using and then the art is in the

designing of that G function that will help the counter or the surface to get

basically attracted to the correct boundaries to the correct regions.

So by this we are ending Week six. This has been a week where we learned a

lot of mathematical concepts in order to be able to do things like active contours

and isotropic diffusion. And there are many more applications of

partial differential equations in image and video processing.

I just illustrated a few. And I gave you some of the underlying

theory. It, it was as I say probably the most

mathematical-oriented. Week in this class, certainly these six

weeks. But it's also going to be the most

mathematical. Basically BC week of the whole class.

We're going to use PVs, also next week when we talk about image inpainting as in

our example. But as I say, every week is self

contained, so when you come to next week, of course you're going to, you are

already an expert in this area, so its going to be easier for you.

But I'm going to just talk about those equations, the ones that we use.

Next week and next week is not going to be as mathematical as this week.

It's just going to use those PD's and it, there going to be very intuitive to

understand. Imaging painting is a lot of fun and it

has a lot of relationship with biology and we are going to talk about that next

week. Thank you, I hope you had fun.

I know it was a hard week with a lot of math, but that's very useful math.

So I'm very happy I was, I had the chance to teach you about it.

Thank you. See you next week.