Explore
For Enterprise
Join for Free
Log In

Arts and Humanities
Business
Computer Science
Data Science
Information Technology
Health
Math and Logic
Personal Development
Physical Science and Engineering
Social Sciences
Language Learning
Degrees
Certificates

Explore all of Coursera

Browse
Search
For Enterprise
Log In
Join for Free

1.4.2. GMM Parameter Estimation via EM

Loading...

Robotics: Estimation and Learning

University of Pennsylvania

4.2 (382 ratings) | 22K Students Enrolled

Course 5 of 6 in the Robotics Specialization

How can robots determine their state and properties of the surrounding environment from noisy sensor measurements in time? In this module you will learn how to get robots to incorporate uncertainty into estimating and learning from a dynamic and changing world. Specific topics that will be covered include probabilistic generative models, Bayesian filtering for localization and mapping.

Skills You'll Learn

Particle Filter, Estimation, Mapping

Reviews

4.2 (382 ratings)

5 stars
220 ratings
4 stars
84 ratings
3 stars
45 ratings
2 stars
15 ratings
1 star
18 ratings

AV

Jun 25, 2016

A tough course with few hours of lecture material and some good programming assignments.You will be satisfied by those assignments however .

VM

Sep 19, 2018

This is a really comprehensive course which gave me a good knowledge about Gaussian Model and Kalman Filter ...

From the lesson

Gaussian Model Learning

We will learn about the Gaussian distribution for parametric modeling in robotics. The Gaussian distribution is the most widely used continuous distribution and provides a useful way to estimate uncertainty and predict in the world. We will start by discussing the one-dimensional Gaussian distribution, and then move on to the multivariate Gaussian distribution. Finally, we will extend the concept to models that use Mixtures of Gaussians.

1.4.1. Gaussian Mixture Model (GMM)4:56

1.4.2. GMM Parameter Estimation via EM7:20

1.4.3. Expectation-Maximization (EM)6:07

Taught By

Daniel Lee
Professor of Electrical and Systems Engineering

Transcript

In this lecture, you're going to learn how to compute and

estimate of the Gaussian mixture model parameters from observed data.

We will first try to follow what we did before

to get the maximum likelihood estimate for single Gaussians.

But as I will show, we are going to fail to obtain nice analytics solution.

Instead, I will introduce a new way to get a locally optimal solution,

using an iterative algorithm called Expectation Maximization or EM for short.

While a single Gaussian has only two parameters, mu and

sigma, GMM has multiple mu and sigma, plus weights, and

the number of Gaussian components, denoted here as K.

As I mentioned in the previous lecture, we will use uniform weights 1 over k and

focus on understanding how to estimate the mean and covariance matrix parameters.

Let's begin to find the maximum likelihood estimate

of GMM parameters, as we did for single Gaussians.

To briefly review, maximum likelihood estimation means we want to find

the parameters of the model that is most likely to produce the observed data.

As before, instead of maximizing the joint probability, we can assume

independence of all observations and maximize the product of each probability.

Next, we take the log and maximize the sum of the log of probabilities,

instead of the products of probabilities.

So far, we get the same results.

But when we apply the specific probability model of the GMM

into the equation, which is a sum of Gaussians, we have this.

It turns out that we cannot further simplify this formula analytically,

because there appears a summation of Gaussians inside the log function.

This implies we can estimate the parameters

only via iterative computations.

And any solution found might not be globally optimal but,

don't be too disappointed.

We are going to learn the EM Algorithm shortly, and

the algorithm gives a reasonable solution under certain conditions.

The way we are going to learn the EM Algorithm is first,

see how we compute the GMM parameters as a special case.

And then move to the general ideas of EM for interested advanced learners.

Having said that, let us first talk about EM for computing GMM parameters specifically.

We need two additional things, an initial guess of mu and sigma and

a new variable Z that we are going to call a latent variable.

It might sound strange that we need a guess to solve the parameters,

which we want to estimate.

However, in this class of complicated problems called non-convex optimization,

there exists many suboptimal solutions, which are called local minimum.

And the initial guess affects the solution found.

While the initial guess could be important to this problem,

it will not be the focus of this lecture.

So now let's turn to the new variable, Z.

The latent variable Z of the ith points with respect to the kth Gaussian

is defined as the relative ratio of the kth Gaussian density at that point.

Essentially, Z indicates the probability that

the ith point is generated from the kth Gaussian components.

Let's look at a 1D example with K equals 2.

We have this point Xi.

And for the moment, we have some prior of two Gaussians G1 and G2.

If the value of G1 of Xi is P1 and

the value of G2 of Xi is P2.

And the value of Zi1 is computed

as P1 over P1+P2, and

Zi2 is computed as P2 over P1+P2.

In this particular case, P1 is larger than P2.

So it is more probably that Xi is generated by G1 than G2.

Now given that the latent variable values for all data points and

all the Gaussian elements, we can redefine the mean and

covariance matrices weighted by Z.

If the probability that a point belongs to the kth

Gaussian is small, then the points contributes less for

computing the parameters of that particular kth Gaussian.

Putting everything we have discussed so far together, you can compute

the parameters and the latent variables iteratively until the values converge.

First, we initialize the parameters.

Then use the initial value of mu and sigma to compute Z, the latent variables.

Once Z is updated, now we fix them and update mu and sigma.

You will alternate between these two steps

until the change of parameters becomes very small.

We are not going to prove it here, but

it can be shown that this iteration converges to a local optimum.

I have introduced a use of the EM Algorithm for

parameter estimation of GMMs, which work through iterations.

We have seen that with an initial setting of the mean and

covariance and by introducing a latent variable,

we've surprisingly turned the problem tractable in a locally optimal sense.

In the next lecture, I will introduce the EM Algorithm more generally.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play

Coursera
About
Leadership
Careers
Catalog
Certificates
MasterTrack™ Certificates
Degrees
For Enterprise
For Government

Community
Learners
Partners
Developers
Beta Testers
Translators

Connect
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Tech Blog

More
Terms
Privacy
Help
Accessibility
Press
Contact
Directory
Affiliates