In the last video, we learned about

orthogonal projections onto one-dimensional subspaces.

In this video, we look at the general case of

orthogonal projections onto n dimensional spaces.

For this, we exploit the same concepts that worked in the one-dimensional case.

Let's start with an illustration.

We're going to look at a case where we have

a vector x that is living in a three-dimensional space,

and we define a subspace, a two-dimensional subspace,

u which has basis vectors b1 and b2,

which are for example,

this vector, and b2 is this vector.

So, we write u is spanned by b1 and b2.

So, u in this case would be the plane down here, this is u.

So, now we're looking at the orthogonal projection of x onto u,

and we're going to denote this by pi u of x.

So, that projection is going to look something like this.

That's the projection point.

So, this is the orthogonal projection of x onto the subspace u.

So, we can already make two observations.

The first thing is that because pi u of x is an element of u,

it can be represented as a linear combination of the basis vectors of u.

So, it means we can write pi u of x is lambda one times b1,

plus lambda two times b2,

for appropriate values of lambda one, and lambda two.

And the second property is that the difference vector of x minus pi u of x,

so this vector over here is orthogonal to u,

which means it's a orthogonal to all basis vectors of u.

And we can now use the inner product for this,

and we can write that x minus pi u of x inner product with b1 must be zero,

and the same is true for b2.

But now, let's formulate our intuition for the general case,

where x is a D-dimensional vector,

and we are going to locate an M-dimensional subspace u.

Okay. Let's derive this result.

I copied our two insights up here,

and I have defined two quantities,

a lambda vector which consists of all these lambda i here,

and a B-matrix where we just concatenate all basis vectors of our subspace u.

Now, with this definition we can also

write pi u of x equals B times lambda.

Let's assume we use the dot products as our inner product.

Now, if we use our second property, we'll get that,

so pi u of x minus x inner product with b_i is now equivalently

written as the inner product of

B lambda minus x and bi,

and this needs to be zero,

where I just used the definition of pi u of x in here.

So, now we can simplify this by exploiting the linearity of the inner product,

and we'll get B lambda times or inner product

with bi minus the inner product of x with bi needs to be zero,

and this holds for i equals one to M. With the inner product,

we can now write this in the following way.

We can write this as lambda transpose times

B transpose times bi

minus x transpose times bi equals zero,

for i equals one to M. And now,

we can write this as a set of conditions,

and if we summarize this,

we would get lambda transpose times

B transpose times B minus x transpose times B must be zero.

Now, we need to talk here about an M-dimensional zero vector.

What we would like to do now,

is we would like to identify lambda.

For this, we are going to

right multiply the inverse of B transpose times B onto the entire equation,

and then we get lambda transpose equals x transpose

times B times B transpose B inverse,

which then also means we can write lambda as the transpose of this entire expression,

we get B transpose B inverse,

so this matrix is symmetric,

so its transpose is the same as the original matrix,

times B transpose x.

So, now we have identified lambda to be this,

but we also know that our projection point can be written as B times lambda.

So, this means we will get pi u of x as B times Lambda,

which is B times B transpose B inverse times B transpose x.

We can now identify

this expression as the projection matrix similar to the one-dimensional case.