Matrices changing basis

Loading...

From the course by Imperial College London

Mathematics for Machine Learning: Linear Algebra

892 ratings

Imperial College London

Mathematics for Machine Learning: Linear Algebra

892 ratings

Course 1 of 3 in the Specialization Mathematics for Machine Learning

In this course on Linear Algebra we look at what linear algebra is and how it relates to vectors and matrices. Then we look through what vectors and matrices are and how to work with them, including the knotty problem of eigenvalues and eigenvectors, and how to use these to solve problems. Finally we look at how to use these to do fun things with datasets - like how to rotate images of faces and how to extract eigenvectors to look at how the Pagerank algorithm works. Since we're aiming at data-driven applications, we'll be implementing some of these ideas in code, not just on pencil and paper. Towards the end of the course, you'll write code blocks and encounter Jupyter notebooks in Python, but don't worry, these will be quite short, focussed on the concepts, and will guide you through if you’ve not coded before. At the end of this course you will have an intuitive understanding of vectors and matrices that will help you bridge the gap into linear algebra problems, and how to apply these concepts to machine learning.

From the lesson

Matrices make linear mappings

In Module 4, we continue our discussion of matrices; first we think about how to code up matrix multiplication and matrix operations using the Einstein Summation Convention, which is a widely used notation in more advanced linear algebra courses. Then, we look at how matrices can transform a description of a vector from one basis (set of axes) to another. This will allow us to, for example, figure out how to apply a reflection to an image and manipulate images. We'll also look at how to construct a convenient basis vector set in order to do such transformations. Then, we'll write some code to do these transformations and apply this work computationally.

Matrices changing basis11:14

Doing a transformation in a changed basis6:30

Meet the Instructors

David Dye
Professor of Metallurgy
Department of Materials
Samuel J. Cooper
Lecturer
Dyson School of Design Engineering
A. Freddie Page
Strategic Teaching Fellow
Dyson School of Design Engineering

We have said before that the columns of a transformation matrix,

are the axes of the new basis vectors of the mapping in my coordinate system.

We're now going to spend a little while looking at how to transform

a vector from one set of basis vectors to another.

Let's now have two new basis vectors that describe the world of Panda Bear here,

and Panda's world is orange.

So Panda has got first a basis vector there,

and then another basis vector there say.

And let's say in my world,

Panda Bear's basis vectors are at

three, one and at one, one.

And my basis vectors here are e2 hat equals nought, one.

and and e1 hat is equal to one, nought.

So those are my basis vectors and the orange ones are Panda's basis vectors.

Now, and so Panda's basis vectors,

the first one for Panda, is his one, zero,

and the second one is zero, one

in Panda's world.

So Bear's basis vectors are

three, one and one, one in my blue frame.

That is, I can write down Bear's transformation matrix as three, one, one, one.

Now, let's take some vector I want to transform.

Let's say that vector is in Bear's world,

is the vector a half of three,one

in Bear's world.

So it's three over two, one over two.

So the instruction there is do three over two have of three,one

and then do one over two a half of one,one

in my frame if you like.

So in my world,

that's going to give me the answer of three times three over two,

plus one times one over two is nine, ten halves,

which is five, and one times three over two,

plus one times a half,

so that's a total of two.

So that's the vector five, two

in my world, five, two.

Those two are the same thing.

So this is Bear's vector,

and this is my vector.

So this transformation matrix here are

Bear's basis in my coordinates,

in my coordinate system.

So that transforms Bear's vectors into my world,

which is a bit of a problem.

Usually, I'd want to translate my world into Bear's world.

So we need to figure out how to go the way.

So my next question is,

how do I perform that reverse process?

Well, it's going to involve the inverse.

So if I call Bear's transformation matrix B,

I'm going to want B inverse,

B to the minus one.

And the inverse of this matrix well,

it's actually pretty easy.

We can write down the inverse of that matrix pretty easily.

It's going to be a half of one, three,

flip the elements of the leading diagonal and put a minus on the off diagonal terms.

And we can see the determinant of that's three minus one over two.

So we divide by the determinant, that's a half.

So that's going to be B to the minus one.

And that's my basis vectors in Bear's coordinates.

So that's my basis in

Bear's world.

So my one, zero is going to be a half of one minus one in Bear's system,

and my zero, one is going to be a half of minus one, three

in Bear's system.

And we can verify that this is true if we take this guy,

a half one minus one,

and compose it with Bear's vectors,

we've got one plus minus one of those is going

give me three plus one is three minus one is two,

one minus one is zero, so that's two, zero halve it, gives you one zero.

So that really does work.

If I take a half one minus one of Bear's vectors,

I'll get my unit vector.

Okay. So that really does do the reverse thing.

So then if I take my vector, which was five, two,

and then I do that sum,

I should get the world in Bear's basis.

So I've got five times a half minus a half using that guy,

plus two times minus one, three

and that will give me a half of three, two

when I multiply all out.

And if I do the same thing here I got five times one,

minus one times two gives me three over two,

gives me three over two,

it all works out if you do it that way,

or if you do it that way, you'll still get that answer.

So that's Bear's vector again,

which is the vector we started out with.

So that's how you do the reverse process.

If you want to take Bear's vector into my world,

you need Bear's basis in my coordinate frame,

and if you want to do the reverse process,

you want my basis in Bear's co-ordinate frame.

That's probably quite counter-intuitive.

So let's try another example where this time

Bear's world is going to be an orthonormal basis vector set.

So here's our basis vectors one, zero, zero, one

in my world in blue,

and Bear's world is in orange,

Bear's world has one, one times,

and I've made a unit length so it's one over root two,

a minus one, one again,

unit lengths of one over root two, so there are those two.

And those you could do a dot product to

verify that those two are at 90 degrees to each other,

and they're Bear's vectors

one, zero and zero, one.

So then I can write down Bear's transformation matrix that transforms a vector of Bear's.

Now, if I've got the vector in Bear's world, this two, one,

then I can write that down,

and I will therefore get the vector in my world.

So when I multiply that out,

what I get is I'll get one over root two,

times two minus one, which is one,

and then one times two,

plus one times one, gives me three.

So in my world of vector is as I've written down,

one over root two times one, three.

So if I want to do the reverse transformation,

I need B to the minus one,

B to the minus ones is actually quite easy because this is an orthonormal basis.

The determinant of this matrix is one,

so it all becomes quite easy.

So I just get one over root two, keep the leading terms the same,

flip the sign of the off diagonal terms because

it's a two by two matrix, that's really simple.

And if you go and put that in, if you say,

if I take one over root two times one minus one,

so I take one of those plus one of those,

multiply by root two, I do in fact,

get one, zero and the same for zero, one

it all works.

So then if I take the vector in my world,

which is one, three,

I multiply it out,

then what I get is the vector in Bear's world.

So that's one plus three, which is four, one,

minus one plus three is two,

and I've got one over root two times one over root two, so that's a half.

So in Bear's world, this vector is two,one

which is what we actually said so it really works.

Now, this was all prep really for the fun part, which is,

we said before in the vectors module that we could do this just by using projections,

if the new basis vectors were orthogonal, which these are.

So let's see how this works with projections.

So let's try it with projections.

What we said before was that if I take my version of the vector,

and dot it with Bear's axis,

so the first of Bear's axis is that in my world,

then I will get the answer of the vector in Bear's world.

So that gives me one over root two, times one over root two,

which is a half of one plus three, which is four.

So that gives me two.

And that's going to be the first component of

Bear's vector because it's the first of Bear's axis.

And I can do it again with the other of Bear's axes.

So that's one over root two, one, three,

that's the vector in my world with the other of Bear's axis,

which is one over root two, times minus one, one.

And when I do that dot product,

what I'll get of one over root two is we'll multiply to give me a half again,

and I've got one times minus one,

plus three times one,

is a total of two,

which is one, and that's Bear's vector notice two, one.

So I've used projections here to translate

my vector to Bear's vector just using the dot product.

Now remember, with the vector product,

what I'd have to do is I'd have to remember to

normalize when I did the multiplication by Bear's vectors,

I'd have to normalize by their lengths.

But in this case, their lengths are all one.

So it's actually really easy.

So we don't have to do the complicated matrix maths,

we can just use the dot product if Bear's vectors are orthogonal.

Now, there is one last thing.

If you try this with the example we did before with Bear's vectors of

three, one and one, one.

So before we had those being Bear's vectors.

If you try the dot product with those because

they're not orthogonal to each other, it won't work.

Give it a go for yourself and verify that that it really won't work,

that they need to be orthogonal for this to work.

If you have them not being orthogonal,

you can still do it with the matrix transformation,

you just can't do it with a dot product.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2018 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play