0:08

Let's start by talking about matrix

Â inverse, and as

Â usual we'll start by thinking about

Â how it relates to real numbers.

Â In the last video, I said

Â that the number one plays the

Â role of the identity in

Â the space of real numbers because

Â one times anything is equal to itself.

Â It turns out that real numbers

Â have this property that very

Â number have an, that

Â each number has an inverse,

Â for example, given the number

Â three, there exists some

Â number, which happens to

Â be three inverse so that

Â that number times gives you

Â back the identity element one.

Â And so to me, inverse of course this is just one third.

Â And given some other number,

Â maybe twelve there is

Â some number which is the

Â inverse of twelve written as

Â twelve to the minus one, or

Â really this is just one twelve.

Â So that when you multiply these two things together.

Â the product is equal to

Â the identity element one again.

Â Now it turns out that in

Â the space of real numbers, not everything has an inverse.

Â For example the number zero

Â does not have an inverse, right?

Â Because zero's a zero inverse, one over zero that's undefined.

Â Like this one over zero is not well defined.

Â And what we want to

Â do, in the rest of this

Â slide, is figure out what does

Â it mean to compute the inverse of a matrix.

Â 1:39

Here's the idea: If

Â A is a n by

Â n matrix, and it

Â has an inverse, I will say

Â a bit more about that later, then

Â the inverse is going to

Â be written A to the

Â minus one and A

Â times this inverse, A to

Â the minus one, is going to

Â equal to A inverse times

Â A, is going to

Â give us back the identity matrix.

Â Okay?

Â Only matrices that are

Â m by m for some the idea of M having inverse.

Â So, a matrix is

Â M by M, this is also

Â called a square matrix and

Â it's called square because

Â the number of rows is equal to the number of columns.

Â Right and it turns out

Â only square matrices have inverses,

Â so A is a square

Â matrix, is m by m,

Â on inverse this equation over here.

Â Let's look at a concrete example,

Â so let's say I

Â have a matrix, three, four,

Â two, sixteen.

Â So this is a two by

Â two matrix, so it's

Â a square matrix and so this

Â may just could have an and

Â it turns out that I

Â happen to know the inverse

Â of this matrix is zero point

Â four, minus zero point

Â one, minus zero point

Â zero five, zero zero seven five.

Â And if I take this matrix

Â and multiply these together it

Â turns out what I get

Â is the two by

Â two identity matrix, I,

Â this is I two by two.

Â Okay?

Â And so on this slide,

Â you know this matrix is

Â the matrix A, and this matrix is the matrix A-inverse.

Â And it turns out

Â if that you are computing A

Â times A-inverse, it turns out

Â if you compute A-inverse times

Â A you also get back the identity matrix.

Â So how did I

Â find this inverse or how

Â did I come up with this inverse over here?

Â It turns out that sometimes

Â you can compute inverses by hand

Â but almost no one does that these days.

Â And it turns out there is

Â very good numerical software for

Â taking a matrix and computing its inverse.

Â So again, this is one of

Â those things where there are lots

Â of open source libraries that

Â you can link to from any

Â of the popular programming languages to compute inverses of matrices.

Â Let me show you a quick example.

Â How I actually computed this inverse,

Â and what I did was I used software called Optive.

Â So let me bring that up.

Â We will see a lot about Optive later.

Â Let me just quickly show you an example.

Â Set my matrix A to

Â be equal to that matrix on

Â the left, type three four

Â two sixteen, so that's my matrix A right.

Â This is matrix 34,

Â 216 that I have down

Â here on the left.

Â And, the software lets me compute

Â the inverse of A very easily.

Â It's like P over A equals this.

Â And so, this is right,

Â this matrix here on my

Â four minus, on my one, and so on.

Â This given the numerical

Â solution to what is the

Â inverse of A. So let me

Â just write, inverse of A

Â equals P inverse of

Â A over that I

Â can now just verify that A

Â times A inverse the identity

Â is, type A times the

Â inverse of A and

Â the result of that is

Â this matrix and this is

Â one one on the diagonal

Â and essentially ten to

Â the minus seventeen, ten to the

Â minus sixteen, so Up to

Â numerical precision, up to

Â a little bit of round off

Â error that my computer

Â had in finding optimal matrices

Â and these numbers off the

Â diagonals are essentially zero

Â so A times the inverse is essentially the identity matrix.

Â Can also verify the inverse of

Â A times A is also

Â equal to the identity,

Â ones on the diagonals and values

Â that are essentially zero except

Â for a little bit of round

Â dot error on the off diagonals.

Â 5:45

If a definition that the inverse

Â of a matrix is, I had

Â this caveat first it must

Â always be a square matrix, it

Â had this caveat, that if

Â A has an inverse, exactly what

Â matrices have an inverse

Â is beyond the scope of this

Â linear algebra for review that one

Â intuition you might take away

Â that just as the

Â number zero doesn't have an

Â inverse, it turns out

Â that if A is say the

Â matrix of all zeros, then

Â this matrix A also does

Â not have an inverse because there's

Â no matrix there's no A

Â inverse matrix so that this

Â matrix times some other

Â matrix will give you the

Â identity matrix so this matrix of

Â all zeros, and there

Â are a few other matrices with properties similar to this.

Â That also don't have an inverse.

Â But it turns out that

Â in this review I don't

Â want to go too deeply into what

Â it means matrix have an

Â inverse but it turns

Â out for our machine learning

Â application this shouldn't be

Â an issue or more precisely

Â for the learning algorithms where

Â this may be an to namely

Â whether or not an inverse matrix

Â appears and I will tell when

Â we get to those learning algorithms

Â just what it means for an

Â algorithm to have or not

Â have an inverse and how to fix it in case.

Â Working with matrices that don't

Â have inverses.

Â But the intuition if you

Â want is that you can

Â think of matrices as not

Â have an inverse that is somehow

Â too close to zero in some sense.

Â So, just to wrap

Â up the terminology, matrix that

Â don't have an inverse Sometimes called

Â a singular matrix or degenerate

Â matrix and so this

Â matrix over here is an

Â example zero zero zero matrix.

Â is an example of a matrix that is singular, or a matrix that is degenerate.

Â Finally, the last special

Â matrix operation I want to

Â tell you about is to do matrix transpose.

Â So suppose I have

Â matrix A, if I compute

Â the transpose of A, that's what I get here on the right.

Â This is a transpose which is

Â written and A superscript T,

Â and the way you compute

Â the transpose of a matrix is as follows.

Â To get a transpose I am going

Â to first take the first

Â row of A one to zero.

Â That becomes this first column of this transpose.

Â And then I'm going to take

Â the second row of A,

Â 3 5 9, and that becomes the second column.

Â of the matrix A transpose.

Â And another way of

Â thinking about how the computer transposes

Â is as if you're taking this

Â sort of 45 degree axis

Â and you are mirroring or you

Â are flipping the matrix along that 45 degree axis.

Â so here's the more formal

Â definition of a matrix transpose.

Â Let's say A is a m by n matrix.

Â 8:31

And let's let B equal A

Â transpose and so BA transpose like so.

Â Then B is going to

Â be a n by m matrix

Â with the dimensions reversed so

Â here we have a 2x3 matrix.

Â And so the transpose becomes a

Â 3x2 matrix, and moreover,

Â the BIJ is equal to AJI.

Â So the IJ element of this

Â matrix B is going to be

Â the JI element of that

Â earlier matrix A. So for

Â example, B 1 2

Â is going to be equal

Â to, look at this

Â matrix, B 1 2 is going to be equal to

Â this element 3 1st row, 2nd column.

Â And that equal to this, which

Â is a two one, second

Â row first column, right, which

Â is equal to two and some [It should be 3]

Â of the example B 3

Â 2, right, that's B

Â 3 2 is this element 9,

Â and that's equal to

Â a two three which is

Â this element up here, nine.

Â And so that wraps up

Â the definition of what it

Â means to take the transpose

Â of a matrix and that

Â in fact concludes our linear algebra review.

Â So by now hopefully you know

Â how to add and subtract

Â matrices as well as

Â multiply them and you

Â also know how, what are

Â the definitions of the inverses

Â and transposes of a matrix

Â and these are the main operations

Â used in linear algebra

Â for this course.

Â In case this is the first time you are seeing this material.

Â I know this was a lot

Â of linear algebra material all presented

Â very quickly and it's a

Â lot to absorb but

Â if you there's no need

Â to memorize all the definitions

Â we just went through and if

Â you download the copy of either

Â these slides or of the

Â lecture notes from the course website.

Â and use either the slides or

Â the lecture notes as a reference

Â then you can always refer back

Â to the definitions and to figure

Â out what are these matrix

Â multiplications, transposes and so on definitions.

Â And the lecture notes on the course website also

Â has pointers to additional

Â resources linear algebra which

Â you can use to learn more about linear algebra by yourself.

Â