Hi everyone. This is Professor [inaudible]. Now we're going to study the mathematics for the AI in our part one, linear algebra, Week 6 of session one. So far we study about linear algebra to understand AI in a more deep way. So far we study about system of linear algebra equation, which is AX=B. We can find the solution by using our row operations. After that, we also can find the inverse of square matrix A by using the row operations tool. Also determinant of square matrix A, we could find those determinant by using the row operation. Big pictures, how do threes of the major part of the linear algebra are related to each other, connected by linearly independent vectors. Also, we studied about the matrix eigenvalue problem. This week, we are going to study about diagonalization problem, which is very essential part for finding the solutions and also finding the sum key factors in a lot of Big Data. This diagonalization problem is very useful to use in some search engine like Google or some AI application for finding some spatial pressure some figures of the inside of the Big Data. Once you understand this diagonalization problem, we can also use this our linear algebra background, on the AI applications such as deep learning and support vector machine, et cetera. Let's start the diagonalization problem. Now we're going to study diagonalize and we are going to diagonalize our square matrix. Diagonalization problem means that we are going to diagonalize our square matrix. What does it mean to diagonalize a square matrix? How can it be done? Also, the diagonalization problem is related to the matrix eigenvalue problem. The problem of diagonalizing our matrix may be stated as : in a given an N by N matrix A, can you find the invertible N by N matrix P and an N by N diagonal matrix D such that A equals P times D times P inverse. We try to make matrix A to the different form, which is PDP inverse. Here, P and P inverse is, we are going to figure out by using the eigenvalue problem. The D is also diagonal matrix is such as similar form like this. This is 5 times 5 diagonal matrix. Diagonal matrix has only element in the diagonal direction here, 1,2,3,4,5. In order of the diagonal direction and diagonal place and the other elements are all zero. Also, this diagonal matrix only define in the square matrix, such as here, like 5 times 5 or N by N matrix. The task of diagonalizing A is simply to find the matrix P and D, such that A can be written as PDP inverse. Here P also record as our hexagonal matrix. We tried to find orthogonal matrix P and diagonal matrix D, so that we can finally diagonalize matrix A, which is we decompose the A to the PDP inverse. Is it all N by N matrix can be diagonalized? No, it's not. Not the all square matrix can be diagonalized. We will study about what kind of square matrix A can be diagonalized, and if we can diagonalize matrix A, then how can you do the diagonalization? We can diagonalize the N by N matrix A. Then that means we can write A equals PDP inverse, if and only if A has N linearly dependent eigenvectors. Now, this is a condition for the matrix A having diagonalization matrix. If A has N linearly independent eigenvectors, then we can diagonalize matrix A. If matrix A does not have N linearly independent eigenvectors, then this matrix cannot be diagonalized. If A has N linearly independent eigenvectors such as X_1, X_2, X_N, so here X is our linearly independent eigenvectors. How do we construct P and D? Once you have P and D, we can find diagonalization of matrix A. P orthogonal matrix is composed with, we can write like this, X_1, X_2 to X_N here. X_1, X_2 to X_N is eigenvectors. It's N by 1 matrix or vectors, and dimensional vectors or N by N vectors. You can imagine X_1 has N numbers, X_2 has N numbers, and X_N has N numbers on there. P is N by N matrix. It looks like P is just a vector form, but it's not. Just be careful. We just simplify the notation like this. P equals matrix with X_1, X_2, X_3, X_N. Those X are linearly independent eigenvectors of A. To formulate this orthogonal matrix P, first thing we need to do is we need to find linearly independent eigenvectors of A, then we can construct this orthogonal matrix P. This first column of P will be a first eigenvector of linearly independent eigenvectors of A. The second column of P will be the second linearly independent eigenvectors of A. Like this. The last column, Nth column of P, will be the Nth independent eigenvectors. The first element of diagonal matrix D here, which is Lambda 1 is eigenvalue for the X_1. We need to place those Lambda 1, Lambda 2 to Lambda N, which is eigenvalues of A in a corresponding orders when we formulate D and P. Again, when you put the Lambda 1 in the first column of D, then we should put the X_1 in the first column of P. If we put the second eigenvectors, which is X_2 in the second column of P, then we need to put Lambda 2, which is corresponding to the X_2 in the second column of D. Let me explain more easy way with example. Let's diagonalize the matrix A here. A is 0, 0, 2, minus 2, 2, 2, and 0, 0, 2. This is 3 by 3 matrix. In this example of A, A has only two distinct eigenvalues here, 0 and 2. You can find the eigenvalues by using the elastic Week 5. Our contention about how to find eigenvalues and eigenvectors. Corresponding eigenvectors, we need to find it. For the Lambda equals 0, which is first eigenvalues, you can find eigenvectors x, y, z equals t times 1, 1, 0. For the second eigenvector, which second and third is duplicated, so Lambda_2, then x, y z equals what? R times 1, 0, 1 plus s times 0, 1, 0. Here, we need to find three linearly independent eigenvector to diagonalize the matrix A. We can find orthogonal matrix P and diagonal matrix Lambda. Here, we take t equals 1 to give one eigenvectors here. Which is, when t equals 1, it becomes 1, 1, 0 which is first eigenvector. The second one, here before that, we cannot take the second eigenvector here, why? Can you answer that? Because if for the eigenvalue Lambda equals 0, then eigenvector is t times 0, 1, 0. If we take another eigenvector for the Lambda equals 0, then it will be what? Linear independent eigenvectors. We only need linearly independent eigenvectors. We can only take one single linearly independent eigenvector for Lambda equals 0. For Lambda equal 2, let's take r equals 1 and s equals 0 to obtain 1, 0, 1. If you take r equals 0 and x equals 1, we can obtain 0,1, 0. Now, we have three linearly independent eigenvectors, which is 1, 1, 0, 1, 0, 1, 0, 1, 0. Those three linearly independent eigenvectors, we can do the diagonalizing A, which is A equals PDP inverse. As I introduced in the first, P is orthogonal matrix, is formulated with the corresponding eigenvectors , linearly independent eigenvectors. We have three linearly independent eigenvectors here. First eigenvectors, we should put in the first column of orthogonal matrix P. So 1, 1, 0 will be placed in the first column of P, and second column of orthogonal matrix P can be used with 1, 0,1, which is second linearly independent eigenvector , 1, 0, 1 here. Third linearly independent eigenvector here is 0, 1, 0, this 0, 1, 0 will be placed in the third column of P. Then you make a diagonal matrix D. We should put the corresponding, we should match with the eigenvectors and eigenvalue here. For the eigenvector 1, 1, 0, we should put the Lambda equals 0 because eigenvector 1, 1, 0 is come by using the Lambda equal 0. The second eigenvector 1, 0, 1, we use the corresponding eigenvalue Lambda equals 2. We should put Lambda 2 in the second column in the D. Third eigenvectors is 0, 1, 0. Corresponding eigenvalues is Lambda equal 2. We should put the Lambda equals 2 in the third, in our diagonal matrix. We can calculate P inverse by using row operation. Finally, we have P and DP inverse, so we can diagonalize it. Please check A equals PDP inverse, and P times D times P inverse then should have matrix A. Up to here, we've studied about how to diagonalize matrix A. In the next section, we will study more about the different cases for the diagonalization. Thank you very much.
