Hi, everyone. In this video, we're going to talk about matrix factorization. Last time we talked about collaborative filtering, which we talked about how to use a similarity measure to identify similar items and similar users. These methods are often called the neighborhood method because by similarity, we partition different two items in the similar region, they are close to each other in the feature space, so they are in neighborhood. Another very different approach is called latent factor models, which instead of focusing on the relationship between two items or two users, we focus on identifying latency meant vectors. Oftentimes these latent semantic vectors are some subgroups of these items. For example, if you had users, so these are the users and these are the movies. When we find a latent vector out of these data metrics, we can extract the factor such as genre. But sometimes latent factors can pick up something else that genre that characterizes some subgroups of these different items. There can be different methods in latent factor models, but metrics factorization is very useful and widely used. We're going to talk about those. What is matrix factorization? Matrix factorization means that we factorize matrices into different pieces. For example, if we want to do factorization with the two matrices, we make this big matrix with the m-users and n-items into m users and some category or genre d times that genre, number of genres, or categories times the number of items. When you think about this original utility matrix, its size is n times n, but we factor it into smaller matrices so it's m times d and then these symmetricity times n. In terms of number of elements, when we say d is much smaller than either of m or n, then we can extract some information out of these huge and times n matrices into these smaller two matrices. That's a matrix factorization and in particular, this case, these is two matrices. We're factoring those utility metrics into two matrices. In some cases, we're going to have also three matrices factorization. In that case, we have m times n matrix and vector into this d times d, times d times n matrices so in this case, it's a three matrices vectors. In this case, this matrix in the middle oftentimes gives us strength. It's oftentimes it's diagonal and then it gives us strength and talk about what's the importance among these vectors. When you think about genres, it's going to tell you how much of this relationship is concentrated on that genre or etc. Well, can you have more than two or three factors in the matrix factorization? Yes, we could however, it's not very useful. The two-factor matrix factorization is very common because it's easier and we use it when we care about how many genres are there and separate the data and vectors such that the user and genre interaction and genre and item interactions can be found. Surrey matrix, matrix factorization is also very common because it can give us information about the strings or importance about the genre or the categories. This importance or those trends that are in the diagonal elements in this middle matrix can be obtained by solving eigenvalue problem. We're going to talk about those later. Also, matrix factorization can be done analytically or it can be done using approximation method, which is even more powerful when the matrix is very large. We're going to talk about those as well. Matrix factorization can be much more flexible, we can also impose some constraints on the elements. For example, if we say all of these elements in these matrices need to be positive or non-negative. Oftentimes these additional constraints on the matrix factorization can be useful to find the solutions that makes sense more. Let's take some examples of what we can do with matrix factorization. This work, 1999, made matrix factorization really popular, this is called non-negative matrix factorization. This original matrix of a person's face image can be factored into two matrices. These two matrices are constrained, all the elements to be non-negative so everything is greater than or equal to zero. By doing that, they factor into two matrices, we can call it W and H. This W is basis matrix, so it's a basis for face so it has some shapes like maybe mouth and some nose and things like that. They have some basis here, and this H means it's a weight to each basis. By having the original matrix factored into the basis and then weight to that basis, they can reconstruct these facial images. Matrix factorization can be used in image analysis. Similarly, it can be also used in multi-spectrum images. For example, this geographic images, they use the matrix factorization to factorize different spectrum that comes from different materials in the images. Matrix factorization can be also useful to separate out different sources or different spectrum groups from the images or even sound. As we mentioned, all of these different sources or spectrum groups can be thought to be latent vectors that's hidden in the original image, but it can be factored out in the vectorized images or matrices. Another really good application area of matrix factorization is text mining or we also call it topic modeling. Using matrix factorization, we want to find out some semantic latent features such as topics of the document or articles. How it works, utility matrix is a vocabulary in all the document. Each column can be thought to be a document and the row can be words or the vocabulary or maybe word embeddings. Once we have this big matrices that has a lot of vocabularies or words in a lot of documents, we can factor into two matrices. This matrix will be still vocabulary on the row, and the column of this first matrix will be topics. Think about some news articles, we have a lot of news articles that we script from Internet and then we want to find out what the topics are in each document. As I mentioned, these are the vocabularies and this is topic, and this matrix is about the topic and the document. We are interested to find out what each document's topics are, maybe this news article was politics, this news article was about sports, entertainment, and so on. Not only the image or text analysis, matrix factorization can be used in variety of different fields. For example, we can use matrix factorization to separate the sources of the audio signals. In analytic chemistry, we can separate the spectrum of different chemical mixtures to find out the pure components and its concentration. We can use matrix factorization in gene analysis. When we have a microarray data that has different genes and different samples and gene expression level in each element, we can factor out some subgroups of genes. These are all the genes and these are some group of genes which are called the metagenes, and each of these metagenes get activated in the samples. Also matrix factorization can be very useful in recommender systems. There are many different types and techniques in matrix factorization, we are not going to cover them all but we're going to focus on three things. The first concept is called the singular value decomposition, which is a generalization of eigenvalue decomposition. It is a useful when we want to know the importance of the strengths of the latent vectors. Practically very useful matrix factorization technique is non-negative matrix factorization. A lot of cases, non-negative values of the elements makes sense, and then it can be very flexible, so we're going to talk about this. Another technique that is useful is approximation method. In the approximation method we approximate the matrix values, so we are now going to use exact method to decompose. However, we're going to factorize our matrix into two or three pieces, and by using optimization technique we're going to figure out the values of each element in the matrices.
