In this course you will learn a variety of matrix factorization and hybrid machine learning techniques for recommender systems. Starting with basic matrix factorization, you will understand both the intuition and the practical details of building recommender systems based on reducing the dimensionality of the user-product preference space. Then you will learn about techniques that combine the strengths of different algorithms into powerful hybrid recommenders.
Hybridization through Matrix Factorization
Loading...
From the course by University of Minnesota
Matrix Factorization and Advanced Techniques
109 ratings
From the lesson
Hybrid Recommenders
This is a three-part, two-week module on hybrid and machine learning recommendaton algorithms and advanced recommender techniques. It includes a quiz (due in the second week), and an honors assignment (also due in the second week). Please pace yourself carefully -- it will be difficult to finish the honors track in two weeks unless you start the assignments during the first week.
Meet the Instructors
Michael D. EkstrandAssistant Professor
Dept. of Computer Science, Boise State University
Joseph A KonstanDistinguished McKnight Professor and Distinguished University Teaching Professor
Computer Science and Engineering
Okay, welcome back.
In this video, we're going to talk about a different approach to hybridization that
does matrix factorization, a single algorithm on hybrid sources of data.
We've seen a few different kinds of hybrid recommenders, and
most of these recommenders have focused on
hybridizing a single recommender out of multiple component recommenders.
You might mix a content based filter and
a matrix factorization collaborative filter or a nearest neighbor collaborative
filter in order to produce your final recommendations.
In this video, we're going to look at single families of algorithms,
particularly matrix factorization,
where we're going to be doing the hybridization at the data level.
We're going to be performing this dimensionality reduction on what we
call heterogeneous data, which is data of multiple forms, users rate items,
users have demographic characteristics.
Items have cast members, or genre descriptors, or things like that.
We're going to see some specific examples of this in the interviews at the end of
this module.
This is going to provide the overview the high level framework and
talk about a couple of the algorithms that do this kind of thing.
Sometimes in the research literature this kind of work is called side information.
So if we think about the big ratings matrix r.
So we've got this ratings matrix, and we've got users,
items, and they rate things, with stars or whatever.
From a collaborative filtering perspective,
that's our main piece of information and then we have this side information.
Movie reviews.
Restaurant menus.
Whatever other information that we can obtain about the items that we're
recommending, or about the users that we're going to be recommending them to,
and we call this side information.
And this is because it's additional information on the side that adds
to what we have from our collaborative filtering dataset,
the ratings matrix, or any kind of user item interaction matrix,
such as purchase counts, play counts, that kind of thing.
So, backed up just a little bit we saw and we're talking about matrix factorization.
The machine learning approach, the learning models for recommendation.
We define some kind of a cost utility function, this might be RMSE.
That's what we did when we derive from K-SVD.
We define a model such as SVD.
R = P sigma Q transpose.
We optimize the model parameters
to achieve the best performance on our utility or our cost function.
We can make many different kinds of models.
The matrix factorization R = P sigma Q transpose, or PQ transpose is just one
kind of model we can do, and we can use many different cost functions.
So in the rest of this video, we're going to look at some additional models.
You can train them using the same kinds of approaches.
You can use the gradient decent we talked about.
You can use expectation maximization.
You can use various least square solving methods, depending on your cost function.
There's many different training methods.
In later videos, we're going to look at
some additional cost functions that you can use in the matrix factorization.
For now, we're still going to be operating in the RMSE matrix completion framework,
if we want to do the rating prediction.
But the same concepts can apply when you swipe out the cost function to
optimize for something else and you can train these same kinds of models.
This paradigm gives us basically a modular approach to recommendation.
Well we've got the cost function, we've got the model,
we've got the optimization strategy, and not all combinations make sense.
But we can swap each of those components out to change how the recommender behaves.
So to warm up, let's start by talking a little bit again about bias models.
So we've got this bias model, baseline model where the score of the user and
the item is a global bias which is generally going to be the mean rating if
we're operating with ratings data.
If we're operating with implicit feedback data, it's going to be something else.
We have an item bias which is how much better or
worse this item is than the average item.
And a user bias,
which is how positive this user is overall, compared to the average user.
When we're doing rating prediction, this can personalize scores, but
it can't really personalize rankings,
because the only user base term is this bu, which is the same for all items.
So it's basically going to rank items by their mean rating or
their popularity or however we're computing that item bias term.
We can learn this bias model using gradient descent.
We can optimize these parameters to minimize the squared
error predicting ratings.
We can put it into a logistic regression to predict whether or
not a user's going to buy an item.
In that paradigm we can also then apply regularization but
also we can just compute a mean.
So b is going to be the user's average rating.
b sub i is the average offset from the user's average rating.
And then b sub u is the average of the offset from that.
We can extend this to other kinds of linear models.
Where we've got out baseline and this is just the whole baseline we just saw.
And then we have coefficients for
maybe what time of week it is which effects how much positive the user is.
The genre, because some genres are more popular.
And then may be the age of the user, we can throw some of these in there.
We can also have interaction terms that may be we have user specific
coefficient for some of these and then we can start doing some personalized ranking.
And again, we can learn these parameters, in our machine learning terms,
these coefficients are going to be the parameters,
we could learn those using any of our optimization techniques.
So, SVD++ is to bring this out extending our feature
base to thinking into actually working with matrix factorization.
SVD++ is matrix factorization that also takes into
account whether or not an item has been rated at all.
So traditional matrix factorization R = PQ
transpose that only takes
into account the rating values.
We have no way in this training process to account for whether the user and
the prediction process to encounter for what the user has actually rated.
It kind of explicit feedback late in the rating matrix,
this model here extends that paradigm to take it that in the account.
So piece of you is our standard P vector, q
sub i is our standard q vector, but we have these additional vectors that
are the contribution to the user's preference for latent features.
These are all latent feature vectors.
That is account that we can infer from the fact that they rated some other item.
So rather than encoding all of the users
preference for features in the P matrix
We split it into two components.
We say that the fact that the user has rated a bunch of movies.
The user watched Blade Runner, and they watched My Fair Lady, and
they watched The African Queen.
What this model does is it says that the fact that they watched
those movies tells us something about their preferences.
So what we do is we say that the y sub j
that corresponds for My Fair Lady.
What that does, we have one,
what that does is it encodes what can we learn about a user's preference for
the latent features from the fact that they watched My Fair Lady.
What latent feature preferences basically
are in common among all users that watched My Fair Lady?
And we do that for all of the movies, and we then,
p sub u, what it stars is the leftover feature,
that the user's feature preferences that are not accounted for
in their choice of movies that they watch.
We train all these parameters, our baselines,
our q, p, and now y matrices using gradient descent and
now we have another way to incorporate new users, we might not know a P vector for
user, but we know they've watched three movies.
We can just take their y vectors for those movies, and
use them to start doing recommendations for
that user really fast without even needing to try to train a p vector.
Provides a really natural way to incorporate new users, but
it also allows us to kind of distribute out this knowledge of
what the user's profile is, break it down some more, and
take into account what movies have they even rated in the first place?
So to take this further SVD, so one way we can think about this is that
each user has a bunch of features, which is the movies they've rated.
Basically, if you think about a user has a vector of movies.
It's a one if they've rated the movie, a 0 if they haven't.
You could do the same thing with purchase data, as well.
Each of those features,
the feature being I watched Blade Runner,
or I watched the Iron Giant contributes a y
vector to the user's overall preference.
SVDFeature takes this a step further, and it says,
okay, we have features about movies, or items, and about users.
So, maybe this is a book that was published by Tour Publishers,
it was written by a particular author,
it's about dragons and those are all features of the item.
So what SVD in a traditional linear model we'd say, okay,
we're going to learn a weight that says, okay, this user likes dragons or
this user likes Tour Publishers or whatever.
In SVDFeature what we do is we say that,
each of these features contributes a latent feature vector.
So we kind of have two levels of features, we have
the latent feature vector, like we have in matrix factorization.
We do not know what those features mean.
The first feature is, who knows what is actually means it's often very difficult
to interpret what this feature means.
But we have observable features as well,
such as who wrote the book, what the books about.
And what we say is that the observable features
tell us things about the latent features.
And so we can then, we're going to build latent feature vectors.
Okay, this book has these particular latent feature vectors,
this user has this particular latent feature vector.
But what we're also going to do is we're going to push
off a bunch of that latent feature information into the observable features.
So we say okay, words of radiance has some latent features of itself, but it also has
some latent features that we can know about it because it was published by Tour,
because it is 1,200 pages, because it is high fantasy.
And then when we get a new book that comes in, and nobody's rated it yet,
we don't have interactions.
But we know that it's published by Tour, and it's high fantasy and
this one has 300 pages.
We can already start building up some of the latent feature vector for this book.
Let us start getting these latent feature vectors in
a cold start situation a lot more easily.
So the overall model looks like this, where the scorers
come from a baseline scorer, and
then, we have these user features, and we have item features.
And there's a feature value, for that feature for
the item it might be page count or one if it's about dragons,
times a latent feature vector for that feature.
We have the same for the user features.
Then we also can break the bias down into individual feature components and
this breaks down the matrix factorization problem into many different matrices.
Now these features can come from any available data.
We can take any data we have about the items, maybe the fact that this movie is
rated PG, and it's 90 minutes long, and it's animated, and
it has Brent Spiner doing one of the voices.
Whatever features we have available we can use those to derive,
to learn additional latent feature vectors and break down
the factorization process based on the available data and then reassemble it.
There's more techniques that do this.
Generalized probabilistic matrix factorization takes the probabilistic
matrix factorization that we looked at earlier and it integrates additional data.
There's a variety of matrix co-factorization or
collective matrix factorization techniques that process multiple matrices.
You can hear a little bit more about this in a couple of later interviews.
And there's a lot more you can read, we're going to put a few links
particularly to the SVDFeature paper up in the resources.
Explore our Catalog
Join for free and get personalized recommendations, updates and offers.
Coursera provides universal access to the world’s best education,
partnering with top universities and organizations to offer courses online.
© 2019 Coursera Inc. All rights reserved.
