Operations with vectors

Loading...

From the course by Imperial College London

Mathematics for Machine Learning: Linear Algebra

2367 ratings

Imperial College London

Mathematics for Machine Learning: Linear Algebra

2367 ratings

Course 1 of 3 in the Specialization Mathematics for Machine Learning

In this course on Linear Algebra we look at what linear algebra is and how it relates to vectors and matrices. Then we look through what vectors and matrices are and how to work with them, including the knotty problem of eigenvalues and eigenvectors, and how to use these to solve problems. Finally we look at how to use these to do fun things with datasets - like how to rotate images of faces and how to extract eigenvectors to look at how the Pagerank algorithm works. Since we're aiming at data-driven applications, we'll be implementing some of these ideas in code, not just on pencil and paper. Towards the end of the course, you'll write code blocks and encounter Jupyter notebooks in Python, but don't worry, these will be quite short, focussed on the concepts, and will guide you through if you’ve not coded before. At the end of this course you will have an intuitive understanding of vectors and matrices that will help you bridge the gap into linear algebra problems, and how to apply these concepts to machine learning.

From the lesson

Introduction to Linear Algebra and to Mathematics for Machine Learning

In this first module we look at how linear algebra is relevant to machine learning and data science. Then we'll wind up the module with an initial introduction to vectors. Throughout, we're focussing on developing your mathematical intuition, not of crunching through algebra or doing long pen-and-paper examples. For many of these operations, there are callable functions in Python that can do the adding up - the point is to appreciate what they do and how they work so that, when things go wrong or there are special cases, you can understand why and what to do.

Operations with vectors11:29

Meet the Instructors

David Dye
Professor of Metallurgy
Department of Materials
Samuel J. Cooper
Lecturer
Dyson School of Design Engineering
A. Freddie Page
Strategic Teaching Fellow
Dyson School of Design Engineering

So, we have these things called vectors like this guy here.

What we want to do first is get an idea of what makes a vector, a vector.

What we'll do in this video is explore the operations we can do with vectors,

the sort of things that we can do with them that define what they are,

and the sort of spaces they can apply to.

So, a vector, we can think of as an object that

moves us about space like this guy here.

This could be a physical space,

or a space of data.

At school, you probably thought of a vector as something

that moved you around a physical space,

but in computer and data science,

we generalise that idea to think of a vector as

maybe just a list of attributes of an object.

So, we might think of a house, say,

so here's a house,

and this might have a number of attributes,

we could say it was 120 square metres in floor area,

it might have two bedrooms,

say, it might have one bathroom,

that would be sort of sensible,

and it might be worth 150,000 Euros, say.

And I could write that down as the vector,

120 square metres, 2 bedrooms,

1 bathroom, and 150,000 Euros.

While in physics, we think of this as being a thing that moves us about space,

in data science, we think of this vector as

being a thing that describes the object of a house.

So, we've generalized the idea of moving about space to

include the description of the attributes of an object.

Now, a vector is just something that are based on two rules.

Firstly, addition, and secondly,

multiplication by a scalar number.

We'll do this first, think of a vector as

just a geometric object starting at the origin, so something like this.

So we get a vector, r, there.

Vector addition is then when we just take another vector,

so let's take another vector like this guy here,

let's call him, s,

and where we put s on the end of r. So then,

that's s, and therefore,

if we put s on the end of r,

we get a sum that's r then going along s. We call that guy r plus s. Now,

we could do this the other way round.

We could do s and then r,

and that would be s plus r. So,

you go along s and along that way,

and that would be s plus r there. s plus r.

And we see that they actually give us the same thing, the same answer.

So, r plus s is equal to s plus r,

so it doesn't matter which way round we do the addition.

So, the other thing we want to be able to do is scalar multiplication,

that is to scale vectors by a number.

So a number a, say,

make it twice as long or half as long, something like that.

So we say that, say 3r was doing r three times, that would be 3r there,

where a was three, or we could do a half r,

which should be something like that.

The only tricky bit is what we mean by minus number,

and by minus r, we mean going back the other way by a whole r. So,

we take r, we go back the other way the same distance,

that would be minus r. So,

in this framework, minus means going back the other way.

At this point, it's convenient to define a coordinate system.

So, let's define space by two vectors.

First, call the first one that takes us from left to right,

and is of unit length, length one.

Let's call that a vector, i.

We'll have another vector here that goes up-down, a vector j,

it's also of unit length, of length one.

And then we'd say,

just use our vector addition rules,

if we wanted a vector r here, something like this,

that was 3, 2,

by which we mean we go 3i's, 1i, 2i,

2i, and then 2j's.

So, we go 3i's plus 2j's and that gives us a vector r

here just from our vector sum.

And what we mean in the 3, 2 is

do 3i's added together,

or scalar multiple of 3i's,

and then do a scalar multiple of 2j's as a vector sum.

And that is what we mean by a coordinate system of defining r as being 3, 2.

So then if I have another vector, s,

let's say s is equal to -1i's and 2j's, that is,

it takes us back 1i and up 2j's, that's s. Then,

r plus s would be that,

we just put s on the end of r,

and then r plus s is going to be therefore that total vector.

That's going to be r plus s. And we can just add up the components, right?

So, r is 3i's and s takes us back 1.

So, it's three plus minus one,

this gives us 2i's,

and in the j's,

r takes us up two and s takes us up another two.

So, that's a total of 4j's.

So we can just add up the components when we're doing vector addition.

So, we can see that because we're doing this component by component,

then vector addition must be what's called associative.

Formally, what this means is that if we have three vectors,

r, s and another one, t,

it doesn't matter whether we add r plus s and then add t,

or whether we add r to s plus t,

it doesn't matter where we put the bracket.

We can do this addition and then that one,

or we can do this addition and then that one.

So, a consequence of it not mattering what order we had,

so s plus r is equals to r plus s,

we can also see that therefore it doesn't matter what order we do the additions and if

we've got three and that's called associativity.

That's formally that definition.

And vector addition, we can see,

when we're adding it up like this, will be associative.

So, I've just got rid of the s's and so on.

So, we can talk about another issue,

which is in a coordinate system,

what do we mean by multiplication by a scalar?

So, if you want to take a multiplication by a scalar, let's say, 2,

then we define this to mean that 2r would be equal to

2 times the components of r. So 2 times 3 for i's,

and 2 times 2 for the j's,

so we've got 2 there multiply by 2,

and that will give us 6, 4.

So, 2r will be doing r, and then doing another r,

that would be 2r,

which should be at the vector 6, 4,

going along 3i's, 4, 5,

6i's, and up 4j's.

Now, you need to think about another question,

which is minus r. So r is this,

minus r is then that,

which will be -3, -2.

So then, we see sort of obviously, kind of, that r,

plus minus r is equal to three plus,

minus three on the i's, and two plus,

minus two on the j's,

which is equal to 0, 0.

So, if we do r and then add minus r,

we end up back at the origin, duh.

And therefore, we've defined what we mean by vector subtraction here.

Vector subtraction it's just addition of

minus one times whatever I'm doing, putting after the minus sign.

So, if we think of another vector,

s, we had s was -1, 2 before, right?

-1i plus 2j's.

So then, r minus s would be this.

So, that's minus s there is equal to go along one on the i's,

and minus two on the j's.

So, r minus s, add up the components,

let's switch to an addition.

So, r minus s is this vector here,

that's r minus s. If we add up the components of that,

it's 3i's plus 1,

three plus one on the i's,

and two plus minus two on the j's,

so that gives us the vector 4,0.

So, if we do r is go along three,

and minus s is go along one,

we've got a total of four.

And if r is go up two,

and minus s is go down two,

we've ended up going up-down zero in total.

So, then we've not only done addition by components,

we've done now what we mean by vector subtraction as well,

as being addition of

a negative one multiple of the thing that we're doing in the minus part.

And that's vector subtraction and addition by components.

So, let's come back to the house example for a moment.

So we said, we had a house, that's my house,

that was 120 square metres,

two bedrooms, one bathroom, and 150,000 Euros.

So, if I put the unit in, that's square metres,

that's its number of beds,

that's its number baths,

and that's its thousands of Euros that it's worth.

So, two houses now is equal to, the vector addition of those things is equal to 2,

and the way we're defining vector addition times 120, 2, 1,

150, which will be equal to 240, 4, 2, 300.

So, we'd say that in the scheme,

the way we're defining it,

then two houses would be 240 square metres,

that would makes sense,

four bedrooms, two bathrooms,

and worth 300,000 Euros,

if I bought two houses identically next to each other.

And that would be a scalar multiple or an addition of one house to another.

One house plus one house,

so we could keep on doing that with three houses,

or differently shaped houses,

or whatever it was, or negative houses.

The way we've defined vectors,

that will still apply to these objects of houses.

So, that's vectors.

We've defined two fundamental operations that vectors satisfy, that is addition,

so like r plus s here,

a multiplication by a scalar,

so like 2r here and minus s here.

And we've explored the properties that those

imply like associativity of addition and subtraction,

what subtraction really means of vectors r plus minus s,

being r minus s. And we've noticed that it can be

useful to define a coordinate system in which to do our addition and scaling,

so like r 3,2 here,

using these fundamental basis vectors.

These things that define the space, i and j,

which we call the basis vectors or the things that define the coordinate system.

We've also seen that although, perhaps,

it's easiest to think of vector operations geometrically,

we don't have to do it in a real space.

We could do it with vectors that a

data science lists of different types of things like the attributes of a house.

So, that's vectors, that's all the fundamental operations.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play