0:00

>> In this module we'll review vectors. I will go over definitions of vectors,

Â what are row vectors, column vectors. We're going to define linear independence,

Â bases, transposes, inner products, lengths of vectors or norms.

Â All of these concepts are going to be needed for doing some of the elementary

Â linear algebra that's needed in this course.

Â What's a vector? It's just a collection of real numbers.

Â You can collect them, and put them in a row or you can put them in a column.

Â So here is an example of a row vector. Where I've put everything as a row here's

Â an example of a column vector and if you notice carefully both of these vectors

Â have n components. So we will call a vector to have n

Â components if it consists of n real numbers.

Â A row vector or column vector and we will denote any of these vectors by the symbol

Â r to the n. R to the n means that every component is a

Â real number and they, there are n of them in the place.

Â To just fix ideas it might be easier to think in terms of R2 which means these are

Â vectors with just two components and it's easiest to think of these vectors as

Â belonging to a plane. And we're used to calling this plane by

Â the x axis and the y axis. So think of x axis being one of the

Â components of the vector. Y axis being another component of the

Â vector. So, let me just give you some examples of

Â vectors that we going to use later on in this module.

Â So, here's one vector. It's x component or the first component is

Â going to be one and the second component is say going to be equal to two.

Â So that the vector will be this vector. At the end of the dot the dot corresponds

Â to a y axis of two and x axis of one. So was labeled this as V.

Â This will simply be the vector one two. The first component equal to one in the

Â second component equal to two. Here's another example.

Â The x axis, or the first component equal to 4 and y axis equal to 1.

Â So that's that point, and I'm going to be connecting it with this vector.

Â And in the rest of this module, I'm going to referring to this vector as w.

Â It has 4 in the first component, and 1 in the second component.

Â By default in this course unless otherwise specified we will assume that all vectors

Â are column vectors, that is all the components have been arranged as a column.

Â So we now know what a vector is, they are collections of real numbers without, by

Â default it's going to be a column vector. Now we want to understand what can we

Â represent using these vectors, what happens with them and so on So the first

Â thing that we want to do is multiply these vectors with real numbers and add them up.

Â So v1 and v2 are vectors. And I want to multiply them by a real

Â number, alpha 1. And a real number, alpha 2.

Â And then get a final vector w. So, to give you an example, here is my

Â vector, v1. One, here is my vector V two.

Â And just to fix ideas, each of these V one and V two actually lives in R to the

Â three. Why R to the three?

Â Every component is a real number, and there are three components.

Â So the three refers to the fact that there are three components.

Â I'm going to multiply them by two real numbers, so two is going to play the role

Â of alpha one. 4 is going to play the role of alpha 2.

Â And what do I do? I take these real numbers, multiply them

Â component by component and add them up. So, in order to get the first component, I

Â take the 2, multiply it to the 1, take the 4, multiply it to the 0.

Â I get 2 times 1 plus 4 times 0 equals 2. That gives me the first component.

Â If I want to go to the second component then I take the 2 and multiply it to the

Â second component 1. Take the 4 multiply it to the second

Â component which is also 1, 2 times 1 is 2, 4 times 1 is 4 add it up together you get

Â 6. Want to go to the third component the same

Â thing, 2 times 0. 4 times z, 1, 2 times 0 is 0.

Â 4 times 1 i 4. You get a total component 4.

Â So when you write a vector w as a combination of vectors v1 and v2, we're

Â going to say that this Vector w is linearly dependent on v1 and v2.

Â Why the linear? Because I'm multiplying by a real number,

Â and adding them up. Other words are, w's a linear combination

Â of v1 and v2. Get another word.

Â W belongs to the linear span of v1 and v2. All of these 3 things mean the same.

Â Linear dependency, linear combination, linear span.

Â Now, we'll, in the next set of ideas, we'll need a concept of linear

Â independence. When can we say that we cannot write a

Â vector as a linear combination of other vectors.

Â To fix ideas, let's again go back to r2, and I'm going to use my favorite two

Â vectors, v and w. So here's my vector v and here's my vector

Â w. For now we don't really need what the

Â components are so I'm not going to bother with that.

Â Now we want to understand what does a linear span mean, what can I, what kind of

Â vectors can I generate by scaling the w? What can I do by multiplying w by areal

Â number? And you can convince yourself very easily

Â That all the vector that you can get, are going to be on that straight line.

Â When you scale it up you get, by a positive number you get up the line if you

Â multiply by a negative number you go down the line.

Â Similarly all the vector that can be gotten as linear combinations of v rely on

Â this straight line. Now we want to ask ourself can I represent

Â v as A multiple of w. Clearly that's not true because I can't, I

Â can't, when I scale the w I get points on this line, v doesn't belong to that line

Â so I can't do anything about that. Similarly, w is not written as a linear

Â combination of v and therefore we'll say that v and w are linearly independent.

Â Let's throw in another vector now, x. It's again a vector in r 2 and I want to

Â ask myself, is this linearly independent of v and w?

Â Is it possible that x can not be written as a linear combination of v and w?

Â Again it's very easy to convince yourself that if you just draw a line parallel to

Â v, you can write, your vector x as a combination of vector that starts from the

Â origin, it's aligned to the vector w and comes up to this point.

Â So therefore, this vector is some alpha 1 times w.

Â This vector here is, is parallel to the vector v.

Â So I can write it at sum scale multiple alpha 2 times v.

Â And because, now, x is the sum of this vector and that vector, you end up getting

Â that x is actually equal to alpha 1 times w, plus alpha 2 times v which means that x

Â is linearly dependent, LD just for short, linearly dependent on v and w.

Â In fact we'll see in the next slide that in R2 if I give you any 2 linearly

Â independent vectors you can write any other vector as a linear combination of

Â these 2. And that set of vectors, say these two, v

Â and w, would actually be called a basis. So that's the next concept that I want to

Â learn about vectors. A basis is a linearly independent set of

Â vectors that spans the entire space. Any basis for Rn, meaning a vector which

Â has n components, has exactly n elements. So basis for Rn has exactly n elements.

Â In the last page I showed you an example of R2, and this should have just two

Â elements. V and W are linearly independent, there

Â were two of them, and therefore I know, that this must be a basis.

Â Now it will turn out that its much easier to think in terms of a standard basis.

Â What's a standard basis is shown here. Its a collection of vectors such that they

Â have only one in one of the components and all of the other components are equal to

Â zero. So the vector E1 will have 1 in the first

Â component, the vector E2 is going to have a 1 in the second component, and the

Â vector En is going to have a 1 in the last, or the nth, component.

Â Now if you give me any vector W which belongs to Rn meaning that it has n

Â components, I can very easily write it as a linear combination of E1 through En.

Â Why, because in each of these vectors is exactly one element that's not zero.

Â So if I want to get the first element of w, right?

Â I have to, say, take it to be w1 times e1, because everything else has zero

Â contribution. If I want the second component, right, I

Â have to take w2 times e2, nnd so on up to wn times en.

Â And therefore, this basis. I can split any vector w as a linear

Â combination very easily. And we will see that, in practice, these

Â terms ought to be very convenient. Again, to put it in perspective.

Â Here's r2. Here is my x asix.

Â Here is my y axis. And when I showed you in the first slide,

Â x axis refers to the first component. And y axis refers to the second component.

Â So the vector e1 is just this one. It has one component one in the x

Â direction and zero in the y direction. E2 is this one, it has a component one in

Â the y direction and zero in every other direction and it's very easy if I take a

Â vector x, all I have to do is drop it down here and This, this length down here on

Â the x-axis is x1, on the y-axis is x2. And it's a very easy way to combine.

Â But any other set of linearly independent vectors, two of them will make sure that

Â this is going to be a basis. So, again, this was v, and this was w from

Â the last page. And v and w is also a basis.

Â This is also a basis. And E1, E2, which is are these 2 special

Â ones, are also a basis. The second basis is special.

Â And we'll just call it the standard basis, because it turns out it's very convienent

Â to work in terms of this basis. Alright.

Â So, so far what do we know? We know what are vectors.

Â We know linear dependence, we know linear independence.

Â We know that if I am a vector in Rn, meaning it has n components, I can find a

Â basis of n linearly independent vectors, such that every vector can be written as a

Â combination of these. So what's the next concept?

Â The next concept that we want to learn, is that of a length of a vector.

Â Vector. So let's start with the basics.

Â And then we'll generalize is to what I want here.

Â So, if I give you r2. And let's take a vector which has the x

Â component equal to, let's say, 4. And the y component equal to, let's say.

Â So this vector, has a representation four, three.

Â Then our high school trigonometry tells us that the length of this vector is nothing

Â but 4 squared plus 3 squared. So I'm taking the x-axis and squaring it.

Â I'm taking the y-axis and squaring it. Square root.

Â So that gives me 16. Plus 9 square root which gives me 25

Â square root equaled to 5. So that's nice.

Â I have a, I have a definition of length. What does this definition of length

Â satisfy? It satisfies several properties.

Â It satisfies that the length of any vector is always great than equal to 0.

Â If the length of a vector is equal to zero, then that vector itself must be

Â zero. So you cannot have a vector which is not

Â equal to zero who's length is equal to zero.

Â And the intuition follows from this triangular expression as well.

Â If I scale a vector by an amount alpha it doesn't matter whether I scale it

Â positively or negatively. The length just gets scaled by the

Â absolute value. So, here's the idea here's the value of

Â vector v if I double it. >> I get this vector if I multiply it by

Â minus 2 I get that vector, the length of this vector and the length of that vector

Â is the same. The direction has changed, but the length

Â remains the same. That's what this absolute value does.

Â The third one here. Tells you the relationship between lengths

Â of additional vectors and the original lengths, and this is known as a triangle

Â inequality. The way you remember this geometrically is

Â that if you've got one vector here, you've got another vector there, the sum of the

Â vectors is this one, the length, this length is always going to be, let's call

Â it l. Three is always going to be lessthan equal

Â to L1 plus L2. This is the basic geometic fact L3 has to

Â be less than equal to L1 plus L2. And this particular fact is encapsulated

Â in this little >> Statement over here, and that's why it's also called the tirangle

Â inequality. Now mathematicians have looked at this

Â concept of length, this particular concept of length that I've been talking about

Â which is take each of the components, square it, add it up, take the square root

Â and now they're calling it the I2 norm. The 2 stands for the fact that I'm

Â squaring it and take the square root and you'll notice that the properties that I

Â want. The fact that the length is greater or

Â equal to zero. The fact that triangle inequality holds.

Â The fact that if I scale things remain the same is true for other definitions of

Â length. Now, since I am taping this in New York,

Â I'm going to try to tell you about a particular notion of language, which is

Â called the L1 norm, otherwise known as the Manhattan distance.

Â I'm sitting at a particular point here, I want to travel to another point, I can

Â only go In blocks north and south or east and west, and I want to figure out how

Â much do I need to walk in order to get from this point to that point.

Â So I have to walk from this point to this point and then walk from that point to

Â that point. So this distance is 4, that distance is 3,

Â so the L1 distance, or the L1 length, or the Manhattan distance is 4 plus 37.

Â The L two distance is five because I can sort of cut across.

Â The L one distance is seven. L one also satisfies all the properties of

Â a norm or a length and therefor it's sometimes convenient to encapsulate all of

Â them as just norms and lengths. For the purposes of this course we will

Â mostly be focusing on L two. But I just wanted you to know, that there

Â are other definitions of length, that are interesting, and sometimes become

Â important in applications. Alright, now we know vectors, we know

Â linear combination, we know lengths. Now we want to go to the next concept,

Â which is that of an angle. And in order to get to an angle, I have to

Â introduce this idea of an inner product or a dot product.

Â So the dot product of any two vectors v and w, so v and w are vectors in Rn.

Â They have n components. So the dot product between v and w is

Â simply going to be taking the ith component of v, taking the ith component

Â of w, multiplying them together, and adding them up.

Â For those of you who are experts in Excel, this is nothing but sum product, take the

Â product of the components and add them up. The sum product function in excel is

Â nothing but an inner product or dot product.

Â If you review back, into the last slide we had defined the L 2 norm there to be every

Â component's squared square root and that can now be written that length L 2 norm of

Â the vector is simply take the vector v, take its dot product.

Â So v, dot product with v, and take the square root.

Â So now we want to understand an angle between these two vectors, so to do that,

Â here's a picture. Here's my w, here's my v, here's an angle

Â theta between them. So in order to understand how the inner

Â product relates to the angle, the length of this vector is The length we, the

Â length of this vector is w. So that inner product of v and w, simply

Â is, take a, take the component of v along w and multiply to the length of w.

Â So if you drop down an orthogonal point over here, this component.

Â Is exactly equal to the norm, or the length, of v cosine of theta, and that I'm

Â going to multiply with the length of w, and that should give me v dot w, which

Â ends up giving me the cosine of theta Is exactly equal to vw divided by the norm of

Â v, and the norm of w, or the length of v, and the length of w.

Â And, to emphasize the fact that these are all 2 norms.

Â I'm just going to put a 2 there. All of that is encapsulated in this slide.

Â Cosine of theta is v.w divided by v and w. And this is true, not just in our 2, but

Â in our n. You an define angles in rn.

Â I'm going to show you in the next module that v dot w is actually a combination of

Â two operations, the transpose operation and the matrix multiplication operation.

Â But that has to wait until we get to the module of matrixes.

Â So that pretty much brings us to the end of the introduction to vectors.

Â This is all we need to do in terms of vectors, and this is all we're going to be

Â using in the course. In the next prerequisite module, the next,

Â the module that comes up next, we're going to review concepts about matrixing

Â