1 00:00:17,099 --> 00:00:24,589 So this is the process by which given a set of linearly independent vectors in a inner 2 00:00:24,589 --> 00:00:25,749 product space 3 00:00:25,749 --> 00:00:43,130 So x is my inner product space and I am given some set of linearly independent vectors So 4 00:00:43,130 --> 00:00:59,969 I have this set s which is a subset of x and s corresponds to say x1 x2 and so on I have 5 00:00:59,969 --> 00:01:05,500 given a subset of vectors in a inner product space This could have finite number of vectors 6 00:01:05,500 --> 00:01:10,140 it could have infinite number of vectors Right now I am not worried about how many vectors 7 00:01:10,140 --> 00:01:15,390 are there in this set They could be finite they could be infinite 8 00:01:15,390 --> 00:01:30,259 All that I know is that this vectors are linearly independent But this is not an orthogonal 9 00:01:30,259 --> 00:01:44,520 set s is not an orthogonal set What is an orthogonal set The vectors are 10 00:01:44,520 --> 00:01:51,630 mutually orthogonal You take any pair of vectors and find a inner product inner product will 11 00:01:51,630 --> 00:01:58,829 be 0 So that is where you have a set to be called as orthogonal set 12 00:01:58,829 --> 00:02:02,679 So this is not an orthogonal set and I would like to generate an orthogonal set because 13 00:02:02,679 --> 00:02:07,759 orthogonal sets are very very useful when you do modeling when you do applied mathematics 14 00:02:07,759 --> 00:02:22,280 numerical computations So how do I do that I start by defining a unit vector 15 00:02:22,280 --> 00:02:37,380 So e1 my e1 is going to be x1 divided by norm x1 okay Well inner product defines a norm 16 00:02:37,380 --> 00:02:53,329 so my norm x1 is nothing but 2 norm is nothing but inner product of x1 itself raise to half 17 00:02:53,329 --> 00:02:58,829 So this is my first vector this is a unit vector I want to create a set starting from 18 00:02:58,829 --> 00:03:06,760 this set I want to create a set which is not just orthogonal but which is ortho normal 19 00:03:06,760 --> 00:03:13,790 okay I want to create unit vectors which are orthogonal to each other okay 20 00:03:13,790 --> 00:03:21,950 So this is my first vector What I do next is well orthogonality allows us to split a 21 00:03:21,950 --> 00:03:30,680 vector into two components One along a direction and one orthogonal to the direction okay That 22 00:03:30,680 --> 00:03:34,120 is the concept which I am going to use in Gram minus Schmidt Process So what is my first 23 00:03:34,120 --> 00:03:46,100 thing First thing is I pick up now this vector x2 here and then I create a vector z2 which 24 00:03:46,100 --> 00:04:00,541 is x2 minus x2e1 inner product of x2e1 So this gives me component of x2 along e1 okay 25 00:04:00,541 --> 00:04:02,840 times e1 26 00:04:02,840 --> 00:04:17,100 You can very easily check that this vector and so if I define another vector say v2 which 27 00:04:17,100 --> 00:04:33,680 is I have two components of vector x2 z2 and v2 okay This is one component x2 minus v2 28 00:04:33,680 --> 00:04:43,350 is another component This is nothing but this is what I am calling as v2 x2 minus v2 and 29 00:04:43,350 --> 00:04:48,120 v2 I am decomposing the vector x2 into two orthogonal components 30 00:04:48,120 --> 00:05:06,460 It is very easy to check that inner product of v2 and z2 = 0 okay Just substitute and 31 00:05:06,460 --> 00:05:13,880 find out We will get inner product to be 0 not at all difficult these two are orthogonal 32 00:05:13,880 --> 00:05:30,860 components I am splitting vector x2 okay So x2 is 33 00:05:30,860 --> 00:05:40,240 okay So now what I do next Well I got two directions which are orthogonal One is e1 34 00:05:40,240 --> 00:05:49,380 and other is z2 See because v2 is just some scalar times e1 right So one direction is 35 00:05:49,380 --> 00:06:08,560 e1 and z2 these are orthogonal to each other and then I define e2 which is z2 by norm z2 36 00:06:08,560 --> 00:06:09,560 okay 37 00:06:09,560 --> 00:06:16,400 So I got two directions e1 here starting from the first vector then I removed the component 38 00:06:16,400 --> 00:06:28,400 along e1 from x2 I created z2 then I just normalized z2 to create e2 okay So now there 39 00:06:28,400 --> 00:06:36,560 are two directions e1 and e2 both of them are unit magnitude This is a unit magnitude 40 00:06:36,560 --> 00:06:45,520 vector right and e1 and e2 are orthogonal okay I just do this by induction So I take 41 00:06:45,520 --> 00:06:55,820 x3 I remove component along e1 and e2 whatever remains I make it unit magnitude I go to x4 42 00:06:55,820 --> 00:07:01,460 I remove from e1 e2 e3 just go on doing this 43 00:07:01,460 --> 00:07:16,680 So this process is called as So my next step would be define z3 which is x3 minus x3 inner 44 00:07:16,680 --> 00:07:43,000 product e1 e1 minus x3 inner product e2 This is my vector z3 and then using z3 I can define 45 00:07:43,000 --> 00:08:01,560 e3 which is z3 by norm z3 and so on okay So given the set of vectors which are not orthogonal 46 00:08:01,560 --> 00:08:10,830 I can just follow this systematic procedure to split to create a set which is ortho normal 47 00:08:10,830 --> 00:08:18,680 okay and creation of this ortho normal set is called as Gram minus Schmidt Process okay 48 00:08:18,680 --> 00:08:24,410 So now let us start today doing something Let us actually look at some examples and 49 00:08:24,410 --> 00:08:34,169 let us create some orthogonal sets starting from some non orthogonal sets 50 00:08:34,169 --> 00:08:35,649 okay 51 00:08:35,649 --> 00:08:51,449 So my first example is going to be in R3 My inner product space is simply R3 and a inner 52 00:08:51,449 --> 00:09:08,040 product between any two vectors is simply x transpose y okay And I am given three vectors 53 00:09:08,040 --> 00:09:21,800 x1 which is 1 1 1 x2 now I want you to do this by hand want to start doing it 1 minus 54 00:09:21,800 --> 00:09:36,800 1 1 and x3 is 1 1 minus 1 Are these linearly independent Are these three directions linearly 55 00:09:36,800 --> 00:09:46,899 independent in R3 These are linearly independent Are they orthogonal They are not orthogonal 56 00:09:46,899 --> 00:09:58,290 You take inner product of any two you will not get 0 So these are not orthogonal directions 57 00:09:58,290 --> 00:10:02,959 I want to construct a orthogonal set starting from this non minus orthogonal set I want 58 00:10:02,959 --> 00:10:15,610 to apply this process So just start doing this what will be e1 E1 will be simply 1 by 59 00:10:15,610 --> 00:10:37,730 root 3 1 1 1 What will be z2 Just calculate 60 00:10:37,730 --> 00:10:56,480 So we have to start with what is inner product of x2e1 what is this quantity X2 is this vector 61 00:10:56,480 --> 00:11:19,209 1 by root 3 So what is this second vector What is z2 Will be 1 minus 1 1 minus 1 by 62 00:11:19,209 --> 00:11:35,300 root 3 times 1 by root 3 1 1 1 So what is this vector 63 00:11:35,300 --> 00:11:39,439 Two third minus four third and two third 64 00:11:39,439 --> 00:11:57,980 So this gives you z2 is you said two third minus four third two third transpose right 65 00:11:57,980 --> 00:12:20,180 okay So this is my z2 So what is e2 You have to help me with this 2 by 3 root 6 two third 66 00:12:20,180 --> 00:12:31,819 minus 4 by 3 2 by 3 This is my e2 Just check whether e1 and e2 are orthogonal What do you 67 00:12:31,819 --> 00:12:50,540 get if you do e1 transpose e2 what do you get Do you get 0 If you do not get 0 you have 68 00:12:50,540 --> 00:13:01,949 made a calculation error You must get a 0 here if you take this inner product 69 00:13:01,949 --> 00:13:11,920 And those who have done this just go ahead to e3 Compute e3 Does this turn out to be 70 00:13:11,920 --> 00:13:21,070 0 It does Just check If you take inner product of this with e1 you should get 0 vector not 71 00:13:21,070 --> 00:13:26,130 0 vector 0 magnitude Inner product should be 0 E1 transpose e2 should be 0 perfectly 72 00:13:26,130 --> 00:13:40,509 0 If you are not getting it there is some error somewhere Is it 0 I do not hear Yes 73 00:13:40,509 --> 00:14:03,059 its 0 okay What about next What about x3 what about z3 1 1 minus 1 What is inner product 74 00:14:03,059 --> 00:14:08,019 of x3 and e1 75 00:14:08,019 --> 00:14:46,639 1 by root 3 into 1 by root 3 1 1 1 then minus what is the inner product here So what is 76 00:14:46,639 --> 00:15:05,480 the number that should appear here inner product E2 with minus 2 by root 6 So this becomes 77 00:15:05,480 --> 00:15:18,430 2 by root 6 Is that fine Because there was Did we have a minus Is this correct Is this 78 00:15:18,430 --> 00:15:29,559 minus correct This is correct What about here this becomes plus Does it become plus This 79 00:15:29,559 --> 00:15:55,499 is fine okay So what is this vector finally Can somebody help me with this 80 00:15:55,499 --> 00:16:10,619 What are these 3 numbers Well after you find this z3 you have to make it indeed magnitude 81 00:16:10,619 --> 00:16:12,559 you have to divide it by its magnitude 82 00:16:12,559 --> 00:16:26,179 But that is a simpler part What will be 83 00:16:26,179 --> 00:16:49,569 Has anyone completed You tell me This could be I am just writing here I am not doing the 84 00:16:49,569 --> 00:16:53,100 calculations I am just writing here Somebody is prompting me So you have to tell me whether 85 00:16:53,100 --> 00:16:58,600 this plus is correct or it should be minus minus here plus is correct here So what is 86 00:16:58,600 --> 00:17:14,610 the total 1 0 minus 1 See our friend says 1 0 minus 1 everyone agrees or there are some 87 00:17:14,610 --> 00:17:20,250 different calculations or 88 00:17:20,250 --> 00:17:28,860 By the way 1 0 minus 1 is this orthogonal to this and this vector It seems to be this 89 00:17:28,860 --> 00:17:34,789 is this 1 0 minus 1 is orthogonal to this vector this direction Forget about the multiplying 90 00:17:34,789 --> 00:17:42,679 factor this direction is orthogonal to this What about here 1 1 1 Yes it is So these are 91 00:17:42,679 --> 00:17:47,179 mutually orthogonal because this and this are orthogonal this and this are orthogonal 92 00:17:47,179 --> 00:17:52,049 this and this are orthogonal So 1 0 minus 1 seems to be I am not sure about the multiplying 93 00:17:52,049 --> 00:17:53,049 factor 94 00:17:53,049 --> 00:18:17,330 Is that correct 1 by root 2 okay z3 is 1 0 minus 1 So e3 becomes 1 by root 2 1 0 minus 95 00:18:17,330 --> 00:18:30,139 1 So I started here with three vectors in R3 okay which were not orthogonal and then 96 00:18:30,139 --> 00:18:38,639 systematically I could construct set of three vectors which are unit magnitude orthogonal 97 00:18:38,639 --> 00:18:42,799 vectors okay 98 00:18:42,799 --> 00:18:52,179 So if you try to you have started with something like this in R3 three vectors which are linearly 99 00:18:52,179 --> 00:19:03,440 independent okay Let us say x1 x2 and x3 starting from this what you have done is you have created 100 00:19:03,440 --> 00:19:20,690 a set which is ortho normal okay 101 00:19:20,690 --> 00:19:26,700 starting from a set which was linearly independent but not orthogonal okay So we had a situation 102 00:19:26,700 --> 00:19:33,789 like this We moved to orthogonal set okay That is what we have done 103 00:19:33,789 --> 00:19:42,220 Now specific vectors that you get here will depend upon how you define the inner product 104 00:19:42,220 --> 00:19:47,789 okay I just wanted to repeat one calculation Well subsequently all the calculations will 105 00:19:47,789 --> 00:19:54,110 change but if I change my definition here of the inner product okay The subsequent calculations 106 00:19:54,110 --> 00:19:59,110 will change The directions may not change in R3 but magnitude calculations can change 107 00:19:59,110 --> 00:20:02,279 And I want to emphasize this one small thing Is this clear 108 00:20:02,279 --> 00:20:07,279 We started with a non orthogonal set and we came up with three directions which are orthogonal 109 00:20:07,279 --> 00:20:15,300 to each other okay So now let me just do one small change here My second example is again 110 00:20:15,300 --> 00:20:23,809 R3 111 00:20:23,809 --> 00:20:33,650 But I am going to change the definition of my inner product to x transpose w y where 112 00:20:33,650 --> 00:20:39,690 w is a symmetric positive definite matrix I am going to pick up one particular matrix 113 00:20:39,690 --> 00:20:50,350 here I am going to pick up this matrix w well there are many ways you can pick up a symmetric 114 00:20:50,350 --> 00:20:56,610 positive matrix Take a matrix which is simply diagonal elements which are positive okay 115 00:20:56,610 --> 00:21:13,990 So that is one way I am going to do with this matrix 2 minus 1 1 Is this a symmetric puzzle 116 00:21:13,990 --> 00:21:17,570 design matrix 117 00:21:17,570 --> 00:21:25,000 This is a symmetric matrix that is for sure Is this puzzle definitely Do you know test 118 00:21:25,000 --> 00:21:35,980 for finding out puzzle design matrix Any other test Principle minors This is greater than 119 00:21:35,980 --> 00:21:43,880 0 2 is > 0 Is this determinant greater than 0 This determinant is greater than 0 What 120 00:21:43,880 --> 00:21:54,320 about this determinant All three put together Calculate is the determine greater than 0 121 00:21:54,320 --> 00:22:04,679 There is simple algebraic test to find whether a matrix is positive definite or not 122 00:22:04,679 --> 00:22:11,270 Look at these matrices constructed by first element and first two cross two matrix then 123 00:22:11,270 --> 00:22:18,159 three cross matrix is it positive Okay it is a simple test to find whether a matrix 124 00:22:18,159 --> 00:22:27,550 is puzzle definite or not So this is a positive definite matrix and x transpose w y will define 125 00:22:27,550 --> 00:22:34,759 a inner product on R3 this inner product is different from what we defined previously 126 00:22:34,759 --> 00:22:48,299 okay So now just remember what is our 2 norm 2 norm is x inner product x raise to half 127 00:22:48,299 --> 00:22:58,389 So in this case it will be x transpose wx raise to half 128 00:22:58,389 --> 00:23:04,900 All their calculations will have to be done using x transpose w so what will be the unit 129 00:23:04,900 --> 00:23:23,720 direction now What is my first vector 130 00:23:23,720 --> 00:23:44,570 What is the very first vector that you what will be e1 E1 is x1 divided by norm x1 What 131 00:23:44,570 --> 00:24:02,570 is norm x1 So you have to work with 1 1 1 transpose this matrix So norm x1 square is 132 00:24:02,570 --> 00:24:24,460 2 minus 1 1 minus 1 2 minus 1 1 minus 1 2 into 1 1 1 So what is this quantity 1 is 2 133 00:24:24,460 --> 00:24:38,750 this multiplication comes to be 2 square is 4 So this comes out to be 4 and then square 134 00:24:38,750 --> 00:24:55,820 root of this 2 okay So what is the first direction the first direction e1 becomes half 1 1 1 135 00:24:55,820 --> 00:24:58,570 this is different from what we got earlier right 136 00:24:58,570 --> 00:25:04,029 Earlier we defined inner product in a different way So the norm which was defined through 137 00:25:04,029 --> 00:25:10,019 the inner product was different and the unit vector was different With this definition 138 00:25:10,019 --> 00:25:16,960 of inner product say this definition of inner product here x transpose w y where w is a 139 00:25:16,960 --> 00:25:27,629 symmetric puzzle definite matrix that induces a 2 norm the 2 norm of 1 1 1 using this definition 140 00:25:27,629 --> 00:25:35,240 of inner product turns out to be 2 2 norm square is 4 right 141 00:25:35,240 --> 00:25:44,080 And unit vector the direction is same but the vector is different right Earlier we got 142 00:25:44,080 --> 00:25:49,126 1 by root 3 1 1 1 okay Now I am getting 1 by 2 So what I want to do is further what 143 00:25:49,126 --> 00:25:55,690 I want to stress here is all further calculations will have to be done using this inner product 144 00:25:55,690 --> 00:26:05,039 Do not forget this okay So in exam if I give you a problem which has a matrix w see earlier 145 00:26:05,039 --> 00:26:11,011 we had a special case w was identity matrix 1 1 1 okay 146 00:26:11,011 --> 00:26:16,100 If I give you a different matrix w you have to keep using that matrix every time you calculate 147 00:26:16,100 --> 00:26:22,789 inner product in that example Because R3 with this inner product is a different inner product 148 00:26:22,789 --> 00:26:30,500 space than what with w=i R3 with w=I and R3 with w = this matrix are two different inner 149 00:26:30,500 --> 00:26:39,620 product spaces okay Calculations will be different very very important Is this clear I am not 150 00:26:39,620 --> 00:26:42,799 doing the further calculations We will move on to some other example okay 151 00:26:42,799 --> 00:26:50,700 Well now I want to graduate from finite dimension spaces to infinite dimension spaces and then 152 00:26:50,700 --> 00:26:59,659 we will see how we start meeting some of the old friends that you have known in your undergraduate 153 00:26:59,659 --> 00:27:15,430 curriculum So now inner product space is any set which satisfies certain axioms right And 154 00:27:15,430 --> 00:27:22,429 we have generalized the concept of inner product space Now I am going to look at set of continuous 155 00:27:22,429 --> 00:27:24,409 functions 156 00:27:24,409 --> 00:27:28,509 My inner product space is going to change my inner product definition is going to change 157 00:27:28,509 --> 00:27:34,690 So my third example and this is where now you have to do some work out okay And you 158 00:27:34,690 --> 00:27:43,779 have to help me on the board and how do we come up with vectors which are unit vectors 159 00:27:43,779 --> 00:27:52,591 okay Then we start developing vectors which are ortho normal We start with a non minus 160 00:27:52,591 --> 00:27:55,650 orthogonal set and develop an orthogonal set same idea okay 161 00:27:55,650 --> 00:28:06,360 Now my x is set of continuous functions over 0 to 1 My inner product space is set of continuous 162 00:28:06,360 --> 00:28:15,970 functions over intervals