1
00:00:17,099 --> 00:00:24,589
So this is the process by which given a set
of linearly independent vectors in a inner
2
00:00:24,589 --> 00:00:25,749
product space
3
00:00:25,749 --> 00:00:43,130
So x is my inner product space and I am given
some set of linearly independent vectors So
4
00:00:43,130 --> 00:00:59,969
I have this set s which is a subset of x and
s corresponds to say x1 x2 and so on I have
5
00:00:59,969 --> 00:01:05,500
given a subset of vectors in a inner product
space This could have finite number of vectors
6
00:01:05,500 --> 00:01:10,140
it could have infinite number of vectors Right
now I am not worried about how many vectors
7
00:01:10,140 --> 00:01:15,390
are there in this set They could be finite
they could be infinite
8
00:01:15,390 --> 00:01:30,259
All that I know is that this vectors are linearly
independent But this is not an orthogonal
9
00:01:30,259 --> 00:01:44,520
set s is not an orthogonal set
What is an orthogonal set The vectors are
10
00:01:44,520 --> 00:01:51,630
mutually orthogonal You take any pair of vectors
and find a inner product inner product will
11
00:01:51,630 --> 00:01:58,829
be 0 So that is where you have a set to be
called as orthogonal set
12
00:01:58,829 --> 00:02:02,679
So this is not an orthogonal set and I would
like to generate an orthogonal set because
13
00:02:02,679 --> 00:02:07,759
orthogonal sets are very very useful when
you do modeling when you do applied mathematics
14
00:02:07,759 --> 00:02:22,280
numerical computations So how do I do that
I start by defining a unit vector
15
00:02:22,280 --> 00:02:37,380
So e1 my e1 is going to be x1 divided by norm
x1 okay Well inner product defines a norm
16
00:02:37,380 --> 00:02:53,329
so my norm x1 is nothing but 2 norm is nothing
but inner product of x1 itself raise to half
17
00:02:53,329 --> 00:02:58,829
So this is my first vector this is a unit
vector I want to create a set starting from
18
00:02:58,829 --> 00:03:06,760
this set I want to create a set which is not
just orthogonal but which is ortho normal
19
00:03:06,760 --> 00:03:13,790
okay I want to create unit vectors which are
orthogonal to each other okay
20
00:03:13,790 --> 00:03:21,950
So this is my first vector What I do next
is well orthogonality allows us to split a
21
00:03:21,950 --> 00:03:30,680
vector into two components One along a direction
and one orthogonal to the direction okay That
22
00:03:30,680 --> 00:03:34,120
is the concept which I am going to use in
Gram minus Schmidt Process So what is my first
23
00:03:34,120 --> 00:03:46,100
thing First thing is I pick up now this vector
x2 here and then I create a vector z2 which
24
00:03:46,100 --> 00:04:00,541
is x2 minus x2e1 inner product of x2e1 So
this gives me component of x2 along e1 okay
25
00:04:00,541 --> 00:04:02,840
times e1
26
00:04:02,840 --> 00:04:17,100
You can very easily check that this vector
and so if I define another vector say v2 which
27
00:04:17,100 --> 00:04:33,680
is I have two components of vector x2 z2 and
v2 okay This is one component x2 minus v2
28
00:04:33,680 --> 00:04:43,350
is another component This is nothing but this
is what I am calling as v2 x2 minus v2 and
29
00:04:43,350 --> 00:04:48,120
v2 I am decomposing the vector x2 into two
orthogonal components
30
00:04:48,120 --> 00:05:06,460
It is very easy to check that inner product
of v2 and z2 = 0 okay Just substitute and
31
00:05:06,460 --> 00:05:13,880
find out We will get inner product to be 0
not at all difficult these two are orthogonal
32
00:05:13,880 --> 00:05:30,860
components I am splitting vector x2 okay So
x2 is
33
00:05:30,860 --> 00:05:40,240
okay So now what I do next Well I got two
directions which are orthogonal One is e1
34
00:05:40,240 --> 00:05:49,380
and other is z2 See because v2 is just some
scalar times e1 right So one direction is
35
00:05:49,380 --> 00:06:08,560
e1 and z2 these are orthogonal to each other
and then I define e2 which is z2 by norm z2
36
00:06:08,560 --> 00:06:09,560
okay
37
00:06:09,560 --> 00:06:16,400
So I got two directions e1 here starting from
the first vector then I removed the component
38
00:06:16,400 --> 00:06:28,400
along e1 from x2 I created z2 then I just
normalized z2 to create e2 okay So now there
39
00:06:28,400 --> 00:06:36,560
are two directions e1 and e2 both of them
are unit magnitude This is a unit magnitude
40
00:06:36,560 --> 00:06:45,520
vector right and e1 and e2 are orthogonal
okay I just do this by induction So I take
41
00:06:45,520 --> 00:06:55,820
x3 I remove component along e1 and e2 whatever
remains I make it unit magnitude I go to x4
42
00:06:55,820 --> 00:07:01,460
I remove from e1 e2 e3 just go on doing this
43
00:07:01,460 --> 00:07:16,680
So this process is called as So my next step
would be define z3 which is x3 minus x3 inner
44
00:07:16,680 --> 00:07:43,000
product e1 e1 minus x3 inner product e2 This
is my vector z3 and then using z3 I can define
45
00:07:43,000 --> 00:08:01,560
e3 which is z3 by norm z3 and so on okay So
given the set of vectors which are not orthogonal
46
00:08:01,560 --> 00:08:10,830
I can just follow this systematic procedure
to split to create a set which is ortho normal
47
00:08:10,830 --> 00:08:18,680
okay and creation of this ortho normal set
is called as Gram minus Schmidt Process okay
48
00:08:18,680 --> 00:08:24,410
So now let us start today doing something
Let us actually look at some examples and
49
00:08:24,410 --> 00:08:34,169
let us create some orthogonal sets starting
from some non orthogonal sets
50
00:08:34,169 --> 00:08:35,649
okay
51
00:08:35,649 --> 00:08:51,449
So my first example is going to be in R3 My
inner product space is simply R3 and a inner
52
00:08:51,449 --> 00:09:08,040
product between any two vectors is simply
x transpose y okay And I am given three vectors
53
00:09:08,040 --> 00:09:21,800
x1 which is 1 1 1 x2 now I want you to do
this by hand want to start doing it 1 minus
54
00:09:21,800 --> 00:09:36,800
1 1 and x3 is 1 1 minus 1 Are these linearly
independent Are these three directions linearly
55
00:09:36,800 --> 00:09:46,899
independent in R3 These are linearly independent
Are they orthogonal They are not orthogonal
56
00:09:46,899 --> 00:09:58,290
You take inner product of any two you will
not get 0 So these are not orthogonal directions
57
00:09:58,290 --> 00:10:02,959
I want to construct a orthogonal set starting
from this non minus orthogonal set I want
58
00:10:02,959 --> 00:10:15,610
to apply this process So just start doing
this what will be e1 E1 will be simply 1 by
59
00:10:15,610 --> 00:10:37,730
root 3 1 1 1 What will be z2 Just calculate
60
00:10:37,730 --> 00:10:56,480
So we have to start with what is inner product
of x2e1 what is this quantity X2 is this vector
61
00:10:56,480 --> 00:11:19,209
1 by root 3 So what is this second vector
What is z2 Will be 1 minus 1 1 minus 1 by
62
00:11:19,209 --> 00:11:35,300
root 3 times 1 by root 3 1 1 1 So what is
this vector
63
00:11:35,300 --> 00:11:39,439
Two third minus four third and two third
64
00:11:39,439 --> 00:11:57,980
So this gives you z2 is you said two third
minus four third two third transpose right
65
00:11:57,980 --> 00:12:20,180
okay So this is my z2 So what is e2 You have
to help me with this 2 by 3 root 6 two third
66
00:12:20,180 --> 00:12:31,819
minus 4 by 3 2 by 3 This is my e2 Just check
whether e1 and e2 are orthogonal What do you
67
00:12:31,819 --> 00:12:50,540
get if you do e1 transpose e2 what do you
get Do you get 0 If you do not get 0 you have
68
00:12:50,540 --> 00:13:01,949
made a calculation error You must get a 0
here if you take this inner product
69
00:13:01,949 --> 00:13:11,920
And those who have done this just go ahead
to e3 Compute e3 Does this turn out to be
70
00:13:11,920 --> 00:13:21,070
0 It does Just check If you take inner product
of this with e1 you should get 0 vector not
71
00:13:21,070 --> 00:13:26,130
0 vector 0 magnitude Inner product should
be 0 E1 transpose e2 should be 0 perfectly
72
00:13:26,130 --> 00:13:40,509
0 If you are not getting it there is some
error somewhere Is it 0 I do not hear Yes
73
00:13:40,509 --> 00:14:03,059
its 0 okay What about next What about x3 what
about z3 1 1 minus 1 What is inner product
74
00:14:03,059 --> 00:14:08,019
of x3 and e1
75
00:14:08,019 --> 00:14:46,639
1 by root 3 into 1 by root 3 1 1 1 then minus
what is the inner product here So what is
76
00:14:46,639 --> 00:15:05,480
the number that should appear here inner product
E2 with minus 2 by root 6 So this becomes
77
00:15:05,480 --> 00:15:18,430
2 by root 6 Is that fine Because there was
Did we have a minus Is this correct Is this
78
00:15:18,430 --> 00:15:29,559
minus correct This is correct What about here
this becomes plus Does it become plus This
79
00:15:29,559 --> 00:15:55,499
is fine okay So what is this vector finally
Can somebody help me with this
80
00:15:55,499 --> 00:16:10,619
What are these 3 numbers Well after you find
this z3 you have to make it indeed magnitude
81
00:16:10,619 --> 00:16:12,559
you have to divide it by its magnitude
82
00:16:12,559 --> 00:16:26,179
But that is a simpler part What will be
83
00:16:26,179 --> 00:16:49,569
Has anyone completed You tell me This could
be I am just writing here I am not doing the
84
00:16:49,569 --> 00:16:53,100
calculations I am just writing here Somebody
is prompting me So you have to tell me whether
85
00:16:53,100 --> 00:16:58,600
this plus is correct or it should be minus
minus here plus is correct here So what is
86
00:16:58,600 --> 00:17:14,610
the total 1 0 minus 1 See our friend says
1 0 minus 1 everyone agrees or there are some
87
00:17:14,610 --> 00:17:20,250
different calculations or
88
00:17:20,250 --> 00:17:28,860
By the way 1 0 minus 1 is this orthogonal
to this and this vector It seems to be this
89
00:17:28,860 --> 00:17:34,789
is this 1 0 minus 1 is orthogonal to this
vector this direction Forget about the multiplying
90
00:17:34,789 --> 00:17:42,679
factor this direction is orthogonal to this
What about here 1 1 1 Yes it is So these are
91
00:17:42,679 --> 00:17:47,179
mutually orthogonal because this and this
are orthogonal this and this are orthogonal
92
00:17:47,179 --> 00:17:52,049
this and this are orthogonal So 1 0 minus
1 seems to be I am not sure about the multiplying
93
00:17:52,049 --> 00:17:53,049
factor
94
00:17:53,049 --> 00:18:17,330
Is that correct 1 by root 2 okay z3 is 1 0
minus 1 So e3 becomes 1 by root 2 1 0 minus
95
00:18:17,330 --> 00:18:30,139
1 So I started here with three vectors in
R3 okay which were not orthogonal and then
96
00:18:30,139 --> 00:18:38,639
systematically I could construct set of three
vectors which are unit magnitude orthogonal
97
00:18:38,639 --> 00:18:42,799
vectors okay
98
00:18:42,799 --> 00:18:52,179
So if you try to you have started with something
like this in R3 three vectors which are linearly
99
00:18:52,179 --> 00:19:03,440
independent okay Let us say x1 x2 and x3 starting
from this what you have done is you have created
100
00:19:03,440 --> 00:19:20,690
a set
which is ortho normal okay
101
00:19:20,690 --> 00:19:26,700
starting from a set which was linearly independent
but not orthogonal okay So we had a situation
102
00:19:26,700 --> 00:19:33,789
like this We moved to orthogonal set okay
That is what we have done
103
00:19:33,789 --> 00:19:42,220
Now specific vectors that you get here will
depend upon how you define the inner product
104
00:19:42,220 --> 00:19:47,789
okay I just wanted to repeat one calculation
Well subsequently all the calculations will
105
00:19:47,789 --> 00:19:54,110
change but if I change my definition here
of the inner product okay The subsequent calculations
106
00:19:54,110 --> 00:19:59,110
will change The directions may not change
in R3 but magnitude calculations can change
107
00:19:59,110 --> 00:20:02,279
And I want to emphasize this one small thing
Is this clear
108
00:20:02,279 --> 00:20:07,279
We started with a non orthogonal set and we
came up with three directions which are orthogonal
109
00:20:07,279 --> 00:20:15,300
to each other okay So now let me just do one
small change here My second example is again
110
00:20:15,300 --> 00:20:23,809
R3
111
00:20:23,809 --> 00:20:33,650
But I am going to change the definition of
my inner product to x transpose w y where
112
00:20:33,650 --> 00:20:39,690
w is a symmetric positive definite matrix
I am going to pick up one particular matrix
113
00:20:39,690 --> 00:20:50,350
here I am going to pick up this matrix w well
there are many ways you can pick up a symmetric
114
00:20:50,350 --> 00:20:56,610
positive matrix Take a matrix which is simply
diagonal elements which are positive okay
115
00:20:56,610 --> 00:21:13,990
So that is one way I am going to do with this
matrix 2 minus 1 1 Is this a symmetric puzzle
116
00:21:13,990 --> 00:21:17,570
design matrix
117
00:21:17,570 --> 00:21:25,000
This is a symmetric matrix that is for sure
Is this puzzle definitely Do you know test
118
00:21:25,000 --> 00:21:35,980
for finding out puzzle design matrix Any other
test Principle minors This is greater than
119
00:21:35,980 --> 00:21:43,880
0 2 is > 0 Is this determinant greater than
0 This determinant is greater than 0 What
120
00:21:43,880 --> 00:21:54,320
about this determinant All three put together
Calculate is the determine greater than 0
121
00:21:54,320 --> 00:22:04,679
There is simple algebraic test to find whether
a matrix is positive definite or not
122
00:22:04,679 --> 00:22:11,270
Look at these matrices constructed by first
element and first two cross two matrix then
123
00:22:11,270 --> 00:22:18,159
three cross matrix is it positive Okay it
is a simple test to find whether a matrix
124
00:22:18,159 --> 00:22:27,550
is puzzle definite or not So this is a positive
definite matrix and x transpose w y will define
125
00:22:27,550 --> 00:22:34,759
a inner product on R3 this inner product is
different from what we defined previously
126
00:22:34,759 --> 00:22:48,299
okay So now just remember what is our 2 norm
2 norm is x inner product x raise to half
127
00:22:48,299 --> 00:22:58,389
So in this case it will be x transpose wx
raise to half
128
00:22:58,389 --> 00:23:04,900
All their calculations will have to be done
using x transpose w so what will be the unit
129
00:23:04,900 --> 00:23:23,720
direction now What is my first vector
130
00:23:23,720 --> 00:23:44,570
What is the very first vector that you what
will be e1 E1 is x1 divided by norm x1 What
131
00:23:44,570 --> 00:24:02,570
is norm x1 So you have to work with 1 1 1
transpose this matrix So norm x1 square is
132
00:24:02,570 --> 00:24:24,460
2 minus 1 1 minus 1 2 minus 1 1 minus 1 2
into 1 1 1 So what is this quantity 1 is 2
133
00:24:24,460 --> 00:24:38,750
this multiplication comes to be 2 square is
4 So this comes out to be 4 and then square
134
00:24:38,750 --> 00:24:55,820
root of this 2 okay So what is the first direction
the first direction e1 becomes half 1 1 1
135
00:24:55,820 --> 00:24:58,570
this is different from what we got earlier
right
136
00:24:58,570 --> 00:25:04,029
Earlier we defined inner product in a different
way So the norm which was defined through
137
00:25:04,029 --> 00:25:10,019
the inner product was different and the unit
vector was different With this definition
138
00:25:10,019 --> 00:25:16,960
of inner product say this definition of inner
product here x transpose w y where w is a
139
00:25:16,960 --> 00:25:27,629
symmetric puzzle definite matrix that induces
a 2 norm the 2 norm of 1 1 1 using this definition
140
00:25:27,629 --> 00:25:35,240
of inner product turns out to be 2 2 norm
square is 4 right
141
00:25:35,240 --> 00:25:44,080
And unit vector the direction is same but
the vector is different right Earlier we got
142
00:25:44,080 --> 00:25:49,126
1 by root 3 1 1 1 okay Now I am getting 1
by 2 So what I want to do is further what
143
00:25:49,126 --> 00:25:55,690
I want to stress here is all further calculations
will have to be done using this inner product
144
00:25:55,690 --> 00:26:05,039
Do not forget this okay So in exam if I give
you a problem which has a matrix w see earlier
145
00:26:05,039 --> 00:26:11,011
we had a special case w was identity matrix
1 1 1 okay
146
00:26:11,011 --> 00:26:16,100
If I give you a different matrix w you have
to keep using that matrix every time you calculate
147
00:26:16,100 --> 00:26:22,789
inner product in that example Because R3 with
this inner product is a different inner product
148
00:26:22,789 --> 00:26:30,500
space than what with w=i R3 with w=I and R3
with w = this matrix are two different inner
149
00:26:30,500 --> 00:26:39,620
product spaces okay Calculations will be different
very very important Is this clear I am not
150
00:26:39,620 --> 00:26:42,799
doing the further calculations We will move
on to some other example okay
151
00:26:42,799 --> 00:26:50,700
Well now I want to graduate from finite dimension
spaces to infinite dimension spaces and then
152
00:26:50,700 --> 00:26:59,659
we will see how we start meeting some of the
old friends that you have known in your undergraduate
153
00:26:59,659 --> 00:27:15,430
curriculum So now inner product space is any
set which satisfies certain axioms right And
154
00:27:15,430 --> 00:27:22,429
we have generalized the concept of inner product
space Now I am going to look at set of continuous
155
00:27:22,429 --> 00:27:24,409
functions
156
00:27:24,409 --> 00:27:28,509
My inner product space is going to change
my inner product definition is going to change
157
00:27:28,509 --> 00:27:34,690
So my third example and this is where now
you have to do some work out okay And you
158
00:27:34,690 --> 00:27:43,779
have to help me on the board and how do we
come up with vectors which are unit vectors
159
00:27:43,779 --> 00:27:52,591
okay Then we start developing vectors which
are ortho normal We start with a non minus
160
00:27:52,591 --> 00:27:55,650
orthogonal set and develop an orthogonal set
same idea okay
161
00:27:55,650 --> 00:28:06,360
Now my x is set of continuous functions over
0 to 1 My inner product space is set of continuous
162
00:28:06,360 --> 00:28:15,970
functions over intervals