1
00:00:17,060 --> 00:00:27,160
so we have been looking at this method of
optimization numerical optimization to solve
2
00:00:27,160 --> 00:00:33,070
linear algebraic equations and what we derived
in the last class was the gradient method
3
00:00:33,070 --> 00:00:45,010
or the cauchy method the gradient method does
a search iterative search in the negative
4
00:00:45,010 --> 00:00:47,370
of the gradient direction
5
00:00:47,370 --> 00:01:02,140
and if i summarize this algorithm this was
so we are trying to solve a x=b and we have
6
00:01:02,140 --> 00:01:22,759
this we are assuming that a is symmetric or
a=a transpose and a is well in applied mathematics
7
00:01:22,759 --> 00:01:30,399
or in linear algebra you often write that
a is positive definite by saying a>0 okay
8
00:01:30,399 --> 00:01:44,680
that means a is positive definite that does
not mean all elements of a are >0 when you
9
00:01:44,680 --> 00:01:49,320
write a>0 it means a is positive definite
a is asymmetric
10
00:01:49,320 --> 00:01:54,450
and then we formulated an objective function
well we know what to do if a is not symmetric
11
00:01:54,450 --> 00:02:00,189
positive definitely we pre-multiply both sides
by a transpose and then we get we work with
12
00:02:00,189 --> 00:02:10,899
the modified problem so phi is 1/2 x transpose
a x-x transpose b we want to minimize this
13
00:02:10,899 --> 00:02:23,250
with respect to x phi and then we use this
gradient method to come up with the search
14
00:02:23,250 --> 00:02:33,740
direction so what was this gradient method?
the gradient method was we find the gradient
15
00:02:33,740 --> 00:02:40,010
direction
16
00:02:40,010 --> 00:02:51,060
so dou phi/dou=a x-b this is the gradient
and we want to move in the negative of the
17
00:02:51,060 --> 00:03:08,260
gradient so we take g k=-a x k-b this is my
negative of the gradient direction and then
18
00:03:08,260 --> 00:03:16,670
we move how much do we move this time we saw
a one-dimensional optimization problem and
19
00:03:16,670 --> 00:03:32,489
then we move actually x k+1=x k+ lambda k
g k and we found that lambda k is nothing
20
00:03:32,489 --> 00:03:47,250
but b transpose g k okay so we make an optimal
move every time in the negative of the gradient
21
00:03:47,250 --> 00:03:48,250
direction
22
00:03:48,250 --> 00:03:52,310
this is negative of the gradient direction
lambda k times g k that is the move that we
23
00:03:52,310 --> 00:04:01,480
make every time and this is the classical
gradient method or the cauchy method and then
24
00:04:01,480 --> 00:04:14,379
these iterations are stopped when you reach
a point or gradient is not changing either
25
00:04:14,379 --> 00:04:19,519
way of them is fine x k is not changing the
new iterations are not adding too much information
26
00:04:19,519 --> 00:04:25,500
we specified an epsilon epsilon could be something
like 10 to the power-8 10 to the power-10
27
00:04:25,500 --> 00:04:27,890
a very small number okay
28
00:04:27,890 --> 00:04:35,120
and we say that the relative change in x as
the new iterates are generated this is insignificant
29
00:04:35,120 --> 00:04:40,890
that is when we stop okay so this is iterative
scheme and if you remember if you see here
30
00:04:40,890 --> 00:04:57,420
every time you need to compute a x k-b k you
need to compute a x k-b this is the residual
31
00:04:57,420 --> 00:05:06,280
actually when you solve this this g k will
be 0 gradient direction is 0 which is the
32
00:05:06,280 --> 00:05:10,889
necessary condition for optimality for this
particular objective function okay
33
00:05:10,889 --> 00:05:19,710
so finding the optimum is equivalent to reaching
the or meeting the necessary conditions for
34
00:05:19,710 --> 00:05:29,680
optimality and
we satisfy or we terminate based on even you
35
00:05:29,680 --> 00:05:35,490
can terminate based on g k if g k is very
very small okay the gradient vector is very
36
00:05:35,490 --> 00:05:41,300
very small now this slight modification of
this because this the problem with this method
37
00:05:41,300 --> 00:05:46,580
it works very well when you are far away from
the optimum when you come close to the optimum
38
00:05:46,580 --> 00:05:49,440
this tends to become very very slow okay
39
00:05:49,440 --> 00:05:53,449
the moves that you make every time when you
come close to the optimum are quite small
40
00:05:53,449 --> 00:05:59,319
so that trouble is this we need to do something
to accelerate the convergence when you come
41
00:05:59,319 --> 00:06:06,449
to closer to the optimum okay now what i am
saying is not just true for this particular
42
00:06:06,449 --> 00:06:11,210
problem it is in general to that gradient
method is one of the very basic methods for
43
00:06:11,210 --> 00:06:17,190
doing numerical search or numerical optimization
but the problem with gradient method is that
44
00:06:17,190 --> 00:06:18,870
it works very well when you are away from
the optimum
45
00:06:18,870 --> 00:06:23,240
and when you come very close to the optimum
it becomes very very slow so convergence to
46
00:06:23,240 --> 00:06:28,599
the optimum is slow okay and we need to do
something about it okay
47
00:06:28,599 --> 00:06:40,690
so there is a variant which is called as conjugate
gradient method okay so what we do is we move
48
00:06:40,690 --> 00:06:55,629
in the conjugate directions a conjugate directions
okay so given a matrix a okay a is our matrix
49
00:06:55,629 --> 00:07:13,490
we call see a set of directions s 0 s 1 let
us say that s 1 s 2 s 3 are directions in
50
00:07:13,490 --> 00:07:21,910
which we want to move okay these directions
are called as a conjugate if this condition
51
00:07:21,910 --> 00:07:37,300
is satisfied if you take 2 directions k and
k-1 not any 2 k and k-1 okay say new direction
52
00:07:37,300 --> 00:07:39,659
and previous direction okay
53
00:07:39,659 --> 00:07:48,050
and if this in a product is 0 okay or if this
whatever you want to call it dot product is
54
00:07:48,050 --> 00:07:53,860
this 0 which is weighted by a matrix okay
then these directions are called as a conjugate
55
00:07:53,860 --> 00:07:59,770
directions okay and what you can show is that
instead of moving along the gradient direction
56
00:07:59,770 --> 00:08:06,060
if you move along the a conjugate directions
then you reach the optimum very very fast
57
00:08:06,060 --> 00:08:10,919
okay if you have the notes there is a small
correction it is on equation 129
58
00:08:10,919 --> 00:08:21,651
so how do you generate this a conjugate directions
okay we are going to use this formula s k=beta
59
00:08:21,651 --> 00:08:39,030
k
s k-1+ so we are going to use the gradient
60
00:08:39,030 --> 00:08:44,500
direction it is not that i am not going to
use gradient direction okay it is the conjugate
61
00:08:44,500 --> 00:08:51,160
gradient algorithm so i am going to use the
gradient vector current gradient vector but
62
00:08:51,160 --> 00:08:59,070
to that i am going to add some correction
which is based on the previous this is based
63
00:08:59,070 --> 00:09:04,680
on the previous you know previous direction
okay
64
00:09:04,680 --> 00:09:17,340
and i kick off this algorithm with s 0= so
what means sorry s 0=0 which means s 1 the
65
00:09:17,340 --> 00:09:24,750
first move that i am going to make okay the
first move that i am going to make is g k
66
00:09:24,750 --> 00:09:35,880
g 1 okay the first gradient let us say i am
starting my iterations at 1 i am taking s
67
00:09:35,880 --> 00:09:42,339
0 to be 0 so the first direction is the gradient
direction what will be the subsequent direction
68
00:09:42,339 --> 00:09:49,630
well how do i choose this beta k and all that
will become clear soon
69
00:09:49,630 --> 00:10:06,190
what is s 2? s 2 will be beta 2 s 1+ g 2 that
means gradient at point 2 negative of the
70
00:10:06,190 --> 00:10:10,980
gradient at the point 2 sorry g here is negative
of the gradient we are going to move in the
71
00:10:10,980 --> 00:10:14,800
negative of the gradient we are minimizing
not the gradient direction g is the negative
72
00:10:14,800 --> 00:10:36,421
of the gradient direction okay so g 1 is nothing
but -of a x 1-b okay and so on okay i cannot
73
00:10:36,421 --> 00:10:47,760
just go on doing this so you can see see what
is this? this is beta 2 g 1+g 2 right
74
00:10:47,760 --> 00:10:54,800
so i am taking the linear combination of the
2 gradient directions not just negative of
75
00:10:54,800 --> 00:11:01,920
the current gradient direction but i am taking
linear combination and so on so this every
76
00:11:01,920 --> 00:11:13,079
iteration i am going to do this okay now i
am going to invoke this a conjugacy so what
77
00:11:13,079 --> 00:11:29,560
i am going to do now is to take s k transpose
and pre-multiply it by a s k-1 okay so what
78
00:11:29,560 --> 00:11:57,600
is this beta k s k -1+g k transport a s k-1
right now these are a conjugate directions
79
00:11:57,600 --> 00:12:07,209
what should be this? this should be 0 so since
these are a conjugate this left hand side
80
00:12:07,209 --> 00:12:24,540
is 0 okay so i get what do i get from here
0=beta k s k-1 a s k s k -1 transpose a s
81
00:12:24,540 --> 00:12:35,670
k see s k-1 let us put it in the bracket become
easier to understand this is s k -1 transpose
82
00:12:35,670 --> 00:13:01,930
a s k-1 okay +g k transpose a s k -1 this
see what is g k is the negative of the current
83
00:13:01,930 --> 00:13:09,899
gradient direction g k is known to me okay
s k -1 is the previous direction okay so using
84
00:13:09,899 --> 00:13:14,100
this i can find out beta k okay
85
00:13:14,100 --> 00:13:23,160
so that gives me a formula for beta k how
much to move what is the linear combination
86
00:13:23,160 --> 00:13:37,670
that should be made okay for a conjugate directions
so now and what is this if i write g k to
87
00:13:37,670 --> 00:13:49,360
be -of a x k-b that completes the formula
so i have this g k which is given by this
88
00:13:49,360 --> 00:13:57,310
this is how you kick off your iterations and
this is how you compute beta k okay this is
89
00:13:57,310 --> 00:14:18,330
how you combine okay so actually i can combine
this and okay i can combined the 2 and i can
90
00:14:18,330 --> 00:14:21,350
find out this as my iteration scheme
91
00:14:21,350 --> 00:14:30,829
this is a scalar remember this is a scalar
ratio of 2 scalars this is a vector direction
92
00:14:30,829 --> 00:14:37,930
and this is current negative of the gradient
direction so this is how you combine and see
93
00:14:37,930 --> 00:14:48,389
how beta k changes with little bit of algebraic
jugglery what you can show is that i am skipping
94
00:14:48,389 --> 00:14:55,110
this i have given the derivation here you
can just go over it it is that s k with some
95
00:14:55,110 --> 00:15:06,709
algebraic manipulation can be shown to be
beta and -beta 1 well if you do a little bit
96
00:15:06,709 --> 00:15:09,089
of index jugglery
97
00:15:09,089 --> 00:15:16,920
if i just come back here i will do a little
bit of index jugglery i will call this -1
98
00:15:16,920 --> 00:15:41,519
s -1=0 okay so if s -1=0 then s 0 becomes
g 0 which is-same as -a x 0-b okay if you
99
00:15:41,519 --> 00:15:47,550
do a little bit of index jugglery you called
0 as the starting point -1 previous to the
100
00:15:47,550 --> 00:15:54,209
starting point okay because if you put k=0
whether it will start from k=1 or k=0 it is
101
00:15:54,209 --> 00:16:01,690
a matter of just writing the algorithm to
be consistent with other notation here i will
102
00:16:01,690 --> 00:16:10,540
just so g 0 is the negative of the initial
gradient direction
103
00:16:10,540 --> 00:16:16,860
so what you can show with the little bit of
jugglery is that so in this case you find
104
00:16:16,860 --> 00:16:23,300
the current direction in which to move using
a linear combination of all the previous gradient
105
00:16:23,300 --> 00:16:27,509
directions you do not use only one current
gradient direction you use a linear combination
106
00:16:27,509 --> 00:16:32,630
of all previous gradient directions okay now
this is the direction in which you want to
107
00:16:32,630 --> 00:16:37,540
move how much do you want to move? you have
to use this lambda business okay
108
00:16:37,540 --> 00:16:48,459
so after we have decided this s k what we
do is okay so my new point that is x k+1=x
109
00:16:48,459 --> 00:17:04,310
k+ lambda s k is only the direction just number
okay and then i find out so lambda k=min with
110
00:17:04,310 --> 00:17:22,900
respect to lambda phi of x k+ lambda s k this
is =min with respect to lambda okay and then
111
00:17:22,900 --> 00:17:36,080
this can be solved very very easily
112
00:17:36,080 --> 00:17:44,650
lambda k turns out to be this scalar how much
to move in this direction is given by this
113
00:17:44,650 --> 00:17:51,800
particular scalar
okay
114
00:17:51,800 --> 00:18:07,400
now what i can show is that theoretically
i can reach the optimum of phi only in n steps
115
00:18:07,400 --> 00:18:13,180
okay i have started with this objective function
where a is symmetric positive definite so
116
00:18:13,180 --> 00:18:27,680
theoretically you can show that 1/2 x transpose
a x-x transpose b okay solution of this can
117
00:18:27,680 --> 00:18:37,220
be reached starting from arbitrary initial
condition only in n steps okay but of course
118
00:18:37,220 --> 00:18:44,870
what happens is there are numerical errors
when you compute in a computer and you cannot
119
00:18:44,870 --> 00:18:46,450
practically reach in n steps
120
00:18:46,450 --> 00:18:56,420
on paper you should converge to the solution
only in n steps very very powerful method
121
00:18:56,420 --> 00:19:06,441
okay now just look at a matrix which is say
1000 cross 1000 okay your gradient to reach
122
00:19:06,441 --> 00:19:14,430
very close to the solution in 1000 steps okay
each step only requires one matrix multiplication
123
00:19:14,430 --> 00:19:23,841
right it requires some matrix multiplications
and you know you converge to the solution
124
00:19:23,841 --> 00:19:30,210
very fast okay so this is actually one of
the popular methods for solving linear algebraic
125
00:19:30,210 --> 00:19:36,550
equations for large-scale systems and very
quickly you can reach the solution so that
126
00:19:36,550 --> 00:19:40,050
is one of the
127
00:19:40,050 --> 00:19:46,150
so you can get very close to the reasonably
accurate solution you can get very quickly
128
00:19:46,150 --> 00:19:59,030
yeah
you the lambda k can have some problem but
129
00:19:59,030 --> 00:20:09,820
you are multiplying by a right g k transpose
a g k no but i do not think that will happen
130
00:20:09,820 --> 00:20:17,510
in conjugate gradient gradient method you
can end up in the problem because of that
131
00:20:17,510 --> 00:20:25,710
you can get not 0/0 you get a small number
by small number
132
00:20:25,710 --> 00:20:32,820
so that can sometimes cause a problem but
more problem with in gradient method is not
133
00:20:32,820 --> 00:20:37,730
because of that the difficulty is because
it becomes very slow as you come and this
134
00:20:37,730 --> 00:20:42,700
does not become slow this homes down to the
solution in the n steps that least theoretically
135
00:20:42,700 --> 00:20:52,890
so it is a much better method okay so we move
on to the next thing now well we have talked
136
00:20:52,890 --> 00:20:58,550
about different methods of solving a x=b let
me just take a brief review of what we have
137
00:20:58,550 --> 00:20:59,800
done till now
138
00:20:59,800 --> 00:21:09,370
we started looking at direct methods we had
a at least taste of what are called as sparse
139
00:21:09,370 --> 00:21:15,120
matrix methods now we try to exploit number
of 0s that are present in the matrix and then
140
00:21:15,120 --> 00:21:24,990
we come up with a way of solving very quickly
in less number of multiplication and division
141
00:21:24,990 --> 00:21:29,620
then we moved on to iterative methods for
solving a x=b which was completely new to
142
00:21:29,620 --> 00:21:33,930
you because you have been looking at gaussian
elimination gauss jordan everything that is
143
00:21:33,930 --> 00:21:34,930
using
144
00:21:34,930 --> 00:21:44,200
but iterative methods can particularly in
let us say in newton-raphson when you are
145
00:21:44,200 --> 00:21:49,800
solving multiple a x=b problems it is not
that important that you exactly get the true
146
00:21:49,800 --> 00:21:56,840
solution at the iterates you can very quickly
get the solution approximate solution very
147
00:21:56,840 --> 00:22:01,480
close to the approximate solution using iterative
methods so jacobi method gauss seidel method
148
00:22:01,480 --> 00:22:06,260
we had a look at those methods we also looked
at why and how they converge how to make them
149
00:22:06,260 --> 00:22:07,590
converge and so on
150
00:22:07,590 --> 00:22:15,650
so we relate to the eigenvalues we said that
inside unit circle know if you have i eigenvalues
151
00:22:15,650 --> 00:22:23,050
of s inverse you can guarantee convergence
to the solution necessary and sufficient conditions
152
00:22:23,050 --> 00:22:27,650
we also derive some sufficient conditions
some necessary and sufficient conditions or
153
00:22:27,650 --> 00:22:33,300
how do you transform the problem to guarantee
solution and so on and we looked at very quickly
154
00:22:33,300 --> 00:22:39,420
gradient based methods optimization based
methods okay gradient method and conjugate
155
00:22:39,420 --> 00:22:42,350
gradient method
156
00:22:42,350 --> 00:22:45,951
if you do a course on optimization you will
see that gradient method and conjugate gradient
157
00:22:45,951 --> 00:22:52,030
method they form you know basis of many many
variants and there are more powerful variants
158
00:22:52,030 --> 00:23:02,590
of but these are 2 basic methods which are
used in numerical optimization the next thing
159
00:23:02,590 --> 00:23:12,620
that i want to move is what is called as a
condition number well what i want to talk
160
00:23:12,620 --> 00:23:22,150
about now is very very important aspect of
numerical computing this is how well-conditioned
161
00:23:22,150 --> 00:23:27,360
or ill-conditioned system of linear algebraic
equation is okay
162
00:23:27,360 --> 00:23:42,610
so now the problems that occur in numerical
computing one of the reasons for these problems
163
00:23:42,610 --> 00:23:49,170
is fixed point arithmetic okay what is fixed
point arithmetic? fixed point arithmetic is
164
00:23:49,170 --> 00:23:56,370
that we have finite number of bits to represent
a number okay and then we truncate okay so
165
00:23:56,370 --> 00:24:01,720
suppose a number goes beyond this representation
or if there is a number which cannot be represented
166
00:24:01,720 --> 00:24:07,800
exactly using finite number of bits we just
take the nearest approximation and work with
167
00:24:07,800 --> 00:24:08,910
it okay
168
00:24:08,910 --> 00:24:17,100
just to give you an idea that if you have
a computer which has 3 bit precision okay
169
00:24:17,100 --> 00:24:39,780
and if we are to add point you know suppose
i want to do this addition okay a 3 bit precision
170
00:24:39,780 --> 00:24:45,460
computer or 3 digit precision let us not say
3 bit 3 digit precision computer cannot actually
171
00:24:45,460 --> 00:24:51,350
look at this part of the number it will ignore
this part of the number it will only take
172
00:24:51,350 --> 00:25:04,770
this+0002 okay and that 3 digit computer will
commit an error of 31 10 to the power-4 in
173
00:25:04,770 --> 00:25:08,030
this addition okay
174
00:25:08,030 --> 00:25:15,250
and just imagine this is now of course we
have numbers floating point which is 16 digit
175
00:25:15,250 --> 00:25:22,150
precision and so on but there is something
that is being neglected okay and even though
176
00:25:22,150 --> 00:25:28,970
every time it is very very small okay each
time when you do it it is very very small
177
00:25:28,970 --> 00:25:32,710
number that is neglected but you know you
are doing millions of operations right you
178
00:25:32,710 --> 00:25:38,910
are doing millions of operations large matrix
you are doing so many 1000s of multiplications
179
00:25:38,910 --> 00:25:39,910
division
180
00:25:39,910 --> 00:25:45,390
each time you are neglecting a small number
okay each time you neglect a small number
181
00:25:45,390 --> 00:25:51,830
that can cascade and grow and build up and
create a problem now this happens only for
182
00:25:51,830 --> 00:26:01,100
those cases where the matrix is problematic
when matrix is not problematic okay the errors
183
00:26:01,100 --> 00:26:07,420
have natural tendency to die down and things
work out you get reasonable solutions so actually
184
00:26:07,420 --> 00:26:15,960
what you have to remember is that when you
are solving a x=b using a computer
185
00:26:15,960 --> 00:26:20,880
you actually never end of solving this exact
equation you end up solving something like
186
00:26:20,880 --> 00:26:41,050
this you end up solving a+ delta a x+ delta
x=b+ delta b i want to solve this a x=b i
187
00:26:41,050 --> 00:26:53,760
end up solving this problem that is because
i cannot represent a accurately inside my
188
00:26:53,760 --> 00:27:00,110
computer i cannot represent b accurately inside
my computer if a and b are not accurately
189
00:27:00,110 --> 00:27:05,430
represented i cannot get exact solution to
my problem okay
190
00:27:05,430 --> 00:27:19,520
now just to give you so what i want to do
is i want to have some idea when this solution
191
00:27:19,520 --> 00:27:33,000
will be far away from what i really want to
get okay what are the factors that influence
192
00:27:33,000 --> 00:27:39,490
how this solution behaves first of all the
right hand side b is something which is specified
193
00:27:39,490 --> 00:27:46,330
by the person who is using this set of equations
by the user let us say so b is not in our
194
00:27:46,330 --> 00:27:55,870
hands okay what is fixed about this system
of equations is matrix a okay
195
00:27:55,870 --> 00:28:04,930
so how the solutions behave it actually depends
upon how you know a matrix is or how well-conditioned
196
00:28:04,930 --> 00:28:15,390
or how ill-conditioned a matrix is so our
main you know culprit or when things go wrong
197
00:28:15,390 --> 00:28:21,800
is a matrix how to recognize whether a matrix
is good or a matrix is bad if a matrix is
198
00:28:21,800 --> 00:28:28,740
bad whatever you do okay to some extent well
good programs like mat lab can recover and
199
00:28:28,740 --> 00:28:33,550
give you a reasonable solution but beyond
the point even mat lab fails
200
00:28:33,550 --> 00:28:39,870
and it is not because there is something wrong
with mat lab because there is a problem with
201
00:28:39,870 --> 00:28:46,340
the matrix the matrix is ill-conditioned in
iterations sometimes you may have received
202
00:28:46,340 --> 00:28:52,620
a message like this matrix is ill-conditioned
the results will not be reliable so mat lab
203
00:28:52,620 --> 00:28:57,340
does its job it tells you that there is something
going on wrong here you should know what it
204
00:28:57,340 --> 00:29:04,570
means you should know what the warning means
you should understand and analyze okay
205
00:29:04,570 --> 00:29:14,300
so let me give you a motivation for what is
going to happen next that is condition numbering
206
00:29:14,300 --> 00:29:21,770
i will need one more lecture to cover this
business of condition numbers but i am going
207
00:29:21,770 --> 00:29:26,620
to give a motivation for this today why do
i want to look at condition number then we
208
00:29:26,620 --> 00:29:39,290
will look at the mathematical details okay
so this example i have taken from strang it
209
00:29:39,290 --> 00:29:53,030
is a very nice simple example which illustrates
he using a very simple example 2 cross 2 example
210
00:29:53,030 --> 00:30:00,080
strang has illustrated the point that sometimes
the calculations go wrong because you are
211
00:30:00,080 --> 00:30:04,890
not done proper ordering of the calculations
if you reorder the calculations things can
212
00:30:04,890 --> 00:30:15,550
be okay but sometimes the matrix is bad whatever
you do there will be a problem okay so this
213
00:30:15,550 --> 00:30:29,370
system 1 he considers is the simple system
see this is the very very simple system 00001
214
00:30:29,370 --> 00:30:43,510
1 1 and 1 okay so now how will you do gaussian
elimination?
215
00:30:43,510 --> 00:30:48,880
if you do it without thinking about the problem
you will just take this is a non 0 number
216
00:30:48,880 --> 00:30:57,580
okay you will take this as a pivot and then
proceed with making you know the matrix to
217
00:30:57,580 --> 00:30:59,330
be lower triangular
218
00:30:59,330 --> 00:31:26,480
so your next step would be 00001 and 1 0 -9999
x1 x2 so this will
219
00:31:26,480 --> 00:31:39,680
give you x2=this and then you will back substitute
and get x1 right so we round this off to 1
220
00:31:39,680 --> 00:31:45,580
let us say you have a computer which has finite
digit position so you round this off to 1
221
00:31:45,580 --> 00:31:55,410
okay 0999 will get rounded off to 1 if this
is rounded off to 1 what is the solution for
222
00:31:55,410 --> 00:32:09,230
x1 if this is rounded off to 1 what will happen
x1 will be 0 okay this is because of the rounding
223
00:32:09,230 --> 00:32:10,230
off error
224
00:32:10,230 --> 00:32:18,970
now what is the exact what is the correct
way of doing calculations here well i will
225
00:32:18,970 --> 00:32:28,410
do maximal pivoting you know what is maximal
pivoting? in gaussian elimination what you
226
00:32:28,410 --> 00:32:39,460
do you rearrange the rows such that the diagonal
elements are maximum so if i do maximal pivoting
227
00:32:39,460 --> 00:32:59,400
this will be 1 1 00001 1 x1 x2=2 1 my modified
problem same system okay i have done maximal
228
00:32:59,400 --> 00:33:11,620
pivoting i have moved this pivot here 1 now
do gaussian elimination okay so now if i do
229
00:33:11,620 --> 00:33:29,090
gaussian elimination here i will get 1 1 0
09999 okay
230
00:33:29,090 --> 00:33:47,750
what is x2 right what is x1 now? if x2=1 x1=1
you see the problem here same system of equations
231
00:33:47,750 --> 00:33:59,700
i re-order the calculation once i got solution
0 and 1 here i got solution 1 and 1 rounding
232
00:33:59,700 --> 00:34:06,110
off okay what mattered here this is the correct
solution 1 1 is the correct solution what
233
00:34:06,110 --> 00:34:12,490
mattered was way i ordered my calculations
if i do wrong ordering my calculations this
234
00:34:12,490 --> 00:34:19,320
particular matrix is not ill-conditioned matrix
it is well-conditioned matrix okay
235
00:34:19,320 --> 00:34:23,010
why this well-conditioned matrix will come
to that little later but right now just take
236
00:34:23,010 --> 00:34:27,540
it for granted this is a well-conditioned
matrix what went wrong was we did ordering
237
00:34:27,540 --> 00:34:36,500
of calculations and okay we got a solution
which is not acceptable but we did not recognized
238
00:34:36,500 --> 00:34:41,240
it of course that type we thought everything
is logical we did rounding off there and then
239
00:34:41,240 --> 00:34:47,520
we got a solution which is but here when you
correct the order of calculations you recover
240
00:34:47,520 --> 00:34:55,370
the solution 1 1 and then
okay
241
00:34:55,370 --> 00:35:10,160
now let me take another system now second
system my system 2 is this 1 1 1 10001 x1
242
00:35:10,160 --> 00:35:25,200
x2=2 2 okay if
i apply you know first step in the gaussian
243
00:35:25,200 --> 00:35:37,310
elimination this will be 0 this will be 00001
right i just did gaussian elimination 2 times
244
00:35:37,310 --> 00:35:47,480
right so i just subtracted this row from this
row this from this i got 0 on this side right
245
00:35:47,480 --> 00:36:08,410
and then what is the solution x2=0 and x1=2
okay is this okay? okay
246
00:36:08,410 --> 00:36:21,720
what i am going to do now i am going to say
that well this was a slight error on representing
247
00:36:21,720 --> 00:36:27,110
the right hand side okay now you would say
well how does it matter if this slight 10
248
00:36:27,110 --> 00:36:33,480
to the power-4 okay i make an error of 10
to the power-4 on the right hand side so instead
249
00:36:33,480 --> 00:36:52,310
of presenting 2 and 2 you know i ended up
representing 2 and 20001 okay instead of typing
250
00:36:52,310 --> 00:36:59,070
2 and 2 let us say i typed 2 and 00001 you
would expect what is there if these are these
251
00:36:59,070 --> 00:37:06,220
are 2 very close vectors error is of the order
of just see what i will happen
252
00:37:06,220 --> 00:37:25,600
if this was instead by mistake you know represented
as 20001 this will become what next if this
253
00:37:25,600 --> 00:37:42,820
happens my x1 will be=1 because 00001/00001
okay x1 will become 1 what will be x2? or
254
00:37:42,820 --> 00:37:57,450
the other way around suppose this was you
know 19999 it will be x1 will be-1 x 2 will
255
00:37:57,450 --> 00:38:07,110
be 3 so a slight error on the right hand side
slight 10 to the power-4 which you think you
256
00:38:07,110 --> 00:38:12,040
know looking at these numbers 10 to the power-4
we are working with 1 1 1 2 and all that 10
257
00:38:12,040 --> 00:38:13,560
to the power-4 error
258
00:38:13,560 --> 00:38:22,960
but such a small error is causing a drastic
change in the solution such a small error
259
00:38:22,960 --> 00:38:27,200
this will not happen for system 1 system once
you reorder the calculations if you perturb
260
00:38:27,200 --> 00:38:34,830
the right hand side slightly it is not going
to change the solution too much okay here
261
00:38:34,830 --> 00:38:45,350
if you have a solution which is if you have
the right hand side which is slightly perturbed
262
00:38:45,350 --> 00:38:57,400
okay you have a problem okay the solution
changes drastically okay so from 0 to 2 it
263
00:38:57,400 --> 00:39:02,190
changes to 1 to 1 it is not a small change
right not small change
264
00:39:02,190 --> 00:39:08,220
so this problem what we will show a little
later is that this is because whatever you
265
00:39:08,220 --> 00:39:14,250
try to do here you rearrange the calculations
you do by pivoting well here there is no scope
266
00:39:14,250 --> 00:39:19,510
for pivoting both pivots are 1 1 so there
is no question of maximal pivoting but here
267
00:39:19,510 --> 00:39:23,062
nothing is going to change this problem there
is a slight error in the representation on
268
00:39:23,062 --> 00:39:29,220
the right hand side you will get drastically
different solution okay so this problem is
269
00:39:29,220 --> 00:39:32,390
ill-conditioned no reordering is going to
help you
270
00:39:32,390 --> 00:39:37,530
the first problem is not ill-conditioned it
is a problem of you know reordering the calculations
271
00:39:37,530 --> 00:39:45,480
to get the correct solution okay now i want
measure that will separate these 2 situations
272
00:39:45,480 --> 00:39:50,560
where i can say well this a matrix is ill-conditioned
whatever i do i will get into trouble this
273
00:39:50,560 --> 00:39:56,430
a matrix is well-conditioned if my solutions
are absurd i have made an error okay which
274
00:39:56,430 --> 00:40:00,060
is not the case for the second case second
case solutions are absurd it is because there
275
00:40:00,060 --> 00:40:03,400
is a limitation we will continue in the next
lecture