1
00:00:18,330 --> 00:00:25,830
so in the last few lectures we have been looking
at convergence of iterative schemes for solving
2
00:00:25,830 --> 00:00:39,690
linear algebraic equations starting from the
basic equation for way the error evolves
3
00:00:39,690 --> 00:00:52,949
so
we have this iteration scheme to solve ax=b
4
00:00:52,949 --> 00:01:07,140
and a was written as s-t and we said that
the error which is defined as
5
00:01:07,140 --> 00:01:18,650
iterative xk-the true solution x star this
evolves according to e k+1 this is a linear
6
00:01:18,650 --> 00:01:28,360
difference equation
it evolves according to this linear difference
7
00:01:28,360 --> 00:01:29,360
equation
8
00:01:29,360 --> 00:01:41,930
now from this we abstracted a linear difference
equation problem we said essentially we have
9
00:01:41,930 --> 00:01:53,340
to look at equations of this type and z zero
is initial condition and then we wanted to
10
00:01:53,340 --> 00:02:02,520
come up with the way of analyzing asymptotic
behaviour of the solution as a tends to infinity
11
00:02:02,520 --> 00:02:15,200
so we came up with analysis based on eigenvalues
we came up with a condition that if rho b
12
00:02:15,200 --> 00:02:30,640
is nothing but m a x/i lambda i that means
if lambda
13
00:02:30,640 --> 00:02:37,060
i are eigenvalues of matrix b we find out
its absolute value where eigenvalues can be
14
00:02:37,060 --> 00:02:38,170
complex
15
00:02:38,170 --> 00:02:49,330
so we find the absolute value and rho b this
is called as spectral radius and we showed
16
00:02:49,330 --> 00:02:57,310
that the necessary and sufficient condition
if rho b that means spectral radius of matrix
17
00:02:57,310 --> 00:03:09,069
be strictly less than 1 we said that if the
spectral radius of matrix b is strictly less
18
00:03:09,069 --> 00:03:18,930
than 1 then the sequence zk norm of that we
tend to 0 as k tends to infinity and from
19
00:03:18,930 --> 00:03:25,739
this we again connected to our original problem
we said which means that spectral radius of
20
00:03:25,739 --> 00:03:30,230
s inverse t is strictly less than 1
21
00:03:30,230 --> 00:03:43,440
then this is necessary and sufficient condition
for convergence of error okay the error between
22
00:03:43,440 --> 00:03:50,930
the true solution and the guess solution will
diminish to zero if this condition is satisfied
23
00:03:50,930 --> 00:03:55,270
okay i am just doing a recap of what we have
done till now so from this point we again
24
00:03:55,270 --> 00:04:01,820
had some difficulty because we have to compute
eigenvalues so we said that eigenvalues computations
25
00:04:01,820 --> 00:04:11,400
are difficult and then we used one more result
to come up with a sufficient condition
26
00:04:11,400 --> 00:04:18,680
so we said that spectral radius of matrix
b is always less than or equal to any induced
27
00:04:18,680 --> 00:04:29,200
norm of matrix b spectral radius of matrix
b is less than induced norm of matrix b what
28
00:04:29,200 --> 00:04:42,169
is induced norm this is induced norm induced
by the norm defined on the rain space and
29
00:04:42,169 --> 00:04:47,320
the domain space so this norm of matrix is
nothing but amplification power or something
30
00:04:47,320 --> 00:04:54,620
like gain of a matrix if you can think about
it as a gain or amplification power
31
00:04:54,620 --> 00:05:01,909
then we came up with a sufficient condition
that if induced norm is less than one obviously
32
00:05:01,909 --> 00:05:07,500
spectral radius is less than one and convergence
is guaranteed if induced norm is greater than
33
00:05:07,500 --> 00:05:13,969
one we cannot say anything okay if induced
norm is less than one we are sure so we had
34
00:05:13,969 --> 00:05:24,490
another condition that if induced norm is
strictly less than one then spectral radius
35
00:05:24,490 --> 00:05:47,439
of b is strictly less than one and then this
implies that asymptotically norm
36
00:05:47,439 --> 00:05:54,979
of iterate zk or difference equation zk will
go to zero as k goes to infinity
37
00:05:54,979 --> 00:06:03,400
so this we can say without actually having
to solve it now in particular we talked about
38
00:06:03,400 --> 00:06:12,099
infinite norm or one norm which are more convenient
to do calculations now based on this i wanted
39
00:06:12,099 --> 00:06:18,509
to derive some results which is even more
simpler i do not probably have to even compute
40
00:06:18,509 --> 00:06:25,639
the norm i can compute what is called as diagonal
dominance so in my last lecture i talked about
41
00:06:25,639 --> 00:06:28,089
diagonal dominance
42
00:06:28,089 --> 00:06:33,930
so i wanted to further cash on this result
that if the induced norm is less than one
43
00:06:33,930 --> 00:06:37,229
then of course the spectral radius is less
than one induced norm is very-very easy to
44
00:06:37,229 --> 00:06:45,250
compute particularly infinite norm as compared
to computing the spectral radius so checking
45
00:06:45,250 --> 00:06:51,539
whether a particular iteration will converge
or not is very-very easy okay now let us move
46
00:06:51,539 --> 00:06:59,309
back to the thing that we have done in my
last lecture
47
00:06:59,309 --> 00:07:08,339
so this was overview of the entire stability
arguments that we have been giving but i wanted
48
00:07:08,339 --> 00:07:15,319
to derive something more specific from the
previous results so we come back here we are
49
00:07:15,319 --> 00:07:31,099
trying to solve for ax=b and a has been split
as s-t so for jacobi method in particular
50
00:07:31,099 --> 00:07:46,110
i analysed jacobi method okay for jacobi method
s=d well we are also writing a=l+d+u this
51
00:07:46,110 --> 00:07:53,749
is strictly lower triangular part of a this
is diagonal part of a and this is strictly
52
00:07:53,749 --> 00:07:56,409
upper triangular part of a
53
00:07:56,409 --> 00:08:23,919
so s=d and t=-l-u and
then i defined the concept of strict diagonal
54
00:08:23,919 --> 00:08:39,260
dominance
55
00:08:39,260 --> 00:08:47,750
if you take sum of absolute values of elements
of matrix a in a row except the diagonal element
56
00:08:47,750 --> 00:08:57,870
and if that sum is strictly less than the
diagonal element then the matrix is called
57
00:08:57,870 --> 00:09:03,810
as diagonally dominant matrix okay just to
give you a simple example
58
00:09:03,810 --> 00:09:29,360
well you just write any matrix let us say
this is my matrix a these are the diagonal
59
00:09:29,360 --> 00:09:38,060
elements here just have a look if i take absolute
sum of this this this it is smaller than this
60
00:09:38,060 --> 00:09:45,010
i take absolute sum of this this this this
smaller than this okay i just look at this
61
00:09:45,010 --> 00:09:55,330
matrix i look at its diagonal elements okay
this particular matrix will obey this condition
62
00:09:55,330 --> 00:10:05,370
this is a strictly diagonal dominant matrix
just look at this this is 2+4+3+1 is always
63
00:10:05,370 --> 00:10:16,610
less than 15 okay same thing here okay 5+3+1+9
is less than 23 so i am taking absolute values
64
00:10:16,610 --> 00:10:25,310
okay so for this particular matrix can you
calculate what is going to be jacobi matrix
65
00:10:25,310 --> 00:10:36,420
which is s inverse t can you calculate that
just do it what is s inverse t
66
00:10:36,420 --> 00:10:43,190
well mind you again that jacobi matrix when
you actually do computations you never compute
67
00:10:43,190 --> 00:10:49,890
s inverse you do row by row calculations okay
this is for analysis this is for getting insights
68
00:10:49,890 --> 00:10:53,380
but what you will realise is that you just
look at the diagonal elements you look at
69
00:10:53,380 --> 00:10:54,990
the sum of all diagonal elements
70
00:10:54,990 --> 00:11:00,840
you can say whether the iteration are going
to converge or not which is very-very powerful
71
00:11:00,840 --> 00:11:07,350
reason you do not have to actually solve it
and this is true of 5x5 for 10x10 for 1000x1000
72
00:11:07,350 --> 00:11:15,630
if this condition holds iterations will converge
okay so you have guarantee convergence if
73
00:11:15,630 --> 00:11:16,750
this condition is satisfied
74
00:11:16,750 --> 00:11:24,780
so what will be s inverse t what will be jacobi
matrix let us call this jacobi matrix for
75
00:11:24,780 --> 00:11:50,250
jacobi matrix s inverse t what will it be
this will be zero it will be -1 1 02/-5 -5
76
00:11:50,250 --> 00:12:24,830
-5 then this will be 1 0 3 -2 2/9 9 9 9 okay
-l-u then what is this 2 4 0 3 -1 this divided
77
00:12:24,830 --> 00:12:56,840
by 15 15 15 and 15 okay then the next one
is 1 -5 -3 0 and -9/-23 -23 -23 and the last
78
00:12:56,840 --> 00:13:26,220
row is -1/5-1/5 1/5 0 0 okay what will be
the infinite norm of this matrix you take
79
00:13:26,220 --> 00:13:35,320
absolute of rows so absolute of this plus
absolute of this plus absolute of this okay
80
00:13:35,320 --> 00:13:38,490
absolute of these things all these numbers
81
00:13:38,490 --> 00:13:42,860
is it always going to be less than 1 it is
always going to be less than 1 because this
82
00:13:42,860 --> 00:13:52,090
matrix is strictly diagonally dominant okay
in the numerator this will appear okay actually
83
00:13:52,090 --> 00:13:59,940
for a diagonally dominant matrix what you
know is that this divided by norm aii this
84
00:13:59,940 --> 00:14:09,880
will be strictly less than 1 you can see here
you add absolute of each one of these rows
85
00:14:09,880 --> 00:14:14,660
okay if each one of them is less than one
the maximum is also going to be less than
86
00:14:14,660 --> 00:14:17,330
one what does it mean
87
00:14:17,330 --> 00:14:26,360
spectral radius of this matrix is strictly
less than one so if a is diagonally dominant
88
00:14:26,360 --> 00:14:36,650
the jacobi matrix which you get by s inverse
t has spectral radius strictly less than one
89
00:14:36,650 --> 00:14:43,150
which means jacobi iterations will converge
without having to solve it for arbitrary initial
90
00:14:43,150 --> 00:14:53,400
guess very-very important for an arbitrary
initial guess okay so any initial guess i
91
00:14:53,400 --> 00:15:01,810
give even if it is completely wrong my iterations
will converge to the true solution okay if
92
00:15:01,810 --> 00:15:03,270
diagonal dominance condition is met
93
00:15:03,270 --> 00:15:08,630
so you can just check diagonal dominance of
a matrix very-very easily and then you know
94
00:15:08,630 --> 00:15:16,230
whether the solution is going to be obtained
or not that is straight forward now there
95
00:15:16,230 --> 00:15:24,779
are many more results of how do you analyse
the convergence behaviour and all of them
96
00:15:24,779 --> 00:15:29,351
i am not going to prove i have stated those
results here and i am just going to state
97
00:15:29,351 --> 00:15:35,130
them and show you how to apply them and the
proofs for each one of them or at least most
98
00:15:35,130 --> 00:15:36,130
of them
99
00:15:36,130 --> 00:15:43,440
some of them you can derive yourself for most
of them are included at the end of the chapter
100
00:15:43,440 --> 00:15:51,150
notes in the appendix okay i do not want to
go over it in the class you go the philosophy
101
00:15:51,150 --> 00:15:59,800
of how it is done and you have to look at
the proofs in the appendix to understand more
102
00:15:59,800 --> 00:16:05,430
of this because we cannot spend time on this
beyond a certain point as long as you get
103
00:16:05,430 --> 00:16:10,300
the philosophy it is fine now what are the
more results
104
00:16:10,300 --> 00:16:16,320
there are some more results which exploit
the structure of matrix a one structure that
105
00:16:16,320 --> 00:16:22,540
we exploit is diagonal dominance right the
other thing we will show is that if matrix
106
00:16:22,540 --> 00:16:30,550
a is symmetric positive definite okay if matrix
a is symmetric positive definite then jacobi
107
00:16:30,550 --> 00:16:42,839
and gauss-seidel method converges also you
can show that if matrix a is diagonally dominant
108
00:16:42,839 --> 00:16:47,380
gauss-seidel method will converge okay
109
00:16:47,380 --> 00:16:53,350
the proof is little more involved and you
should look at the proof given in the appendix
110
00:16:53,350 --> 00:17:01,050
i have given details of the proof in the appendix
so if matrix a is diagonally dominant jacobi
111
00:17:01,050 --> 00:17:08,740
iterations will converge it is also true that
if matrix a is diagonally dominant then gauss-seidel
112
00:17:08,740 --> 00:17:13,689
iteration also will converge okay so you just
have to check for diagonal dominance you know
113
00:17:13,689 --> 00:17:21,279
that gauss-seidel iteration will converge
and in fact is that most of the time gauss-seidel
114
00:17:21,279 --> 00:17:24,490
iterations converge faster than jacobi iterations
okay
115
00:17:24,490 --> 00:17:30,040
so if you know that a matrix is diagonally
dominant your preferred choice of using the
116
00:17:30,040 --> 00:17:41,179
method should be gauss-seidel method not jacobi
method okay for other theorems; so the first
117
00:17:41,179 --> 00:17:47,260
theorem that you should know about convergence
of iteration scheme is that if matrix is diagonally
118
00:17:47,260 --> 00:17:56,770
dominant a matrix okay; then jacobi method
as well as gauss-seidel methods will converge
119
00:17:56,770 --> 00:18:04,450
to the true solution okay
120
00:18:04,450 --> 00:18:11,480
by the way remember this this is a sufficient
condition this is not a necessary condition
121
00:18:11,480 --> 00:18:26,360
what does it mean that if this condition holds
jacobi and gauss-seidel methods will converge
122
00:18:26,360 --> 00:18:30,990
you cannot say if this condition does not
hold you cannot say anything about convergence
123
00:18:30,990 --> 00:18:36,080
you have to go back and check something else
you have to go back and check spectral radius
124
00:18:36,080 --> 00:18:40,080
okay so this is only a sufficient condition
if this happens you are guaranteed convergence
125
00:18:40,080 --> 00:18:47,669
will occur if this does not happen we do not
know we cannot say anything okay so this is
126
00:18:47,669 --> 00:18:53,139
a sufficient condition not necessary condition
127
00:18:53,139 --> 00:19:03,220
so the second result i would say very-very
important result so the symmetric and positive
128
00:19:03,220 --> 00:19:08,200
definite seems to be something very-very nice
it seems to help us everywhere we go okay
129
00:19:08,200 --> 00:19:21,190
now you might start saying well i have been
given this matrix a and you know only in very-very
130
00:19:21,190 --> 00:19:26,289
special cases this a will be symmetric and
positive definite isn’t it
131
00:19:26,289 --> 00:19:35,100
a is a square matrix i am not talking about
the square where we had a tall matrix i am
132
00:19:35,100 --> 00:19:42,320
just talking about a is a square matrix and
the problem which is given to me is such that
133
00:19:42,320 --> 00:19:53,100
a is not symmetric and positive definite okay
but i know that gauss-seidel method will converge
134
00:19:53,100 --> 00:19:59,669
okay sufficient condition for convergence
is that if the matrix in my problem is symmetric
135
00:19:59,669 --> 00:20:06,080
and positive definite is there something i
do to solve this problem to convert this problem
136
00:20:06,080 --> 00:20:09,850
into symmetric and positive definite matrix
137
00:20:09,850 --> 00:20:24,200
i just pre-multiply this equation with a transpose
so this gives me a transpose a okay i do not
138
00:20:24,200 --> 00:20:33,820
have to solve for ax=b i can instead solve
for a transpose a=a transpose b okay i am
139
00:20:33,820 --> 00:20:39,940
guaranteed convergence so i am using my theory
to change the problem in such a way that i
140
00:20:39,940 --> 00:20:48,489
am guaranteed to get converge solution okay
i am going to solve this problem using gauss-seidel
141
00:20:48,489 --> 00:20:51,789
iterations making use of this theorem okay
142
00:20:51,789 --> 00:20:55,980
how do i make use of this theorem to modify
my calculations i pre-multiply both sides
143
00:20:55,980 --> 00:21:02,649
by a transpose this becomes a symmetric positive
definite matrix okay now if i apply gauss-seidel
144
00:21:02,649 --> 00:21:10,049
method to this matrix and this transform problem
i am guaranteed to get a solution this solution
145
00:21:10,049 --> 00:21:17,639
is obviously a solution if it is a solution
of this it is also a solution of this you
146
00:21:17,639 --> 00:21:21,009
have no problem with that so i could solve
this transform problem instead of solving
147
00:21:21,009 --> 00:21:26,899
this problem i get a symmetric positive definite
matrix here okay i am using theory to modify
148
00:21:26,899 --> 00:21:28,970
my calculations
149
00:21:28,970 --> 00:21:46,720
i will just give you an example here so i
want to solve for ax=b and my a matrix=4 5
150
00:21:46,720 --> 00:22:12,639
9 7 1 6 5 2 9 and my n vector is 1 1 1 okay
let say i want to solve this by gauss-seidel
151
00:22:12,639 --> 00:22:20,980
iterations okay well what i will do is i know
this is not a solution procedure this is analysis
152
00:22:20,980 --> 00:22:22,489
okay
153
00:22:22,489 --> 00:22:42,460
from analysis what i know is that if i write
this matrix as a matrix if i call this s and
154
00:22:42,460 --> 00:22:53,809
if i call this
as t then doing gauss-seidel iterations is
155
00:22:53,809 --> 00:23:15,641
equivalent then my s inverse t will be 4 0
0 7 1 0 this is my s inverse t if i am able
156
00:23:15,641 --> 00:23:28,760
to use the row matrix a in this case the spectral
radius of s inverse t turns out to be 73 which
157
00:23:28,760 --> 00:23:36,360
is strictly less than 1 okay if i use gauss-seidel
iterations iterations are not going to converge
158
00:23:36,360 --> 00:23:43,289
because if i just choose the row matrix a
that matrix is neither diagonally dominant
159
00:23:43,289 --> 00:23:49,799
just check it is diagonally dominant it is
not is it symmetric matrix it is not a symmetric
160
00:23:49,799 --> 00:23:56,059
matrix forget about positive definite it is
not symmetric matrix but if i know this little
161
00:23:56,059 --> 00:24:02,239
bit of information okay
162
00:24:02,239 --> 00:24:11,190
if i do this transformation that is a transpose
ax=a transpose b okay then this a transpose
163
00:24:11,190 --> 00:24:32,750
a matrix turns out to be 90 37 okay and this
is a transpose a a transpose b becomes 16
164
00:24:32,750 --> 00:24:46,639
8 24 and now if i apply gauss-seidel method
to this transformed equation then spectral
165
00:24:46,639 --> 00:25:04,340
radius of s inverse t turns out to be 096
okay so for the transform problem guaranteed
166
00:25:04,340 --> 00:25:09,389
convergence of gauss-seidel method this is
a symmetric matrix just see this symmetric
167
00:25:09,389 --> 00:25:13,580
matrix it is a positive definite matrix by
definition
168
00:25:13,580 --> 00:25:17,999
a transpose a is always positive definite
even if a is not positive definite we have
169
00:25:17,999 --> 00:25:23,240
seen this several times okay this is positive
definite matrix symmetric matrix convergence
170
00:25:23,240 --> 00:25:29,679
is guaranteed just pre-multiplying both sides
by a transpose i can ensure that i will get
171
00:25:29,679 --> 00:25:37,230
convergence by iterative method okay so in
the case where obvious things like diagonal
172
00:25:37,230 --> 00:25:43,200
dominance are not there if you want to ensure
that you get convergence just pre-multiply
173
00:25:43,200 --> 00:25:48,289
by a transpose both sides and then use gauss-seidel
you have guaranteed convergence very-very
174
00:25:48,289 --> 00:25:50,269
powerful result
175
00:25:50,269 --> 00:26:02,000
yeah for any given no matter how would always
so that spectral radius should be less than
176
00:26:02,000 --> 00:26:07,679
one is necessary and sufficient condition
if necessary if the convergence occurs spectral
177
00:26:07,679 --> 00:26:13,960
radius should be less than one if spectral
radius is less than one convergence will occur
178
00:26:13,960 --> 00:26:15,960
okay "professor - student conversation ends”
179
00:26:15,960 --> 00:26:21,350
but that is not the case with the norm if
induced norm is less than one convergence
180
00:26:21,350 --> 00:26:26,870
will occur but if induced norm is greater
than one convergence may or may not occur
181
00:26:26,870 --> 00:26:32,529
you may not know that is not the case with
spectral radius spectral radius is the absolute
182
00:26:32,529 --> 00:26:41,350
measure which is necessary and sufficient
condition okay so it is possible to transform
183
00:26:41,350 --> 00:26:55,799
there are more results of this type again
i am not going to go into the proof
184
00:26:55,799 --> 00:27:10,369
for relaxation method we have this result
for relaxation method what you can show again
185
00:27:10,369 --> 00:27:16,970
is the proof again given in the appendix you
should go and have a look at it if omega is
186
00:27:16,970 --> 00:27:21,980
chosen between 0 and 2 well actually for relaxation
method we want to choose it between 1 and
187
00:27:21,980 --> 00:27:27,851
2 because we showed that omega equal to 1
is equivalent to gauss iteration so we want
188
00:27:27,851 --> 00:27:36,059
to choose between 1 and 2 but in general if
omega is between 0 and 2 okay this is a necessary
189
00:27:36,059 --> 00:27:39,109
condition for convergence okay
190
00:27:39,109 --> 00:27:53,629
so you know how to choose omega you have a
guideline here okay so again remember this
191
00:27:53,629 --> 00:27:57,210
is only a necessary condition this is not
sufficient if you choose less than 2 that
192
00:27:57,210 --> 00:28:07,129
does not mean convergence has to occur but
convergence occurs only when you choose omega
193
00:28:07,129 --> 00:28:22,559
is less than 2 this is result 3 and the necessary
condition becomes necessary and sufficient
194
00:28:22,559 --> 00:28:31,940
conditions if extension to this theorem is
another result this is for an arbitrary matrix
195
00:28:31,940 --> 00:28:33,769
okay
196
00:28:33,769 --> 00:28:48,759
now if a is symmetric
and positive definite so if matrix a is symmetric
197
00:28:48,759 --> 00:28:58,509
positive definite okay this necessary condition
becomes necessary and sufficient condition
198
00:28:58,509 --> 00:29:03,470
okay now you know how to transform the problem
which is usually not symmetric positive definite
199
00:29:03,470 --> 00:29:08,029
to symmetric positive definite matrix okay
so what i want to do the take home message
200
00:29:08,029 --> 00:29:15,529
is that all these theorems are very-very useful
in shaping your calculations
201
00:29:15,529 --> 00:29:20,190
you should know how to make sure that convergences
occur convergence is very-very important whenever
202
00:29:20,190 --> 00:29:26,380
you are not sure in a arbitrary large scale
problem you are not sure of a matrix how it
203
00:29:26,380 --> 00:29:34,340
is going to be if you want to use iterative
schemes for solving ax=b it is better to use
204
00:29:34,340 --> 00:29:42,779
a relaxation method in which you transform
the problem because in general relaxation
205
00:29:42,779 --> 00:29:46,929
method will converge faster than gauss-seidel
method
206
00:29:46,929 --> 00:29:52,149
i will just show you a very small example
that jacobi method is the slowest to converge
207
00:29:52,149 --> 00:30:02,929
typcially gauss-seidel method is faster and
if you choose omega properly then the relaxation
208
00:30:02,929 --> 00:30:10,000
method will even converge faster okay now
how do you choose omega such that you get
209
00:30:10,000 --> 00:30:14,399
very-very fast convergence it is very difficult
to tell the you probably have to compute eigenvalues
210
00:30:14,399 --> 00:30:20,749
but that is not desirable you do not want
to really compute eigenvalues
211
00:30:20,749 --> 00:30:29,989
so you have to develop some kind of experience
beyond the point you have use all these theorems
212
00:30:29,989 --> 00:30:34,260
and understand the theory and then develop
experience to tweak with the calculations
213
00:30:34,260 --> 00:30:36,289
that is very-very important okay
214
00:30:36,289 --> 00:30:42,159
so i will just show you one simple example
this is taken from book but it is very-very
215
00:30:42,159 --> 00:30:51,100
illustrative very simple problem so i want
to solve and such a simple problem of course
216
00:30:51,100 --> 00:30:58,129
you do not need any of the iterative methods
2x2 systems you can solve it by hand so my
217
00:30:58,129 --> 00:31:08,979
a matrix is 2 -1 -1 2 well we will say that
this is jacobi and gauss-seidel will converge
218
00:31:08,979 --> 00:31:20,090
why symmetric diagonal dominant okay anyway
that is not the point the point is that for
219
00:31:20,090 --> 00:31:42,389
jacobi method s inverse t will be 0 1/2 1/2
0 and the spectral radius is equal to 1/2
220
00:31:42,389 --> 00:31:50,099
okay
221
00:31:50,099 --> 00:32:08,009
for gauss-seidel method s inverse t this turns
out to be 0 0 1/2 1/4 and spectral radius
222
00:32:08,009 --> 00:32:18,179
is s inverse t okay the spectral radius is
given by this actually spectral radius maximum
223
00:32:18,179 --> 00:32:24,450
magnitude eigenvalue of s inverse t is an
indicator also of the performance okay now
224
00:32:24,450 --> 00:32:32,139
there are two aspects it should be less than
one okay now how much it is less than one
225
00:32:32,139 --> 00:32:37,309
how close it is to zero that also matters
in terms of the rate of convergence
226
00:32:37,309 --> 00:32:41,249
whether the convergence is guaranteed or not
is decided by whether it is strictly less
227
00:32:41,249 --> 00:32:48,059
than one okay that is the stability criteria
the performance is given by how much it is
228
00:32:48,059 --> 00:33:00,029
less than one so this jacobi method in which
spectral radius is 1/2 okay converges slower
229
00:33:00,029 --> 00:33:09,450
than the gauss-seidel method okay because
the spectral radius here is 1/4 in fact if
230
00:33:09,450 --> 00:33:15,759
you start with calculations you will see that
one step of gauss-seidel will be almost equal
231
00:33:15,759 --> 00:33:18,609
to two steps of jacobi okay
232
00:33:18,609 --> 00:33:24,460
so the gauss-seidel can move much faster you
cannot show it for every matrix this is no
233
00:33:24,460 --> 00:33:30,059
proof that gauss-seidel always converges but
in general gauss-seidel convergence faster
234
00:33:30,059 --> 00:33:35,779
than and the reason is typically spectral
radius of s inverse t for gauss-seidel is
235
00:33:35,779 --> 00:33:46,509
less than okay that is the reason now what
if i formulate the relaxation method so you
236
00:33:46,509 --> 00:34:06,059
can almost show that because of this one gs
iteration is equivalent to 2 jacobi iterations
237
00:34:06,059 --> 00:34:08,550
okay because spectral radius in this case
is even smaller
238
00:34:08,550 --> 00:34:21,899
now for relaxation method
s inverse t turns out to be inverse of this
239
00:34:21,899 --> 00:34:40,429
matrix 0 omega -omega here 2 inverse 2*1-omega
omega 0 2*1-omega and of course we should
240
00:34:40,429 --> 00:34:51,050
choose omega between 1 and 2 we want it to
be greater than one because if it is equal
241
00:34:51,050 --> 00:35:00,800
to one it is nothing but gauss-seidel method
if we want to be greater than one now for
242
00:35:00,800 --> 00:35:07,620
this simple case 2x2 matrix you can actually
find out what is the best value of omega that
243
00:35:07,620 --> 00:35:11,720
will enhance the convergence what is the optimum
value okay
244
00:35:11,720 --> 00:35:17,120
for different choices of omega you will get
different spectral radius okay you can actually
245
00:35:17,120 --> 00:35:25,750
find out which value of omega this is just
again to tell you emphasis it this is only
246
00:35:25,750 --> 00:35:33,990
to get insight in real problem i am not going
compute optimum omega by doing some i have
247
00:35:33,990 --> 00:35:37,600
to tune give a guess for omega
248
00:35:37,600 --> 00:35:46,780
so if you use some properties then you know
that lambda 1 and lambda 2 if these two are
249
00:35:46,780 --> 00:35:54,940
eignvalues of s inverse t then that is equal
to determent of s inverse t which turns out
250
00:35:54,940 --> 00:36:05,990
to be in this case if you take determinant
of that it will be 1-omega whole square okay
251
00:36:05,990 --> 00:36:11,430
you know this property multiplication of eigenvalues
for matrix is same as determinant and then
252
00:36:11,430 --> 00:36:23,470
what is the other property trace so lambda
1+lambda 2=trace of
253
00:36:23,470 --> 00:36:42,080
so this will turn out to be 2-2omega+omega
square/4 now if you plot this that is if you
254
00:36:42,080 --> 00:36:55,250
plot s inverse t versus omega if you plot
spectral radius using these two relationships
255
00:36:55,250 --> 00:37:01,760
you can find out lambda 1 and lambda 2 and
spectral radius; and if you plot this you
256
00:37:01,760 --> 00:37:07,510
will find that getting the optimum is not
very difficult
257
00:37:07,510 --> 00:37:20,550
if you plot this you will find that the optimum
value for which the spectral radius is minimum
258
00:37:20,550 --> 00:37:27,200
you know you will get a point where you will
get a minimum value of the spectral radius
259
00:37:27,200 --> 00:37:42,570
so that value turns out to be omega optimum=107okay
and the spectral radius of s inverse t for
260
00:37:42,570 --> 00:37:52,480
this omega is 007 okay i am skipping some
steps you can see here in the notes that is
261
00:37:52,480 --> 00:37:56,930
not important what i want to point out here
is that if you are able to choose omega properly
262
00:37:56,930 --> 00:37:59,840
then this is 007
263
00:37:59,840 --> 00:38:08,710
so we had three situations; you know jacobi
method then gauss-seidel method and relaxation
264
00:38:08,710 --> 00:38:16,550
method the s inverse t spectral radius in
this case was 1/2 this was 1/4 and this is
265
00:38:16,550 --> 00:38:27,710
you know 07 which is almost 1/4 of this so
we said one iteration of gauss-seidel was
266
00:38:27,710 --> 00:38:35,710
two jacobi iterations what can you say about
one relaxation iteration it is like one relaxation
267
00:38:35,710 --> 00:38:43,000
iteration is like four gauss-seidel iterations
almost like eight jacobi iterations
268
00:38:43,000 --> 00:38:51,890
so relaxation method can converge even faster
typically values close to 1 11 12 are used
269
00:38:51,890 --> 00:38:59,780
this is thumb rule and not substantiated i
think gave some clue that you can use it close
270
00:38:59,780 --> 00:39:08,260
to 12 but it is hard to say generally what
value of omega will make convergence very-very
271
00:39:08,260 --> 00:39:15,750
fast okay so the tricks that you should use
is first of all make sure that either the
272
00:39:15,750 --> 00:39:22,660
matrix is diagonally dominant if is not okay
to ensure convergence he should pre-multiply
273
00:39:22,660 --> 00:39:28,220
both the side by a transpose that will make
it symmetric positive definite i have guaranteed
274
00:39:28,220 --> 00:39:29,830
convergence okay
275
00:39:29,830 --> 00:39:36,040
but i want convergence faster than gauss-seidel
gauss-seidel is better than jacobi so i will
276
00:39:36,040 --> 00:39:41,580
apply gauss-seidel and i can make convergence
faster even going to relaxation method so
277
00:39:41,580 --> 00:39:46,840
probably i should all these tricks and use
relaxation method to enhance my convergence
278
00:39:46,840 --> 00:39:55,530
that is how i should proceed with arranging
my calculations so this brings us to end of
279
00:39:55,530 --> 00:40:02,510
this analysis what is important here is that
there are many take home messages
280
00:40:02,510 --> 00:40:10,330
one of the things is that eigenvalues is one
of the prime tools for analysing behaviour
281
00:40:10,330 --> 00:40:16,330
qualitatively asymptotically i do not have
to solve that is the beauty of this tool i
282
00:40:16,330 --> 00:40:21,500
do not have to solve the problem i can just
look at eigenvalues or in this case it turns
283
00:40:21,500 --> 00:40:25,210
out finally that i can just look at diagonal
dominance i can see whether to convert the
284
00:40:25,210 --> 00:40:29,830
problem to a symmetric positive definite matrix
i am guaranteed convergence of my iterative
285
00:40:29,830 --> 00:40:32,330
scheme very-very powerful result
286
00:40:32,330 --> 00:40:39,660
in fact eigenvalues are used for convergence
analysis in engineering literature in many
287
00:40:39,660 --> 00:40:45,840
many many ways okay well most of you i think
have done the first course in chemical engineering
288
00:40:45,840 --> 00:40:53,520
or process control and in process control
well you may not have connected it to the
289
00:40:53,520 --> 00:40:59,100
eigenvalues in the first course but actually
what you can show is that if you write a differential
290
00:40:59,100 --> 00:41:07,250
equation for local linear differential equation
for evaluation of the system dynamics then
291
00:41:07,250 --> 00:41:11,720
the so-called roots of the characteristic
polynomial are nothing but eigenvalues of
292
00:41:11,720 --> 00:41:15,120
certain matrix which governs the system dynamics
okay
293
00:41:15,120 --> 00:41:20,910
if the eigenvalues are on the left of plane
and then you know what was nice thing about
294
00:41:20,910 --> 00:41:25,840
eigenvalues there or roots of the characteristic
polynomial you do not have to solve you just
295
00:41:25,840 --> 00:41:28,600
look at the roots whether they are lying on
this half of the plane or this half of the
296
00:41:28,600 --> 00:41:32,630
plane you can tell how the system is going
to behave asymptotically without having to
297
00:41:32,630 --> 00:41:38,900
solve okay same thing is here without having
to solve i can tell whether my iterations
298
00:41:38,900 --> 00:41:41,590
will converge or not okay
299
00:41:41,590 --> 00:41:46,310
also using necessary sufficient conditions
i can go and modify my problem to make sure
300
00:41:46,310 --> 00:41:52,560
that convergence will occur okay this is more
important than the algorithm per se algorithm
301
00:41:52,560 --> 00:41:59,840
you will learn to program it or nowadays i
think these algorithm will be available on
302
00:41:59,840 --> 00:42:05,620
the net you might download it algorithm might
be very well written that does not mean that
303
00:42:05,620 --> 00:42:08,330
does not mean that convergence is going to
occur okay
304
00:42:08,330 --> 00:42:13,550
you should know why convergence occurs and
then make sure that you transform the problem
305
00:42:13,550 --> 00:42:25,700
in such a way that convergence occurs that
is important okay there are two more things
306
00:42:25,700 --> 00:42:33,930
that i need to do because we missed one lecture
some timetable is disturbed but i will try
307
00:42:33,930 --> 00:42:45,490
to make up for it i will try to cover optimization
based iterative methods for solving ax=b okay
308
00:42:45,490 --> 00:42:51,360
so till now i formulated iterations in one
particular way by splitting the matrix in
309
00:42:51,360 --> 00:42:57,100
fact row by row calculations not really splitting
the matrix the way the iterations were derived
310
00:42:57,100 --> 00:43:00,550
where doing row by row calculations
311
00:43:00,550 --> 00:43:07,330
my next aim is going to be instead of doing
that can i iteratively solve this problem
312
00:43:07,330 --> 00:43:24,310
well if i want to solve ax=b okay well if
i take a guess solution the true solution
313
00:43:24,310 --> 00:43:40,970
is let us say x star
okay and if xk is my guess solution then obviously
314
00:43:40,970 --> 00:43:56,650
ek now my ek has a different definition ek
is b-axkokay this is not going to be 0 when
315
00:43:56,650 --> 00:44:00,700
this is equal to x star it will be equal to
0 if xk is equal to x star this is equal to
316
00:44:00,700 --> 00:44:02,210
0
317
00:44:02,210 --> 00:44:13,460
the way i want to solve this problem is minimize
scalar objective function phi is defined as
318
00:44:13,460 --> 00:44:32,340
e transpose e that is ax-b transpose ax-b
with respect to x okay i do not want to solve
319
00:44:32,340 --> 00:44:39,130
this well you will say that if you apply it
on a necessary condition for optimality you
320
00:44:39,130 --> 00:44:48,030
will get you know dou phi/dou x=0 and we will
give you a transpose ax if you apply this
321
00:44:48,030 --> 00:44:58,340
condition dou phi/dou x-=0 then you will get
a transpose ax=a transpose b
322
00:44:58,340 --> 00:45:04,670
i do not want to solve this i do not want
to go by this route i want to go iteratively
323
00:45:04,670 --> 00:45:16,430
okay i want to guess x0 and by some method
i want to go to x1 then i want to go to x2
324
00:45:16,430 --> 00:45:21,630
and so on and then i want to see whether this
iteration converges we are going to use what
325
00:45:21,630 --> 00:45:26,970
is called as the gradient search okay one
of the fundamental methods in optimization
326
00:45:26,970 --> 00:45:31,670
gradient based search so we will look at gradient
search and then there is one more method called
327
00:45:31,670 --> 00:45:35,450
conjugate gradient search which we will look
at next okay
328
00:45:35,450 --> 00:45:39,750
that is one thing which i want to do after
having done that we have talked about iterative
329
00:45:39,750 --> 00:45:45,680
schemes for solving ax=b and then we move
on to a very-very fundamental issue matrix
330
00:45:45,680 --> 00:45:50,790
conditioning which problems are inherently
ill-conditioned which problems are well-conditioned
331
00:45:50,790 --> 00:45:56,380
how do i classify and say that this is ill-conditioned
problem whatever i do i am going to end up
332
00:45:56,380 --> 00:45:59,070
into some trouble this is a well-conditioned
problem
333
00:45:59,070 --> 00:46:04,970
if i am getting wrong solution i have made
a mistake so well-conditioned problems you
334
00:46:04,970 --> 00:46:09,270
know absurd solutions you have made a mistake
ill-conditioned problems absurd solutions
335
00:46:09,270 --> 00:46:14,560
you cannot do much how do you classify ill-condition
from well-condition is the next thing that
336
00:46:14,560 --> 00:46:17,050
will bring this to end of this module