1
00:00:17,600 --> 00:00:25,750
We have been looking at nonlinear algebraic
equations and we looked at three different
2
00:00:25,750 --> 00:00:35,050
classes of methods One was derivative free
method the other was sloper derivative based
3
00:00:35,050 --> 00:00:44,330
methods and the third was optimization so
which was numerical optimization and we are
4
00:00:44,330 --> 00:00:51,890
looked at the algorithmic aspect of nonlinear
algebraic equations Now today I am going to
5
00:00:51,890 --> 00:01:00,059
touch upon the convergence aspect
6
00:01:00,059 --> 00:01:04,670
So very very important aspect of equations
but I am just going to give a very very brief
7
00:01:04,670 --> 00:01:10,890
introduction I am not going to go deep into
this I just want to sensitize you that there
8
00:01:10,890 --> 00:01:21,620
exists lot of work lot of literature on convergence
of nonlinear iterative schemes For convergence
9
00:01:21,620 --> 00:01:27,200
of linear iterative schemes like Gauss Seidel
method Jacobi method we could actually derive
10
00:01:27,200 --> 00:01:30,000
in the class necessary and sufficient conditions
11
00:01:30,000 --> 00:01:38,350
Whereas for nonlinear cases much more difficult
and the machinery that you require it is fairly
12
00:01:38,350 --> 00:01:45,689
more advance than what we are doing covering
in this course and also many times you only
13
00:01:45,689 --> 00:01:56,520
get sufficient conditions you do not necessary
conditions So nevertheless these tools or
14
00:01:56,520 --> 00:02:01,659
the theorems that actually give sufficient
conditions give lot of insight into how solutions
15
00:02:01,659 --> 00:02:03,590
of nonlinear algebraic equations behave
16
00:02:03,590 --> 00:02:15,069
So I am just going to touch up on it today
not really go into deep of this subject So
17
00:02:15,069 --> 00:02:23,050
one thing that we need to talk about see if
you look at the development that we did for
18
00:02:23,050 --> 00:02:27,130
the linear algebraic equations We had there
is some sense parallel between what we have
19
00:02:27,130 --> 00:02:34,670
done there and what we done here There too
we talked about noniterative schemes iterative
20
00:02:34,670 --> 00:02:39,510
schemes and then we talked about optimization
based schemes
21
00:02:39,510 --> 00:02:44,930
There we talked about a very important issue
called condition number So we said that a
22
00:02:44,930 --> 00:02:50,780
set of linear algebraic equations is well
conditioned or ill conditioned depending upon
23
00:02:50,780 --> 00:02:58,230
some properties of matrix A right and it was
possible to do analytical treatment quite
24
00:02:58,230 --> 00:03:05,970
easily with whatever we have learnt till now
Can we extend this to nonlinear algebraic
25
00:03:05,970 --> 00:03:09,160
equations I am just going to briefly touch
up on this idea
26
00:03:09,160 --> 00:03:16,500
That what is the condition number of a nonlinear
algebraic system and then move onto the convergence
27
00:03:16,500 --> 00:03:38,510
properties of or how do we analyze the convergence
of nonlinear algebraic equations So what was
28
00:03:38,510 --> 00:03:49,850
in the case of so first thing I just want
to touch up on this condition number
29
00:03:49,850 --> 00:03:58,600
So in linear algebraic equations we had defined
condition number as when you have Ax equal
30
00:03:58,600 --> 00:04:01,320
to b okay
31
00:04:01,320 --> 00:04:09,470
One may be defined this condition number was
sensitivity of the solution x to a small change
32
00:04:09,470 --> 00:04:21,979
in b right So if we look at this as a input
and x as a output if you look at this first
33
00:04:21,979 --> 00:04:28,169
time as b as a input and x as output where
A is the operator one maybe defined the condition
34
00:04:28,169 --> 00:04:54,320
number was norm delta x by norm x
35
00:04:54,320 --> 00:05:01,740
So we showed that this ratio that is fractional
change in the solution to fractional change
36
00:05:01,740 --> 00:05:10,680
in the input okay is bounded by this condition
number which is multiplication of norm of
37
00:05:10,680 --> 00:05:12,520
A into norm of A inverse
38
00:05:12,520 --> 00:05:21,920
Now to draw a parallel I am going to consider
nonlinear algebraic equations of the form
39
00:05:21,920 --> 00:05:30,240
f of x u equal to 0 okay Well this kind of
equation very routinely arise in chemical
40
00:05:30,240 --> 00:05:40,820
engineering when you are solving steady state
behavior of say CSTR x are no states concentration
41
00:05:40,820 --> 00:05:48,880
temperature inside the reactor u are inputs
as an input flow rate inlet concentration
42
00:05:48,880 --> 00:05:55,710
inlet temperature all these free parameters
okay input parameters
43
00:05:55,710 --> 00:06:05,700
So if you fix yourself to one input condition
you will get one steady
44
00:06:05,700 --> 00:06:09,960
state of the reactor or let us say you have
distillation column you have this kind of
45
00:06:09,960 --> 00:06:18,900
equation u there is nothing but feed composition
feed flow rate feed temperature reflux rate
46
00:06:18,900 --> 00:06:26,810
heat input okay all these free inputs which
in balance of control you call as disturbances
47
00:06:26,810 --> 00:06:29,980
or manipulate inputs all these inputs are
u
48
00:06:29,980 --> 00:06:36,520
X are all the dependent variables like tray
temperature tray concentrations vapor concentration
49
00:06:36,520 --> 00:06:44,740
liquid concentration everything So moment
to fix one u vector okay for a particular
50
00:06:44,740 --> 00:06:56,062
u vector let us say u equal to u bar you get
f of x u bar equal to 0 this is what you have
51
00:06:56,062 --> 00:07:08,650
to solve Once you fix u bar okay say typically
x belongs Rn and u belongs to Rm and for every
52
00:07:08,650 --> 00:07:22,639
u you fix okay and f
is a n cross 1 vector okay
53
00:07:22,639 --> 00:07:28,280
This particular vector n cross 1 vector is
a state of nonlinear algebraic equations You
54
00:07:28,280 --> 00:07:34,772
have to solve them simultaneously for a given
u for every u okay If I change the feed condition
55
00:07:34,772 --> 00:07:42,419
if I change the feed composition the concentration
or temperature profile in the distillation
56
00:07:42,419 --> 00:07:49,370
column is going to be different okay For every
value of this input conditions you get one
57
00:07:49,370 --> 00:07:54,060
set of steady state solution okay
58
00:07:54,060 --> 00:07:59,090
In some sense this u is parallel with b on
the right hand side If you change the right
59
00:07:59,090 --> 00:08:11,540
hand side b you get different x okay So now
we define condition number as with respect
60
00:08:11,540 --> 00:08:24,490
to solution of f of x plus delta x and u plus
delta u equal to 0 So when I change u for
61
00:08:24,490 --> 00:08:31,470
u bar 2 say u bar plus delta u bar okay When
I introduce a perturbation in u what is the
62
00:08:31,470 --> 00:08:36,889
corresponding perturbation in the solution
x okay
63
00:08:36,889 --> 00:08:43,400
So I am going to define sensitivity of this
equation or sensitivity of the solution with
64
00:08:43,400 --> 00:08:50,170
reference to perturbation in u as my condition
number same idea fractional sensitivity of
65
00:08:50,170 --> 00:08:57,240
the solution to fractional sensitivity or
fractional change in the input okay that is
66
00:08:57,240 --> 00:09:13,930
going to be my condition number So for a nonlinear
system we define cx as supremum over delta
67
00:09:13,930 --> 00:09:37,980
u
68
00:09:37,980 --> 00:09:44,360
okay We define this a supremum over all perturbations
delta u okay
69
00:09:44,360 --> 00:10:09,309
This of course nonzero perturbations delta
u So in other words this delta x
70
00:10:09,309 --> 00:10:23,920
not c of x c of f this should be well this
in general will not be a constant number like
71
00:10:23,920 --> 00:10:32,480
matrix you get you know matrix is a operator
which only consist of we have considered a
72
00:10:32,480 --> 00:10:40,240
matrices which are of real numbers so you
will get you know matrix norm of matrix into
73
00:10:40,240 --> 00:10:42,790
norm of matrix inverse as your condition number
74
00:10:42,790 --> 00:10:47,079
Here that is not going to happen your nonlinear
algebraic equations So the conditioning of
75
00:10:47,079 --> 00:10:52,990
a nonlinear algebraic system could be different
in different regions of the state space Suppose
76
00:10:52,990 --> 00:10:58,829
you are solving this is a abstract way of
putting it I will put it in the simple words
77
00:10:58,829 --> 00:11:06,449
let us say distillation column you are trying
to solve set of algebraic equation for a binary
78
00:11:06,449 --> 00:11:12,850
distillation column in a low purity region
as against in the high purity region okay
79
00:11:12,850 --> 00:11:19,009
Conditioning of these nonlinear algebraic
equations in low purity region okay will be
80
00:11:19,009 --> 00:11:23,850
different from conditioning of these nonlinear
algebraic equations in the high purity region
81
00:11:23,850 --> 00:11:28,399
It might be more difficult to solve for example
high purity region I am not saying it is always
82
00:11:28,399 --> 00:11:32,871
difficult but little bit it might be more
ill condition let us say and it is well condition
83
00:11:32,871 --> 00:11:34,860
when you are away from the high purity region
84
00:11:34,860 --> 00:11:42,100
So if you are trying to solve equations when
the purity is you know 0 99 as against purity
85
00:11:42,100 --> 00:11:49,379
to top purity as against top purity is 0 9
you will have different behavior of the nonlinear
86
00:11:49,379 --> 00:11:57,570
algebraic equations okay So the sensitivity
of the solution to a small change okay on
87
00:11:57,570 --> 00:12:04,800
the right hand side might be different in
different regions It depends upon where in
88
00:12:04,800 --> 00:12:12,929
the state space you are solving this set of
equations that is critical okay
89
00:12:12,929 --> 00:12:19,069
So this actually gives you an upper bound
on this ratio change in the or a perturbation
90
00:12:19,069 --> 00:12:27,100
fractional change in the solution to fractional
change in the input condition okay So analogous
91
00:12:27,100 --> 00:12:33,110
to the linear case one can define something
called a condition number here you can talk
92
00:12:33,110 --> 00:12:38,889
about well conditioned nonlinear systems ill
conditioned nonlinear systems you can talk
93
00:12:38,889 --> 00:12:43,509
about local you have to understand this is
local okay
94
00:12:43,509 --> 00:12:48,629
For a same nonlinear system it could be well
condition in some region it could be ill condition
95
00:12:48,629 --> 00:12:54,499
in some region okay So nonlinear algebraic
equations are much more difficult to handle
96
00:12:54,499 --> 00:13:05,459
in terms of conditioning than linear algebraic
systems So sensitivity what did actually condition
97
00:13:05,459 --> 00:13:11,589
number tell you Sensitivity of the solution
to errors for example okay
98
00:13:11,589 --> 00:13:18,160
So if nonlinear algebraic equations are you
know in some cases if the condition number
99
00:13:18,160 --> 00:13:25,749
is high which means the small change in u
will cause a large change in the solution
100
00:13:25,749 --> 00:13:34,029
x okay A small error in representation of
u will cause a large change in the solution
101
00:13:34,029 --> 00:13:43,119
x and just imagine when we are solving many
of these nonlinear algebraic equations arise
102
00:13:43,119 --> 00:13:51,860
because we are doing discretization of some
nonlinear boundary value problem or some partial
103
00:13:51,860 --> 00:13:52,860
difference equations
104
00:13:52,860 --> 00:14:01,170
When you doing that you are approximating
okay so in some regions a small perturbation
105
00:14:01,170 --> 00:14:08,379
in the input condition can lead to a large
change in the solution because of sensitivity
106
00:14:08,379 --> 00:14:13,589
of the equations in that region But this is
again as I said it is much more difficult
107
00:14:13,589 --> 00:15:04,639
to analyze this than the linear case The next
concept is we just touch up on this
108
00:15:04,639 --> 00:15:07,889
existence of solution and convergence of iteration
schemes
109
00:15:07,889 --> 00:15:14,920
Now you have seen that all the methods that
we have for solving nonlinear algebraic equations
110
00:15:14,920 --> 00:15:24,449
or iterative Quadratic multidimensional equations
can be solved analytically but I am not aware
111
00:15:24,449 --> 00:15:34,559
of solution for the cubic case so majority
of elements in the set of nonlinear algebraic
112
00:15:34,559 --> 00:15:41,059
equations cannot be solved analytically you
have to solve them using some numerical procedure
113
00:15:41,059 --> 00:15:43,980
okay
114
00:15:43,980 --> 00:15:59,299
Invariably any numerical scheme that you come
up with can be written in this form
115
00:15:59,299 --> 00:16:04,449
any numerical scheme that you come up with
okay You start with a guess generate a new
116
00:16:04,449 --> 00:16:11,249
guess x equal to where you want to reach finally
I want to reach finally to what is called
117
00:16:11,249 --> 00:16:25,730
the stationary point I want to reach to a
stationary point x star equal to G of x star
118
00:16:25,730 --> 00:16:37,790
This x star is called as stationary
119
00:16:37,790 --> 00:16:43,889
point it is called as fixed point okay
120
00:16:43,889 --> 00:16:48,009
So we want to actually reach here See for
example when you are solving f of x equal
121
00:16:48,009 --> 00:16:56,049
to 0 if you are solving using Newton Raphson
method okay or Newton’s method Newton’s
122
00:16:56,049 --> 00:17:20,189
method was x k plus 1 equal to xk minus doh
f by doh x at x equal to xk inverse f of right
123
00:17:20,189 --> 00:17:27,549
this was my Newton’s method I wanted to
solve for f of x equal to 0 Now G would be
124
00:17:27,549 --> 00:17:46,169
here in this case G is equivalent to x minus
doh f by doh x inverse fx this is my G of
125
00:17:46,169 --> 00:17:47,960
x okay
126
00:17:47,960 --> 00:18:02,440
And ultimately you are solving for x equal
to G of x right You are solving for x k plus
127
00:18:02,440 --> 00:18:10,429
1 equal to G of xk From the previous guess
you construct a new guess okay So any method
128
00:18:10,429 --> 00:18:14,929
that we are looked at till now for solving
nonlinear algebraic equations iterative method
129
00:18:14,929 --> 00:18:19,789
can be put into this generic form and you
are looking for x star x star is the fixed
130
00:18:19,789 --> 00:18:28,580
point okay I think the word stationary is
not really used here mostly
131
00:18:28,580 --> 00:18:36,970
Stationary point is used in the case of optimization
it is the fixed point So literature on function
132
00:18:36,970 --> 00:18:45,620
analysis will be full of fix point theorems
so doing analysis of iterative equations okay
133
00:18:45,620 --> 00:18:58,620
so how do the okay So now I am going to just
revisit some other terms that we looked at
134
00:18:58,620 --> 00:19:06,990
right in the beginning Banach space and operator
mapping Banach space to Banach space and so
135
00:19:06,990 --> 00:19:08,740
on okay
136
00:19:08,740 --> 00:19:16,820
Why I am worried about Banach space What is
the Banach space Banach space is one in which
137
00:19:16,820 --> 00:19:25,010
every sequence has a limit within the space
is convergence Why I am worried about every
138
00:19:25,010 --> 00:19:37,919
sequence looked here What is this If I am
start from some x not okay I will get a sequence
139
00:19:37,919 --> 00:19:47,080
of vectors x1 x2 x3 x4 x5 and so on This iterative
process we generate a sequence of vectors
140
00:19:47,080 --> 00:19:50,860
right
141
00:19:50,860 --> 00:20:00,950
Now if I give one particular problem okay
and if I ask him to solve the problem he will
142
00:20:00,950 --> 00:20:08,010
start with one x not she will start with another
x not she will start with another x not okay
143
00:20:08,010 --> 00:20:16,419
What is important is that if they are starting
from different initial guesses okay will those
144
00:20:16,419 --> 00:20:22,980
sequences converge to the same solution under
what condition First of all one condition
145
00:20:22,980 --> 00:20:29,330
or one primary condition is that the sequence
should not go to a limit which is outside
146
00:20:29,330 --> 00:20:32,559
the space right
147
00:20:32,559 --> 00:20:37,580
The sequence should remain within the space
that is the first condition Second condition
148
00:20:37,580 --> 00:20:43,140
that is important is that we want to know
is that whether the sequence will converge
149
00:20:43,140 --> 00:20:51,259
to a solution is the solution unique So does
the solution exist and is the solution that
150
00:20:51,259 --> 00:20:56,630
you get is that unique all these questions
are very very important okay
151
00:20:56,630 --> 00:21:02,720
So I am just going to give some hint about
how these are handled in the so in some sense
152
00:21:02,720 --> 00:21:10,419
this would connect to the cherry that we had
done in the beginning you know abstractions
153
00:21:10,419 --> 00:21:48,429
of Banach space and Hilbert space and so on
Now
154
00:21:48,429 --> 00:22:00,009
G is the mapping from x to x where x is the
Banach space or a complete normed linear space
155
00:22:00,009 --> 00:22:07,399
which means moment I say this I am ensuring
that the sequence generated from any initial
156
00:22:07,399 --> 00:22:12,570
guess x not will never leave the space will
always be within the space that is what I
157
00:22:12,570 --> 00:22:14,440
mean here okay
158
00:22:14,440 --> 00:22:44,639
The sequence will never leave the space An
important concept here is contraction mapping
159
00:22:44,639 --> 00:23:07,159
okay A very important concept here is contraction
mapping Now when I am writing here an operator
160
00:23:07,159 --> 00:23:14,059
G implicitly one which was define we have
just define it is also x is Banach space to
161
00:23:14,059 --> 00:23:55,650
Banach space all these things are implicit
I am not writing them on the board okay I
162
00:23:55,650 --> 00:24:40,779
will just I have to complete this definition
but before that let us look at what I have
163
00:24:40,779 --> 00:24:42,669
written here
164
00:24:42,669 --> 00:24:51,100
An operator G is called as a contraction mapping
of the closed ball okay A closed ball is set
165
00:24:51,100 --> 00:25:01,730
of all x belonging to the vector space x such
that x minus x not is less than r r is some
166
00:25:01,730 --> 00:25:06,880
radius okay How do you what is the relation
of this radius and convergence all that will
167
00:25:06,880 --> 00:25:17,070
come to soon but right now I am defining idea
of contraction okay on a small vision in the
168
00:25:17,070 --> 00:25:19,070
neighborhood of x not
169
00:25:19,070 --> 00:25:24,309
This is the way of defining the neighborhood
of x not some region around x not okay So
170
00:25:24,309 --> 00:25:28,880
which norm you use Depends upon you 1 norm
2 norm infinite norm it does not matter any
171
00:25:28,880 --> 00:25:35,960
norm that of your choice but I am defining
a region in the neighborhood of a initial
172
00:25:35,960 --> 00:25:40,860
guess okay What is x not here because we are
solving nonlinear algebraic equations we can
173
00:25:40,860 --> 00:25:43,380
look at x not as my initial guess okay
174
00:25:43,380 --> 00:25:47,880
It is not a fixed point as I said x not can
vary from person to person everyone can take
175
00:25:47,880 --> 00:25:56,539
a different guess okay Just pay attention
to these concepts because these are little
176
00:25:56,539 --> 00:26:00,460
difficult and then you are not the other things
which have been teaching at least you know
177
00:26:00,460 --> 00:26:06,100
something about it okay Whereas these are
little advance concept so you have to understand
178
00:26:06,100 --> 00:26:20,740
them carefully
179
00:26:20,740 --> 00:26:34,120
Now I want to call this mapping into a contraction
mapping if there exists a real number theta
180
00:26:34,120 --> 00:27:36,240
which is strictly less than 1 which is a positive
number strictly less than 1 such that
181
00:27:36,240 --> 00:27:45,490
okay So this completes my definition So when
do I call mapping G to be a contraction mapping
182
00:27:45,490 --> 00:27:59,129
okay If I pick up any two points x1 and x2
in this region okay and take difference between
183
00:27:59,129 --> 00:28:10,249
G of x1 and G of x2 that is always smaller
than x1 minus x2 which means if I draw it
184
00:28:10,249 --> 00:28:11,249
pictorially
185
00:28:11,249 --> 00:28:29,370
Let us say this is my x2 and x1 and this is
my x not initial guess and let us say this
186
00:28:29,370 --> 00:28:39,360
is the region
this is the ball in which I am defining the
187
00:28:39,360 --> 00:28:47,330
contraction mapping okay What I am going to
do is I am going to randomly pick any two
188
00:28:47,330 --> 00:29:01,660
points say here and here okay Now what is
G G is an operator which gives you element
189
00:29:01,660 --> 00:29:07,610
in the same set right G is the mapping from
x to x
190
00:29:07,610 --> 00:29:15,260
So if I apply G on one element it will give
you another element okay So let us say this
191
00:29:15,260 --> 00:29:35,289
is my x1 and this is x2 okay So when I apply
Gx1 see what is G x equal to G of x right
192
00:29:35,289 --> 00:29:40,940
This is the kind of equation we are solving
so when we apply G on x you get another x
193
00:29:40,940 --> 00:29:59,790
okay So let us say this gives me some element
x3 okay I pick up x2 and apply G of x2 this
194
00:29:59,790 --> 00:30:07,399
gives me say x4 okay
195
00:30:07,399 --> 00:30:19,419
Now
196
00:30:19,419 --> 00:30:33,720
we are concerned about this ratio that is
x3 minus x4 upon x1 minus x2 We are saying
197
00:30:33,720 --> 00:30:45,529
if this is less than theta which is less than
1 okay See I get two points let us say when
198
00:30:45,529 --> 00:30:58,299
I apply this I get x3 and when I apply G on
this I get x4 What we are saying is that this
199
00:30:58,299 --> 00:31:15,029
distance between x1 and x2 is larger than
x3 and x4 Sorry I should put norm here It
200
00:31:15,029 --> 00:31:23,549
is not we are working in multiple dimensions
I should put norm okay
201
00:31:23,549 --> 00:31:31,010
What I am saying here is that the distance
between any two points x1 x2 okay let us say
202
00:31:31,010 --> 00:31:41,049
x1 x2 this distance is always larger than
this distance This is x3 which was obtained
203
00:31:41,049 --> 00:31:51,909
by applying G on x1 This is x4 which was obtained
by applying G on x2 okay So this is my x3
204
00:31:51,909 --> 00:32:06,840
this is my x4 okay So if this condition holds
for any two x1 x2 inside this region okay
205
00:32:06,840 --> 00:32:16,899
which means when you apply G on x on any two
separate points okay then the relative distance
206
00:32:16,899 --> 00:32:17,899
contracts
207
00:32:17,899 --> 00:32:28,230
It comes close and it is called as a contraction
map is this clear okay Yeah “Professor student
208
00:32:28,230 --> 00:32:33,419
conversation starts” that is the good question
will come to that okay “Professor student
209
00:32:33,419 --> 00:32:40,450
conversation ends” So that will depend upon
how you are chose this radius and it is a
210
00:32:40,450 --> 00:32:46,950
very good question leading question I will
answer this question soon okay that actually
211
00:32:46,950 --> 00:32:55,490
forms the crucial it is very crucial to the
solution procedure the convergence of solution
212
00:32:55,490 --> 00:32:58,309
method
213
00:32:58,309 --> 00:33:03,960
So let us for the timing assumes that it lies
within the same ball let us assume for the
214
00:33:03,960 --> 00:33:13,960
time being Then every time you apply on any
two points the new two points that you generate
215
00:33:13,960 --> 00:33:19,889
x3 and x4 are closer than the initial two
points you take any two points apply G on
216
00:33:19,889 --> 00:33:23,929
the first point apply G on the second point
you get two new points okay that should be
217
00:33:23,929 --> 00:33:25,940
closer than x1 x2
218
00:33:25,940 --> 00:33:33,070
It should happen for any x1 x2 in this region
then G is called as contraction mapping on
219
00:33:33,070 --> 00:33:50,919
this ball u so this is my u x not r and she
has rightly guessed this critical point is
220
00:33:50,919 --> 00:34:02,470
what is this r will come to that Now in general
we are solving for x k plus 1 equal to G of
221
00:34:02,470 --> 00:34:12,270
xk it is quite likely that G is not a continuous
operator not a differentiable operator it
222
00:34:12,270 --> 00:34:16,770
could be continuous operator but not a differentiable
operator
223
00:34:16,770 --> 00:34:24,970
So actually this theory that has been derived
is not necessarily for all differentiable
224
00:34:24,970 --> 00:34:29,020
operator but if G is differentiable which
is the case in most of the chemical engineering
225
00:34:29,020 --> 00:34:44,110
situations then we can derive some nice conditions
226
00:34:44,110 --> 00:35:19,930
So this is the result well all the other things
hold that is G is the operator from Banach
227
00:35:19,930 --> 00:36:16,940
space to Banach space and it is differentiable
on this ball okay
228
00:36:16,940 --> 00:36:22,170
Well this makes it easier for you to understand
because derivative is something which you
229
00:36:22,170 --> 00:36:29,860
are more comfortable with So if the derivative
of G is norm of derivative of G is strictly
230
00:36:29,860 --> 00:36:46,780
less than 1 for every x belonging to this
is the very nice result It says that if the
231
00:36:46,780 --> 00:36:55,260
operator is differentiable okay then it is
a contraction mapping if and only if necessary
232
00:36:55,260 --> 00:36:59,630
and sufficient condition if and only if the
norm of the derivative is strictly less than
233
00:36:59,630 --> 00:37:03,350
1 okay
234
00:37:03,350 --> 00:37:12,470
So if the norm of the derivative is strictly
less than 1 in some region then it is well
235
00:37:12,470 --> 00:37:18,270
I have to check whether it is necessary and
sufficient I will confirm this If it is strictly
236
00:37:18,270 --> 00:37:23,670
less than 1 okay it is definitely a contraction
but if it is a contraction does not mean that
237
00:37:23,670 --> 00:37:33,160
norm has to be strictly less than 1 that we
have to check I am definitely sure that if
238
00:37:33,160 --> 00:38:05,250
part of it I will confirm this result
239
00:38:05,250 --> 00:38:14,500
So only if part is in doubt so if this is
strictly less than 1 okay Then it is surely
240
00:38:14,500 --> 00:38:24,400
a contraction So if the derivative has norm
strictly less than 1 we are guaranteed that
241
00:38:24,400 --> 00:38:33,390
So this part I am not too sure right now I
have to confirm okay How are you going to
242
00:38:33,390 --> 00:38:42,950
use this contraction mapping business The
literature on theoretical numerical analysis
243
00:38:42,950 --> 00:38:48,290
is full of what are called as fixed point
theorems
244
00:38:48,290 --> 00:38:55,690
They are worried about under what condition
the solutions to x equal to G of x exist under
245
00:38:55,690 --> 00:39:02,430
what condition iteration sequences will converge
to the solution The solutions are local first
246
00:39:02,430 --> 00:39:09,660
of all you understand that unlike linear algebraic
equations when A is nonsingular you have a
247
00:39:09,660 --> 00:39:17,870
unique solution right that is not a case in
nonlinear algebraic equations You can have
248
00:39:17,870 --> 00:39:21,570
multiple solutions to same set of nonlinear
algebraic equations
249
00:39:21,570 --> 00:39:31,250
Simplest example I have given you is you know
from the abstract this thing is eigenvalues
250
00:39:31,250 --> 00:39:41,100
When we looked at eigenvalue problem it was
a set of nonlinear algebraic equations and
251
00:39:41,100 --> 00:39:47,151
any how multiple solutions to that problem
Other example of course is CSTR a CSTR can
252
00:39:47,151 --> 00:39:52,641
have multiple steady states Under the same
input conditions it can have the steady state
253
00:39:52,641 --> 00:39:57,050
operating point and unsteady operating point
depending upon how the heat removal and heat
254
00:39:57,050 --> 00:39:59,580
generation terms are
255
00:39:59,580 --> 00:40:05,320
So same set of nonlinear algebraic equations
under identical input conditions can have
256
00:40:05,320 --> 00:40:10,850
multiple solutions okay So we are talking
about local convergence to local solutions
257
00:40:10,850 --> 00:40:19,630
We are talking about convergence inside a
ball okay this ball which is in the neighborhood
258
00:40:19,630 --> 00:40:27,350
of the initial guess okay Now let us try to
under this theorem this is the contraction
259
00:40:27,350 --> 00:40:50,860
mapping principle one of the fundamental results
in
260
00:40:50,860 --> 00:41:01,590
Now probably you can already guess norm of
an operator strictly less than 1 okay Then
261
00:41:01,590 --> 00:41:10,590
you get convergence We have seen something
similar to this what was that linear algebraic
262
00:41:10,590 --> 00:41:21,070
equations we were analyzing convergence of
iterative schemes and we said that induced
263
00:41:21,070 --> 00:41:32,190
norm is a upper bound on the set of you know
the lower bound of that is the spectral radius
264
00:41:32,190 --> 00:41:36,860
and so if norm is less than 1 norm of the
operator So the norm of the operator there
265
00:41:36,860 --> 00:41:41,260
was A okay
266
00:41:41,260 --> 00:41:48,710
Not A s inverse t now the operator there was
s inverse t and if now the operator s inverse
267
00:41:48,710 --> 00:41:55,790
t was less than 1 we were ensured convergence
So this is something like generalization so
268
00:41:55,790 --> 00:42:11,770
try to compare draw parallels then you will
understand these things better
269
00:42:11,770 --> 00:42:19,220
okay Now I am going to assume something which
she was suspecting okay The theorem assumes
270
00:42:19,220 --> 00:42:25,130
that G is the map which maps you into itself
271
00:42:25,130 --> 00:42:37,300
So which means you take any point inside this
u okay and apply G on it the resultant will
272
00:42:37,300 --> 00:42:59,750
also be inside u that is the first assumption
So actually choose r becomes very very critical
273
00:42:59,750 --> 00:44:01,910
because you know G has to map into itself
274
00:44:01,910 --> 00:44:50,380
okay Now here I am coming up with the condition
how I choose r okay Now see carefully you
275
00:44:50,380 --> 00:44:59,570
have this map G which is the contraction map
first of all G maps u into itself
276
00:44:59,570 --> 00:45:06,140
If I take any element in the set u G will
map it into itself So you will find a new
277
00:45:06,140 --> 00:45:11,060
element also inside u it is not going to be
different Second thing it is a contraction
278
00:45:11,060 --> 00:45:19,930
map okay which means you take any two points
in u and apply G to it okay The new point
279
00:45:19,930 --> 00:45:25,940
generator are going to be closer than the
two initial points any two points okay this
280
00:45:25,940 --> 00:45:38,050
is the second thing What should be the minimum
size of this ball Okay
281
00:45:38,050 --> 00:45:47,690
Looked carefully it is related to the first
x that you produce okay “Professor student
282
00:45:47,690 --> 00:45:52,800
conversation starts” Why this is related
to first x that you produce “Professor student
283
00:45:52,800 --> 00:46:02,190
conversation ends” See what should happen
is that if you take x1 x2 and x2 x3 okay x2
284
00:46:02,190 --> 00:46:11,230
x3 will be shorter than sorry should take
x0 x1 and x1 x2 because it is a contraction
285
00:46:11,230 --> 00:46:23,090
X1 x2 will be shorter than x0 and x1 The very
first x1 that you produce by applying so this
286
00:46:23,090 --> 00:46:33,910
is the okay should be greater than this in
some way it is related to distance x1 minus
287
00:46:33,910 --> 00:46:41,400
x0 How this factor comes you will have to
read the proof Why just 1 minus theta comes
288
00:46:41,400 --> 00:46:49,220
okay but you can appreciate that the radius
is related to the first if you start with
289
00:46:49,220 --> 00:46:54,300
x not the first x1 that you generate okay
that should be within the ball
290
00:46:54,300 --> 00:46:58,390
After that whatever you do will be within
the ball because it is a contraction okay
291
00:46:58,390 --> 00:47:08,960
It will stay within the ball okay
What next Then now if these conditions are
292
00:47:08,960 --> 00:47:40,970
satisfy then
293
00:47:40,970 --> 00:47:47,840
okay First thing that this theorem guarantee
is that G has a unique fix point inside the
294
00:47:47,840 --> 00:47:57,550
ball okay There exists a unique solution inside
the ball What is the solution of the problem
295
00:47:57,550 --> 00:48:02,560
The fix point You want to reach x star equal
to G of x star okay
296
00:48:02,560 --> 00:48:16,190
So this is the unique fixed point inside this
ball okay When the radius of the ball is chosen
297
00:48:16,190 --> 00:48:25,820
according to this condition okay this minimum
radius and when G is a contraction on this
298
00:48:25,820 --> 00:48:32,940
particular ball okay then we are guaranteed
that the solution exists inside the ball that
299
00:48:32,940 --> 00:48:48,070
exists one point okay where this condition
is satisfied okay Moreover with this ball
300
00:48:48,070 --> 00:48:52,000
is only one such point there are no two points
okay
301
00:48:52,000 --> 00:49:00,460
There is only one such point in which a unique
solution that is also Now the second part
302
00:49:00,460 --> 00:49:06,580
is very very important I will just continue
here Second and third part there are three
303
00:49:06,580 --> 00:49:41,560
parts for this result Second part says
304
00:49:41,560 --> 00:49:48,050
that if it is a contraction and if these conditions
are met then applying G repeatedly on the
305
00:49:48,050 --> 00:49:57,830
sequence will take you to the solution okay
that is guaranteed and at what rate you will
306
00:49:57,830 --> 00:50:00,420
go to the solution okay
307
00:50:00,420 --> 00:50:19,130
The distance between xk minus x star this
will reduce with theta to power k Again look
308
00:50:19,130 --> 00:50:26,900
at this result it says that the distance between
xk and x star will be shorter than distance
309
00:50:26,900 --> 00:50:34,470
between x not and x star This is the initial
distance you started with X star is let us
310
00:50:34,470 --> 00:50:43,351
say this is my x star This is the solution
I am starting with some x not here I want
311
00:50:43,351 --> 00:50:48,350
to reach here okay In doing so I might move
around you do not around how it will happen
312
00:50:48,350 --> 00:50:53,870
It is a nonlinear map you might move around
all over the set and then come back to the
313
00:50:53,870 --> 00:51:02,200
solution okay How the path is going to be
you do not know but what you know is that
314
00:51:02,200 --> 00:51:13,440
the initial distance okay Now how is this
result going to shrink rest to theta to power
315
00:51:13,440 --> 00:51:21,840
k Theta is the fraction So theta to power
k as k increases this distance will reduce
316
00:51:21,840 --> 00:51:29,350
okay If theta as you can appreciate if theta
is 0 99 okay rate at which you will go to
317
00:51:29,350 --> 00:51:31,280
x star will be slower
318
00:51:31,280 --> 00:51:38,310
If theta is 0 1 0 1 rise k will go to 0 very
very fast Iterations will converge very fast
319
00:51:38,310 --> 00:51:43,700
So what is the contraction constant we will
decide how fast you converge the solution
320
00:51:43,700 --> 00:52:10,580
okay So that is another message this theorem
gives The last message is very very important
321
00:52:10,580 --> 00:52:17,930
This is very very important message it says
that I do not have to start from x not See
322
00:52:17,930 --> 00:52:23,260
we were talking about here
323
00:52:23,260 --> 00:52:35,070
So this is my x star and say this is my s
not okay I am going to start my iterations
324
00:52:35,070 --> 00:52:49,980
only from x not if I happen to start my iterations
from some other x tilde not in the same ball
325
00:52:49,980 --> 00:52:54,180
okay As I said you know she might take a different
guess he might take a different guess he might
326
00:52:54,180 --> 00:52:59,790
take a different guess okay As long as those
guesses lie within this ball all those sequences
327
00:52:59,790 --> 00:53:05,220
will converge to the solution very very important
okay
328
00:53:05,220 --> 00:53:11,290
There is no unique initial guess If you are
in the region of convergence any initial guess
329
00:53:11,290 --> 00:53:57,720
in that if you give a good initial guess in
that region you are ensure to converge okay
330
00:53:57,720 --> 00:54:03,550
Sequence x tilde k generated by x tilde k
plus 1 equal to G of x tilde k starting from
331
00:54:03,550 --> 00:54:30,440
any x not belonging to this region okay Where
I just continue this here
332
00:54:30,440 --> 00:54:37,260
okay So if I were to start from any other
initial guess than x not okay
333
00:54:37,260 --> 00:54:48,850
As long as G is the contraction in this region
okay I am guaranteed that the sequence will
334
00:54:48,850 --> 00:54:55,321
converge okay All the concepts are important
Why Banach space Any sequence that you start
335
00:54:55,321 --> 00:55:00,960
from any initial guess should remain within
the space very very important okay Next thing
336
00:55:00,960 --> 00:55:10,320
is we have this operator which maps this ball
into itself Then it should be contraction
337
00:55:10,320 --> 00:55:11,320
okay
338
00:55:11,320 --> 00:55:14,380
If it is a contraction if all this conditions
are met these are sufficient conditions if
339
00:55:14,380 --> 00:55:20,660
this sufficient conditions are met you are
guaranteed to get convergence to the solution
340
00:55:20,660 --> 00:55:29,540
okay So this is the famous theorem called
contraction mapping principle or contraction
341
00:55:29,540 --> 00:55:35,160
mapping theorem There are many many variance
of this and I will just presented to you one
342
00:55:35,160 --> 00:55:41,380
particular variant which is easy to understand
and very very powerful
343
00:55:41,380 --> 00:55:46,900
We will just look at one or two examples briefly
in the next lecture and then move onto the
344
00:55:46,900 --> 00:55:50,770
next topic We cannot spend too much time on
this because I will have to take many lectures
345
00:55:50,770 --> 00:55:57,100
if I really go into prove in this theorem
getting more insights but what I want to do
346
00:55:57,100 --> 00:56:03,090
here by this one lecture is to just sensitize
you that you know how do you look at the convergence
347
00:56:03,090 --> 00:56:06,830
properties of nonlinear algebraic equations
okay
348
00:56:06,830 --> 00:56:14,620
One simple message that you can carry is that
look at the knob local Jacobian or G of x
349
00:56:14,620 --> 00:56:19,330
okay If that is not less than 1 maybe you
should try to make it less than 1 so that
350
00:56:19,330 --> 00:56:21,730
you know you can ensure convergence and so
on