1
00:00:00,520 --> 00:00:05,970
Welcome back, so we have been looking at recurrence
relations.
2
00:00:05,970 --> 00:00:13,260
So what - the recurrence relation is a sequence
of numbers where the initial set of numbers
3
00:00:13,260 --> 00:00:20,950
are given and the nth number or nth term in
the sequence is given as a function of previous
4
00:00:20,950 --> 00:00:22,150
terms.
5
00:00:22,150 --> 00:00:29,820
Now, recurrence relation is extensively used
for combinatorics, analysis of algorithms
6
00:00:29,820 --> 00:00:33,370
and in various other topics.
7
00:00:33,370 --> 00:00:39,480
We have seen how recurrence relations can
be used to model various counting problems
8
00:00:39,480 --> 00:00:46,060
and we were been looking at how to solve this
recurrence relations.
9
00:00:46,060 --> 00:00:52,580
So, some of the examples of recurrence relations
that appear are something like T(1) equals
10
00:00:52,580 --> 00:01:01,300
to 1 and T(n) equals to 2 plus T(n minus 1)
or T(1) equals to 1, T(2) equals to 3 and
11
00:01:01,300 --> 00:01:10,280
T(n) equals to T(n minus 1) plus T(n minus
2) or this one is what we get from the tower
12
00:01:10,280 --> 00:01:19,319
of Hanoi which is H(1) equals to 1, H(2) equals
to 3 and H(n) equals to 2 times H(n minus
13
00:01:19,319 --> 00:01:20,799
1) plus 1.
14
00:01:20,799 --> 00:01:26,649
Then, we have the Fibonacci sequence which
is F(1) equals to 1, F(2) equals to 1 and
15
00:01:26,649 --> 00:01:31,179
F(n) equals to F(n minus 1) plus F(n minus
2).
16
00:01:31,179 --> 00:01:36,939
Then we have something b(1) equals to 1 and
b(n) equals to b (n over 2) plus 1, this is
17
00:01:36,939 --> 00:01:39,439
what we get from the binary search algorithm.
18
00:01:39,439 --> 00:01:45,979
And similarly we have this one that we get
from the merge sort algorithm, M(1) equals
19
00:01:45,979 --> 00:01:54,609
to 1, M(n) equals to 2 times M( n over 2)
plus n and the one that we get for Catalan
20
00:01:54,609 --> 00:02:01,509
number C (1) equals to 1 and C (n plus 1)
equals to summation of i equals to 0 to n
21
00:02:01,509 --> 00:02:07,969
C(i) times C (n minus i), Now the question
that comes up is that how do we solve this
22
00:02:07,969 --> 00:02:10,979
recurrence relations?
23
00:02:10,979 --> 00:02:14,160
So we have seen it somehow.
24
00:02:14,160 --> 00:02:19,950
The technique that we have looked at is first
of all, guess the solution and prove using
25
00:02:19,950 --> 00:02:20,950
induction.
26
00:02:20,950 --> 00:02:26,870
So, in other words, if you correctly guess
the solution then we can prove using induction
27
00:02:26,870 --> 00:02:29,420
that the guess is correct.
28
00:02:29,420 --> 00:02:35,890
The question is how do we guess the solution?
29
00:02:35,890 --> 00:02:44,570
So in the last two videos, we saw one particular
technique namely we unfold the definitions
30
00:02:44,570 --> 00:02:48,870
and by unfolding definitions for example,
we write T(1) equals to 1 and T(n) equals
31
00:02:48,870 --> 00:02:58,410
to 2 plus T(n minus 1), then we write T (n
minus 1) as T (n minus 2) plus 2 and so on
32
00:02:58,410 --> 00:03:05,320
and after ith iteration I get T(n) in terms
of some expressions in T(n minus i) and that
33
00:03:05,320 --> 00:03:16,800
helps us to kind of choose the right i for
which we can get a full correct value of T(n)
34
00:03:16,800 --> 00:03:22,690
and by doing so, we could guess that T(n)
equals to 2n minus 1.
35
00:03:22,690 --> 00:03:27,310
In the next one, we could guess that it is
equal to n into (n plus 1) by 2 and in the
36
00:03:27,310 --> 00:03:33,830
Tower of Hanoi, we could guess that H(n) equals
to 2 power n minus 1.
37
00:03:33,830 --> 00:03:42,970
Now, any of these cases, the idea is guess
and then prove by induction.
38
00:03:42,970 --> 00:03:49,880
Now, problem is that many times guessing can
become quite tricky.
39
00:03:49,880 --> 00:03:57,280
For example, if we have a Fibonacci expression
like this, F(1) equals to 1 and F(2) equals
40
00:03:57,280 --> 00:04:02,810
1 and F(n) equals to F(n minus 1) plus F(n
minus 2).
41
00:04:02,810 --> 00:04:10,020
Now the guessing the value of F(n) is quite
complicated because the final value of F(n),
42
00:04:10,020 --> 00:04:16,030
the actual value is something as complicated
as this, 1 plus square root 5 by 2 whole power
43
00:04:16,030 --> 00:04:24,069
n minus 1 minus square root 5 by 2 whole power
n whole divided by square root 5.
44
00:04:24,069 --> 00:04:28,830
You can clearly understand that guessing this
value is not an easy job.
45
00:04:28,830 --> 00:04:37,199
Sometimes, we have expressions like this,
b(1) equals 1 and b(n) equals to b (n over
46
00:04:37,199 --> 00:04:46,790
2) plus 1 but because of this ceiling here,
we can say that getting a simple clean expression
47
00:04:46,790 --> 00:04:50,680
for b(n) is not an easy job.
48
00:04:50,680 --> 00:04:56,389
In fact, there does not exists a neat guess
or nice guess that exists.
49
00:04:56,389 --> 00:05:01,210
So how do we go about solving them?
50
00:05:01,210 --> 00:05:10,090
Let us look at an example, so here is an example
that occurs in the merge sort algorithm, M(1)
51
00:05:10,090 --> 00:05:17,539
equals to 1 and M(n) equals to M(n over 2)
plus M(n over 2) plus n where the first one
52
00:05:17,539 --> 00:05:21,500
is a ceiling, second one is a floor.
53
00:05:21,500 --> 00:05:22,800
What does the ceiling means?
54
00:05:22,800 --> 00:05:31,159
So if it is this means that this is the smallest
integer that is bigger than n over 2 and this
55
00:05:31,159 --> 00:05:38,499
one is the largest integer that is less than
n over 2.
56
00:05:38,499 --> 00:05:49,199
So, if n is even then both of these, n over
2 by 2 is equals to n over 2 which is equals
57
00:05:49,199 --> 00:06:10,411
to just this n over 2 but if n is odd then
n over 2, sorry, n over 2 is actually n plus
58
00:06:10,411 --> 00:06:27,940
1 over 2 and n over 2 is n minus 1 over 2
and as you can see that n is equals to n over
59
00:06:27,940 --> 00:06:31,630
2 plus n over 2.
60
00:06:31,630 --> 00:06:41,249
Now this is the definition of these two expressions
of floor and ceiling.
61
00:06:41,249 --> 00:06:45,900
Now, how do we - Can you guess them.
62
00:06:45,900 --> 00:06:48,419
Can I guess anything?
63
00:06:48,419 --> 00:06:54,090
Of course, one of technique is to unfold it.
64
00:06:54,090 --> 00:06:56,729
Just to unfold it.
65
00:06:56,729 --> 00:07:13,469
So M(n) equals to M(n over 2) plus M(n over
2) plus n which is again, I can write this
66
00:07:13,469 --> 00:07:31,759
one as n over 2 plus M floor of floor n over
2 by 2 plus M by 2 plus something, something,
67
00:07:31,759 --> 00:07:33,180
something.
68
00:07:33,180 --> 00:07:46,110
Already you see this expression is becoming
very ugly, not much that we can do about it.
69
00:07:46,110 --> 00:07:52,020
The reason is that we cannot simplify this,
even if n is even okay, in that case I do
70
00:07:52,020 --> 00:07:56,690
not have to write this one I can, this is
just plain n over 2.
71
00:07:56,690 --> 00:08:12,909
But then here this will become n over 2 by
2 ceiling of that and when this n over 2 by
72
00:08:12,909 --> 00:08:18,819
2 ceiling, it is only when n over 2 is even
or in other words, n is a multiple of 4.
73
00:08:18,819 --> 00:08:27,520
So, in other words, the only way I can simplify
this whole expression is when n is a power
74
00:08:27,520 --> 00:08:30,229
of 2.
75
00:08:30,229 --> 00:08:44,190
So if we can guess, question is that can we
guess this number M(n), when n is of a particular
76
00:08:44,190 --> 00:08:48,430
type and idea is that what if n is equals
to 2 power k.
77
00:08:48,430 --> 00:08:49,710
So let us see what happened.
78
00:08:49,710 --> 00:08:57,650
If n equals to 2 power k, the good thing is
that both of them is just n over 2.
79
00:08:57,650 --> 00:08:58,730
Right?
80
00:08:58,730 --> 00:09:14,280
So I have M(n) equals to so I get 2 times
M n over 2 plus n which is equals to 2 times
81
00:09:14,280 --> 00:09:20,760
M, now, n over 2 is also a power of 2 hence,
also a even number.
82
00:09:20,760 --> 00:09:29,470
And hence I can just again apply this one
and I get n over 4 plus n over 2 plus n which
83
00:09:29,470 --> 00:09:35,120
is 2 times M n over 4 plus 2n.
84
00:09:35,120 --> 00:09:39,310
What is the next one?
85
00:09:39,310 --> 00:09:42,970
Next one is 2 times this, so again 2 times;
sorry I made a mistake here.
86
00:09:42,970 --> 00:09:52,910
It should be 2, so this should be 4.
87
00:09:52,910 --> 00:09:55,720
Right?
88
00:09:55,720 --> 00:10:00,360
So this should be 4.
89
00:10:00,360 --> 00:10:05,520
So what do I have again?
90
00:10:05,520 --> 00:10:28,050
4 times when I expand this Mn over 4, I get
2 times M n over 8 plus n over 4 plus 2n which
91
00:10:28,050 --> 00:10:38,270
is- again you can see here becomes 8 M n over
8 plus 3n.
92
00:10:38,270 --> 00:10:48,310
So, in other words, if I repeat it again and
again and again after ith time I will get
93
00:10:48,310 --> 00:10:59,820
2 power i M n over 2 power i plus i times
n, this 3 is 2 power 3.
94
00:10:59,820 --> 00:11:08,780
And now the idea is that if I have put i equals
to log n, so if i equals to k.
95
00:11:08,780 --> 00:11:16,680
If i equals to k then I have n equals to 2
power k, in that case, if i equals to k, what
96
00:11:16,680 --> 00:11:19,300
do I get?
97
00:11:19,300 --> 00:11:30,830
I get this term to become 2 power k plus k
times n right?
98
00:11:30,830 --> 00:11:36,650
Which is, this is a, this is k plus 1 times
n.
99
00:11:36,650 --> 00:11:52,070
So in other words, we have been able to do
it or guess it for a certain class of n which
100
00:11:52,070 --> 00:12:00,150
is namely for n equals to 2 power k, we could
then get it to be nth of power k which is
101
00:12:00,150 --> 00:12:02,380
n power k.
102
00:12:02,380 --> 00:12:14,180
So this should be plus 1 as , n log n base
2 plus 1.
103
00:12:14,180 --> 00:12:15,320
Good.
104
00:12:15,320 --> 00:12:20,640
So at least for some class of n, we can get.
105
00:12:20,640 --> 00:12:30,320
Question that comes now is that can we say
that this is the right expression for all
106
00:12:30,320 --> 00:12:38,820
n and the fact is that possibly not.
107
00:12:38,820 --> 00:12:43,530
Because for normal n there should be a floor
and the ceiling which will just make this
108
00:12:43,530 --> 00:12:48,260
whole life more complicated, at least ugly
to do the calculation.
109
00:12:48,260 --> 00:13:01,030
Since that we ask - can we get upper bound,
say something like to 2n log n.
110
00:13:01,030 --> 00:13:15,000
And I leave it to you prove that, prove using
induction that M(n) is indeed bigger than
111
00:13:15,000 --> 00:13:33,180
2n log n for n bigger than 5.
112
00:13:33,180 --> 00:13:38,880
Okay?
113
00:13:38,880 --> 00:13:46,090
And so we can have prove that.
114
00:13:46,090 --> 00:13:47,280
But, this is just a upper bound.
115
00:13:47,280 --> 00:13:53,450
Can we come up with a matching lower bound
and again it is so turns out that we can,
116
00:13:53,450 --> 00:14:00,590
we can prove that in lower bound it n over
2 log n.
117
00:14:00,590 --> 00:14:15,260
Again, this is I leave it to an exercise,
prove using induction, M of n is bigger than
118
00:14:15,260 --> 00:14:18,780
n over 2 log 2 base n.
119
00:14:18,780 --> 00:14:27,670
So what we have – so in fact, although we
could not guess the actual value of M(n),
120
00:14:27,670 --> 00:14:41,830
we first guess M(n) for some class of n namely
powers of 2 and then we could say that for
121
00:14:41,830 --> 00:14:54,970
all n greater than or equal to 5, M(n) is
lower bounded and upper bounded by two terms.
122
00:14:54,970 --> 00:15:02,950
It is upper bounded by 2 log n and lower bounded
by n over 2 log n.
123
00:15:02,950 --> 00:15:08,620
So why we could not come up with the exact
formula for M(n)?
124
00:15:08,620 --> 00:15:15,590
We could come up with the inequality, upper
bound and lower bound where the difference
125
00:15:15,590 --> 00:15:16,590
between them is not too much.
126
00:15:16,590 --> 00:15:21,190
The upper bound is just 4 times the lower
bound.
127
00:15:21,190 --> 00:15:29,970
So in many times, particularly for this example,
so this is the number of steps required for
128
00:15:29,970 --> 00:15:34,340
made by a particular program namely the merge
sort.
129
00:15:34,340 --> 00:15:39,790
And when we have looking at the number of
steps, we have not yet worried about two things.
130
00:15:39,790 --> 00:15:43,190
First of all, what happens for small n.
131
00:15:43,190 --> 00:15:50,300
So we are only worried about big n, where
the input size is big what happens, so this
132
00:15:50,300 --> 00:15:51,300
is what is called as asymptotic.
133
00:15:51,300 --> 00:15:53,190
So, as n increases, what happens?
134
00:15:53,190 --> 00:16:02,360
For small n, we do not care and the second
thing is that we really do not care whether
135
00:16:02,360 --> 00:16:04,730
the bounds are tight or not.
136
00:16:04,730 --> 00:16:11,630
In fact, as long as, we understand that the
value that we are looking for is within a
137
00:16:11,630 --> 00:16:16,080
constant fraction of something else, we are
happy.
138
00:16:16,080 --> 00:16:26,530
So, in other words, of course, the question
is that is always good to come up with a exact
139
00:16:26,530 --> 00:16:33,600
answer but if you cannot, we are usually happy,
if I get an answer which is constant factor
140
00:16:33,600 --> 00:16:34,600
away.
141
00:16:34,600 --> 00:16:42,180
I will formalize all these things in the next
class or next video, when we will be looking
142
00:16:42,180 --> 00:16:50,700
at what we know called asymptotic notations
and using that we will formalize when we say
143
00:16:50,700 --> 00:16:56,990
that this is a pretty good estimate.
144
00:16:56,990 --> 00:16:57,370
Thank you.