1 00:00:00,520 --> 00:00:05,970 Welcome back, so we have been looking at recurrence relations. 2 00:00:05,970 --> 00:00:13,260 So what - the recurrence relation is a sequence of numbers where the initial set of numbers 3 00:00:13,260 --> 00:00:20,950 are given and the nth number or nth term in the sequence is given as a function of previous 4 00:00:20,950 --> 00:00:22,150 terms. 5 00:00:22,150 --> 00:00:29,820 Now, recurrence relation is extensively used for combinatorics, analysis of algorithms 6 00:00:29,820 --> 00:00:33,370 and in various other topics. 7 00:00:33,370 --> 00:00:39,480 We have seen how recurrence relations can be used to model various counting problems 8 00:00:39,480 --> 00:00:46,060 and we were been looking at how to solve this recurrence relations. 9 00:00:46,060 --> 00:00:52,580 So, some of the examples of recurrence relations that appear are something like T(1) equals 10 00:00:52,580 --> 00:01:01,300 to 1 and T(n) equals to 2 plus T(n minus 1) or T(1) equals to 1, T(2) equals to 3 and 11 00:01:01,300 --> 00:01:10,280 T(n) equals to T(n minus 1) plus T(n minus 2) or this one is what we get from the tower 12 00:01:10,280 --> 00:01:19,319 of Hanoi which is H(1) equals to 1, H(2) equals to 3 and H(n) equals to 2 times H(n minus 13 00:01:19,319 --> 00:01:20,799 1) plus 1. 14 00:01:20,799 --> 00:01:26,649 Then, we have the Fibonacci sequence which is F(1) equals to 1, F(2) equals to 1 and 15 00:01:26,649 --> 00:01:31,179 F(n) equals to F(n minus 1) plus F(n minus 2). 16 00:01:31,179 --> 00:01:36,939 Then we have something b(1) equals to 1 and b(n) equals to b (n over 2) plus 1, this is 17 00:01:36,939 --> 00:01:39,439 what we get from the binary search algorithm. 18 00:01:39,439 --> 00:01:45,979 And similarly we have this one that we get from the merge sort algorithm, M(1) equals 19 00:01:45,979 --> 00:01:54,609 to 1, M(n) equals to 2 times M( n over 2) plus n and the one that we get for Catalan 20 00:01:54,609 --> 00:02:01,509 number C (1) equals to 1 and C (n plus 1) equals to summation of i equals to 0 to n 21 00:02:01,509 --> 00:02:07,969 C(i) times C (n minus i), Now the question that comes up is that how do we solve this 22 00:02:07,969 --> 00:02:10,979 recurrence relations? 23 00:02:10,979 --> 00:02:14,160 So we have seen it somehow. 24 00:02:14,160 --> 00:02:19,950 The technique that we have looked at is first of all, guess the solution and prove using 25 00:02:19,950 --> 00:02:20,950 induction. 26 00:02:20,950 --> 00:02:26,870 So, in other words, if you correctly guess the solution then we can prove using induction 27 00:02:26,870 --> 00:02:29,420 that the guess is correct. 28 00:02:29,420 --> 00:02:35,890 The question is how do we guess the solution? 29 00:02:35,890 --> 00:02:44,570 So in the last two videos, we saw one particular technique namely we unfold the definitions 30 00:02:44,570 --> 00:02:48,870 and by unfolding definitions for example, we write T(1) equals to 1 and T(n) equals 31 00:02:48,870 --> 00:02:58,410 to 2 plus T(n minus 1), then we write T (n minus 1) as T (n minus 2) plus 2 and so on 32 00:02:58,410 --> 00:03:05,320 and after ith iteration I get T(n) in terms of some expressions in T(n minus i) and that 33 00:03:05,320 --> 00:03:16,800 helps us to kind of choose the right i for which we can get a full correct value of T(n) 34 00:03:16,800 --> 00:03:22,690 and by doing so, we could guess that T(n) equals to 2n minus 1. 35 00:03:22,690 --> 00:03:27,310 In the next one, we could guess that it is equal to n into (n plus 1) by 2 and in the 36 00:03:27,310 --> 00:03:33,830 Tower of Hanoi, we could guess that H(n) equals to 2 power n minus 1. 37 00:03:33,830 --> 00:03:42,970 Now, any of these cases, the idea is guess and then prove by induction. 38 00:03:42,970 --> 00:03:49,880 Now, problem is that many times guessing can become quite tricky. 39 00:03:49,880 --> 00:03:57,280 For example, if we have a Fibonacci expression like this, F(1) equals to 1 and F(2) equals 40 00:03:57,280 --> 00:04:02,810 1 and F(n) equals to F(n minus 1) plus F(n minus 2). 41 00:04:02,810 --> 00:04:10,020 Now the guessing the value of F(n) is quite complicated because the final value of F(n), 42 00:04:10,020 --> 00:04:16,030 the actual value is something as complicated as this, 1 plus square root 5 by 2 whole power 43 00:04:16,030 --> 00:04:24,069 n minus 1 minus square root 5 by 2 whole power n whole divided by square root 5. 44 00:04:24,069 --> 00:04:28,830 You can clearly understand that guessing this value is not an easy job. 45 00:04:28,830 --> 00:04:37,199 Sometimes, we have expressions like this, b(1) equals 1 and b(n) equals to b (n over 46 00:04:37,199 --> 00:04:46,790 2) plus 1 but because of this ceiling here, we can say that getting a simple clean expression 47 00:04:46,790 --> 00:04:50,680 for b(n) is not an easy job. 48 00:04:50,680 --> 00:04:56,389 In fact, there does not exists a neat guess or nice guess that exists. 49 00:04:56,389 --> 00:05:01,210 So how do we go about solving them? 50 00:05:01,210 --> 00:05:10,090 Let us look at an example, so here is an example that occurs in the merge sort algorithm, M(1) 51 00:05:10,090 --> 00:05:17,539 equals to 1 and M(n) equals to M(n over 2) plus M(n over 2) plus n where the first one 52 00:05:17,539 --> 00:05:21,500 is a ceiling, second one is a floor. 53 00:05:21,500 --> 00:05:22,800 What does the ceiling means? 54 00:05:22,800 --> 00:05:31,159 So if it is this means that this is the smallest integer that is bigger than n over 2 and this 55 00:05:31,159 --> 00:05:38,499 one is the largest integer that is less than n over 2. 56 00:05:38,499 --> 00:05:49,199 So, if n is even then both of these, n over 2 by 2 is equals to n over 2 which is equals 57 00:05:49,199 --> 00:06:10,411 to just this n over 2 but if n is odd then n over 2, sorry, n over 2 is actually n plus 58 00:06:10,411 --> 00:06:27,940 1 over 2 and n over 2 is n minus 1 over 2 and as you can see that n is equals to n over 59 00:06:27,940 --> 00:06:31,630 2 plus n over 2. 60 00:06:31,630 --> 00:06:41,249 Now this is the definition of these two expressions of floor and ceiling. 61 00:06:41,249 --> 00:06:45,900 Now, how do we - Can you guess them. 62 00:06:45,900 --> 00:06:48,419 Can I guess anything? 63 00:06:48,419 --> 00:06:54,090 Of course, one of technique is to unfold it. 64 00:06:54,090 --> 00:06:56,729 Just to unfold it. 65 00:06:56,729 --> 00:07:13,469 So M(n) equals to M(n over 2) plus M(n over 2) plus n which is again, I can write this 66 00:07:13,469 --> 00:07:31,759 one as n over 2 plus M floor of floor n over 2 by 2 plus M by 2 plus something, something, 67 00:07:31,759 --> 00:07:33,180 something. 68 00:07:33,180 --> 00:07:46,110 Already you see this expression is becoming very ugly, not much that we can do about it. 69 00:07:46,110 --> 00:07:52,020 The reason is that we cannot simplify this, even if n is even okay, in that case I do 70 00:07:52,020 --> 00:07:56,690 not have to write this one I can, this is just plain n over 2. 71 00:07:56,690 --> 00:08:12,909 But then here this will become n over 2 by 2 ceiling of that and when this n over 2 by 72 00:08:12,909 --> 00:08:18,819 2 ceiling, it is only when n over 2 is even or in other words, n is a multiple of 4. 73 00:08:18,819 --> 00:08:27,520 So, in other words, the only way I can simplify this whole expression is when n is a power 74 00:08:27,520 --> 00:08:30,229 of 2. 75 00:08:30,229 --> 00:08:44,190 So if we can guess, question is that can we guess this number M(n), when n is of a particular 76 00:08:44,190 --> 00:08:48,430 type and idea is that what if n is equals to 2 power k. 77 00:08:48,430 --> 00:08:49,710 So let us see what happened. 78 00:08:49,710 --> 00:08:57,650 If n equals to 2 power k, the good thing is that both of them is just n over 2. 79 00:08:57,650 --> 00:08:58,730 Right? 80 00:08:58,730 --> 00:09:14,280 So I have M(n) equals to so I get 2 times M n over 2 plus n which is equals to 2 times 81 00:09:14,280 --> 00:09:20,760 M, now, n over 2 is also a power of 2 hence, also a even number. 82 00:09:20,760 --> 00:09:29,470 And hence I can just again apply this one and I get n over 4 plus n over 2 plus n which 83 00:09:29,470 --> 00:09:35,120 is 2 times M n over 4 plus 2n. 84 00:09:35,120 --> 00:09:39,310 What is the next one? 85 00:09:39,310 --> 00:09:42,970 Next one is 2 times this, so again 2 times; sorry I made a mistake here. 86 00:09:42,970 --> 00:09:52,910 It should be 2, so this should be 4. 87 00:09:52,910 --> 00:09:55,720 Right? 88 00:09:55,720 --> 00:10:00,360 So this should be 4. 89 00:10:00,360 --> 00:10:05,520 So what do I have again? 90 00:10:05,520 --> 00:10:28,050 4 times when I expand this Mn over 4, I get 2 times M n over 8 plus n over 4 plus 2n which 91 00:10:28,050 --> 00:10:38,270 is- again you can see here becomes 8 M n over 8 plus 3n. 92 00:10:38,270 --> 00:10:48,310 So, in other words, if I repeat it again and again and again after ith time I will get 93 00:10:48,310 --> 00:10:59,820 2 power i M n over 2 power i plus i times n, this 3 is 2 power 3. 94 00:10:59,820 --> 00:11:08,780 And now the idea is that if I have put i equals to log n, so if i equals to k. 95 00:11:08,780 --> 00:11:16,680 If i equals to k then I have n equals to 2 power k, in that case, if i equals to k, what 96 00:11:16,680 --> 00:11:19,300 do I get? 97 00:11:19,300 --> 00:11:30,830 I get this term to become 2 power k plus k times n right? 98 00:11:30,830 --> 00:11:36,650 Which is, this is a, this is k plus 1 times n. 99 00:11:36,650 --> 00:11:52,070 So in other words, we have been able to do it or guess it for a certain class of n which 100 00:11:52,070 --> 00:12:00,150 is namely for n equals to 2 power k, we could then get it to be nth of power k which is 101 00:12:00,150 --> 00:12:02,380 n power k. 102 00:12:02,380 --> 00:12:14,180 So this should be plus 1 as , n log n base 2 plus 1. 103 00:12:14,180 --> 00:12:15,320 Good. 104 00:12:15,320 --> 00:12:20,640 So at least for some class of n, we can get. 105 00:12:20,640 --> 00:12:30,320 Question that comes now is that can we say that this is the right expression for all 106 00:12:30,320 --> 00:12:38,820 n and the fact is that possibly not. 107 00:12:38,820 --> 00:12:43,530 Because for normal n there should be a floor and the ceiling which will just make this 108 00:12:43,530 --> 00:12:48,260 whole life more complicated, at least ugly to do the calculation. 109 00:12:48,260 --> 00:13:01,030 Since that we ask - can we get upper bound, say something like to 2n log n. 110 00:13:01,030 --> 00:13:15,000 And I leave it to you prove that, prove using induction that M(n) is indeed bigger than 111 00:13:15,000 --> 00:13:33,180 2n log n for n bigger than 5. 112 00:13:33,180 --> 00:13:38,880 Okay? 113 00:13:38,880 --> 00:13:46,090 And so we can have prove that. 114 00:13:46,090 --> 00:13:47,280 But, this is just a upper bound. 115 00:13:47,280 --> 00:13:53,450 Can we come up with a matching lower bound and again it is so turns out that we can, 116 00:13:53,450 --> 00:14:00,590 we can prove that in lower bound it n over 2 log n. 117 00:14:00,590 --> 00:14:15,260 Again, this is I leave it to an exercise, prove using induction, M of n is bigger than 118 00:14:15,260 --> 00:14:18,780 n over 2 log 2 base n. 119 00:14:18,780 --> 00:14:27,670 So what we have – so in fact, although we could not guess the actual value of M(n), 120 00:14:27,670 --> 00:14:41,830 we first guess M(n) for some class of n namely powers of 2 and then we could say that for 121 00:14:41,830 --> 00:14:54,970 all n greater than or equal to 5, M(n) is lower bounded and upper bounded by two terms. 122 00:14:54,970 --> 00:15:02,950 It is upper bounded by 2 log n and lower bounded by n over 2 log n. 123 00:15:02,950 --> 00:15:08,620 So why we could not come up with the exact formula for M(n)? 124 00:15:08,620 --> 00:15:15,590 We could come up with the inequality, upper bound and lower bound where the difference 125 00:15:15,590 --> 00:15:16,590 between them is not too much. 126 00:15:16,590 --> 00:15:21,190 The upper bound is just 4 times the lower bound. 127 00:15:21,190 --> 00:15:29,970 So in many times, particularly for this example, so this is the number of steps required for 128 00:15:29,970 --> 00:15:34,340 made by a particular program namely the merge sort. 129 00:15:34,340 --> 00:15:39,790 And when we have looking at the number of steps, we have not yet worried about two things. 130 00:15:39,790 --> 00:15:43,190 First of all, what happens for small n. 131 00:15:43,190 --> 00:15:50,300 So we are only worried about big n, where the input size is big what happens, so this 132 00:15:50,300 --> 00:15:51,300 is what is called as asymptotic. 133 00:15:51,300 --> 00:15:53,190 So, as n increases, what happens? 134 00:15:53,190 --> 00:16:02,360 For small n, we do not care and the second thing is that we really do not care whether 135 00:16:02,360 --> 00:16:04,730 the bounds are tight or not. 136 00:16:04,730 --> 00:16:11,630 In fact, as long as, we understand that the value that we are looking for is within a 137 00:16:11,630 --> 00:16:16,080 constant fraction of something else, we are happy. 138 00:16:16,080 --> 00:16:26,530 So, in other words, of course, the question is that is always good to come up with a exact 139 00:16:26,530 --> 00:16:33,600 answer but if you cannot, we are usually happy, if I get an answer which is constant factor 140 00:16:33,600 --> 00:16:34,600 away. 141 00:16:34,600 --> 00:16:42,180 I will formalize all these things in the next class or next video, when we will be looking 142 00:16:42,180 --> 00:16:50,700 at what we know called asymptotic notations and using that we will formalize when we say 143 00:16:50,700 --> 00:16:56,990 that this is a pretty good estimate. 144 00:16:56,990 --> 00:16:57,370 Thank you.