1 00:00:00,229 --> 00:00:01,290 Welcome Back. 2 00:00:01,290 --> 00:00:04,509 So we have been looking at counting problems. 3 00:00:04,509 --> 00:00:11,199 In particular, we have been looking at problems in Combinatorics which is a branch of mathematics 4 00:00:11,199 --> 00:00:14,279 that involve counting. 5 00:00:14,279 --> 00:00:26,029 So the typical question that one ask is given a set what is the cardinality of the set S. 6 00:00:26,029 --> 00:00:30,419 Now the question is that how is the set given? 7 00:00:30,419 --> 00:00:39,170 Of course the set is given explicitly then counting the number of elements in S is quite 8 00:00:39,170 --> 00:00:40,260 simple. 9 00:00:40,260 --> 00:00:47,719 But most of the time the set is described in words or described in some other way. 10 00:00:47,719 --> 00:00:53,719 And in that case the set is basically given implicitly and counting the number of elements 11 00:00:53,719 --> 00:00:58,800 in the set is a pretty challenging problem. 12 00:00:58,800 --> 00:01:12,220 For example, how many elements are there satisfying a particular set of conditions or equivalently 13 00:01:12,220 --> 00:01:18,120 how many ways can you draw an element from a set of universe satisfying a certain set 14 00:01:18,120 --> 00:01:19,120 of conditions. 15 00:01:19,120 --> 00:01:26,180 So these are the kind of questions that usually we will look at. 16 00:01:26,180 --> 00:01:32,100 So we have been looking at a few examples, for example how many n digit numbers are there 17 00:01:32,100 --> 00:01:36,490 where no consecutive digits are same? 18 00:01:36,490 --> 00:01:42,970 How many non-decreasing functions are there from one to n to one to k? 19 00:01:42,970 --> 00:01:48,190 How can you distribute n identical toffees among k kids? 20 00:01:48,190 --> 00:01:56,170 And the finally how many 0, 1 strings are their of length n which does not have any 21 00:01:56,170 --> 00:02:00,350 consecutive zeros. 22 00:02:00,350 --> 00:02:10,859 Now, the problem with counting is that every problem is unique and it requires a different 23 00:02:10,859 --> 00:02:12,769 technique to solve it. 24 00:02:12,769 --> 00:02:17,840 It is one of the most challenging subjects in mathematics. 25 00:02:17,840 --> 00:02:22,390 Some big names like Srinivasan Ramanujan also worked on counting. 26 00:02:22,390 --> 00:02:30,620 And there are some handy tricks and tools to attack but they are just some kind of a 27 00:02:30,620 --> 00:02:39,879 small tools it does not exactly give a very standard way of solving all the problems. 28 00:02:39,879 --> 00:02:50,069 So once these tools are fixed for this particular special case when we call see how many ways 29 00:02:50,069 --> 00:02:55,090 can we select k objects from n objects. 30 00:02:55,090 --> 00:03:01,499 So there are two different cases that we have to take in number one is whether the objects 31 00:03:01,499 --> 00:03:05,569 are – whether repetition is allowed. 32 00:03:05,569 --> 00:03:09,920 In other word, can I pick a same object from the n objects multiple times? 33 00:03:09,920 --> 00:03:19,920 And the second case that we should look at is whether the ordering which the elements 34 00:03:19,920 --> 00:03:21,090 are picked matter. 35 00:03:21,090 --> 00:03:32,879 And this gives us the four cases and we have seen how to solve these four cases. 36 00:03:32,879 --> 00:03:40,209 Another problem of this kind is how many ways can one distribute n balls into k bins? 37 00:03:40,209 --> 00:03:45,150 And there are certain cases which to be handled, for example whether the bins are distinguishable, 38 00:03:45,150 --> 00:03:46,930 whether the balls are distinguishable. 39 00:03:46,930 --> 00:03:52,110 If the balls are distinguishable does the ordering in the bins matter? 40 00:03:52,110 --> 00:04:00,090 Can some of the bins be empty and are there some other restrictions? 41 00:04:00,090 --> 00:04:07,609 And here also we kind of solve them or kind of give you the idea of most of the various 42 00:04:07,609 --> 00:04:15,390 cases except for this particular case which is a P(n, k) which is a pretty complicated 43 00:04:15,390 --> 00:04:24,380 case by itself and we saw it last video back this is one of the—these are very challenging 44 00:04:24,380 --> 00:04:30,220 problem something that Srinivasan Ramanujan had worked on. 45 00:04:30,220 --> 00:04:38,730 Now using all these tools and tricks we did look at these problems and we got to see how 46 00:04:38,730 --> 00:04:40,870 to solve these top three problems. 47 00:04:40,870 --> 00:04:47,050 Now the question that I have is how to solve the last problem. 48 00:04:47,050 --> 00:04:58,330 Namely number of 0,1 strings of length n which does not have any consecutive zero. 49 00:04:58,330 --> 00:05:02,070 Right? 50 00:05:02,070 --> 00:05:12,270 Now you can try it out by yourself and realize that none of the standard tricks actually 51 00:05:12,270 --> 00:05:15,660 help here. 52 00:05:15,660 --> 00:05:26,160 It is that quite complicated to count them count the number of 0,1 string of length n 53 00:05:26,160 --> 00:05:32,370 which does not contain any consecutive zero. 54 00:05:32,370 --> 00:05:37,410 So let us try to see how can we break this one? 55 00:05:37,410 --> 00:05:45,900 So let us T(n) be the number of 0, 1 – string of length n which does not have any consecutive 56 00:05:45,900 --> 00:05:46,900 zeros. 57 00:05:46,900 --> 00:05:49,110 So it is parameterized by the number n. 58 00:05:49,110 --> 00:05:55,190 Question is that can we answer this T(n) for small values of n? 59 00:05:55,190 --> 00:05:57,080 So what is T(1)? 60 00:05:57,080 --> 00:06:05,760 So T(1) of course says that it is the number of 0,1 string of length one that does not 61 00:06:05,760 --> 00:06:08,390 contain any consecutive zero. 62 00:06:08,390 --> 00:06:14,770 Of course there can be two them namely 0 is one of them and 1 is the other, so the answer 63 00:06:14,770 --> 00:06:16,820 is 2. 64 00:06:16,820 --> 00:06:22,350 What about T(2)? 65 00:06:22,350 --> 00:06:28,770 Again I can have 01, 10, 11 but 00 I cannot have right because 00 has consecutive zero. 66 00:06:28,770 --> 00:06:45,000 So the length of number of strings of length 2 which does not contain consecutive zeros 67 00:06:45,000 --> 00:06:47,410 is 3. 68 00:06:47,410 --> 00:06:55,100 So like this we can possibly even count what is T(3), what is T(4) and so on. 69 00:06:55,100 --> 00:06:56,259 But that is a very bad way of doing it. 70 00:06:56,259 --> 00:07:12,850 The question is that if somehow I know how to count T(1) to T(n-1), is there a way of 71 00:07:12,850 --> 00:07:14,160 computing T(n)? 72 00:07:14,160 --> 00:07:21,350 So it is kind of like the induction hypothesis or induction step. 73 00:07:21,350 --> 00:07:31,990 To count T(n), first count T(1) to T(n-1) and then using them can we somehow get an 74 00:07:31,990 --> 00:07:33,870 estimate of T(n). 75 00:07:33,870 --> 00:07:40,960 So let us see. 76 00:07:40,960 --> 00:07:46,980 So this is the problem and T(n) is the number of 0,1 string of length n. 77 00:07:46,980 --> 00:07:54,130 Now consider a 0,1 string of length n which does not contain any non-zero any consecutive 78 00:07:54,130 --> 00:07:55,130 non-zeros. 79 00:07:55,130 --> 00:07:58,520 How does it look like? 80 00:07:58,520 --> 00:08:06,580 It is something of the form x1, x2 till xn right. 81 00:08:06,580 --> 00:08:09,100 It does not contain any consecutive non-zero. 82 00:08:09,100 --> 00:08:13,580 So I will break it up into two cases. 83 00:08:13,580 --> 00:08:21,910 Of course the cases will be depending on whether the last number xn is 1 or 0. 84 00:08:21,910 --> 00:08:23,820 So let us see. 85 00:08:23,820 --> 00:08:36,700 So Case 1 is the last bit is 1 and Case 2 is last bit is 0. 86 00:08:36,700 --> 00:08:44,519 And then what we will do is that we will try to compute the number of elements satisfying 87 00:08:44,519 --> 00:08:49,800 Case 1 and number of elements satisfying Case 2. 88 00:08:49,800 --> 00:08:55,610 In other words, we will count the number of 0,1 string of length n which does not contain 89 00:08:55,610 --> 00:09:03,910 any consecutive zero and last bit which is 1, that is basically the size of Case 1. 90 00:09:03,910 --> 00:09:10,790 And similarly for Case 2, we will compute the number of 0,1 string of length n which 91 00:09:10,790 --> 00:09:16,890 does not have any consecutive zeros and last bit is 1. 92 00:09:16,890 --> 00:09:20,540 Note that these two cases are disjoint. 93 00:09:20,540 --> 00:09:29,350 Meaning a string can be either in Case 1 or in Case 2 and have to be one of them. 94 00:09:29,350 --> 00:09:30,380 Right. 95 00:09:30,380 --> 00:09:38,649 So if I can compute Case 1 and Case 2 then I can get the whole answer as sum of the number 96 00:09:38,649 --> 00:09:44,230 in Case 1 plus sum of the number in Case 2. 97 00:09:44,230 --> 00:09:48,629 And that is the way we will proceed forward. 98 00:09:48,629 --> 00:09:54,129 So let us look at the first case. 99 00:09:54,129 --> 00:10:06,829 Case 1 the last bit is 1. 100 00:10:06,829 --> 00:10:27,600 Now the question is that how many 0,1 string of length n are there which does not have 101 00:10:27,600 --> 00:10:35,190 any consecutive zero and the last bit is 1. 102 00:10:35,190 --> 00:10:42,389 Now let x1 to xn be a 0,1 string of length n which does not have any consecutive zero 103 00:10:42,389 --> 00:10:43,980 and xn equals to 1. 104 00:10:43,980 --> 00:10:45,720 So how does it look like? 105 00:10:45,720 --> 00:10:49,319 So this one is one. 106 00:10:49,319 --> 00:10:51,029 Now what happens to the rest of them? 107 00:10:51,029 --> 00:11:03,130 So let us look at let me draw again here so I have x1, x2 till xn-1 and xn. 108 00:11:03,130 --> 00:11:07,959 Now xn is 1. 109 00:11:07,959 --> 00:11:13,830 So what are these possible? 110 00:11:13,830 --> 00:11:25,120 I claim that you put any number x1 to xn-1 such that it satisifes the condition that 111 00:11:25,120 --> 00:11:31,709 it does not contain any non-zero string I can add a one at the end and I get one of 112 00:11:31,709 --> 00:11:32,959 these cases. 113 00:11:32,959 --> 00:11:42,230 So clearly one thing is that x1 to xn-1 with the string of length n-1 that does not contain 114 00:11:42,230 --> 00:11:51,559 any consecutive zeros, because there are no consecutive zeros between x1 to xn then it 115 00:11:51,559 --> 00:11:57,930 is clearly no consecutive zeros between x1 to xn-1, right. 116 00:11:57,930 --> 00:11:58,930 So it is trick. 117 00:11:58,930 --> 00:12:04,990 So x1 to xn-1 is a string of length n-1 that does not contain consecutive zeros. 118 00:12:04,990 --> 00:12:15,579 And in fact it is also the other way meaning give me any string of length n-1 that does 119 00:12:15,579 --> 00:12:24,170 not contain any consecutive zero I can add a 1 at the end and get a string of length 120 00:12:24,170 --> 00:12:36,399 n that does not contain any consecutive zero and whose last bit is one. 121 00:12:36,399 --> 00:12:38,709 Why is it so? 122 00:12:38,709 --> 00:12:45,560 Now see by adding this 1 the only time I can solve a problem is by creating a consecutive 123 00:12:45,560 --> 00:12:52,910 zero but since this last bit is 1, so whatever happened to the bit before that I do not care, 124 00:12:52,910 --> 00:13:01,290 these two bit together cannot be a consecutive zero ever because this is one. 125 00:13:01,290 --> 00:13:04,209 So this is a, this cannot happen. 126 00:13:04,209 --> 00:13:15,449 So in other words, what do we have, in other words, the x1 to xn-1 is a 0,1 string of length 127 00:13:15,449 --> 00:13:24,380 n-1 which does not have consecutive zero and any such string can be converted into a 0,1 128 00:13:24,380 --> 00:13:32,079 string of length n that does not contain a consecutive zeros and last bit is 1. 129 00:13:32,079 --> 00:13:35,480 So the answer for this one is T(n-1). 130 00:13:35,480 --> 00:13:44,170 Note that this is followed, this is exactly the definition of T(n-1), right. 131 00:13:44,170 --> 00:13:51,379 So the size of Case 1 or size of number of elements that satisfies Case 1 is actually 132 00:13:51,379 --> 00:13:53,480 T(n-1), right. 133 00:13:53,480 --> 00:14:01,999 Now let us move to the second case, Case 2. 134 00:14:01,999 --> 00:14:10,379 In which case, the last bit is 0. 135 00:14:10,379 --> 00:14:22,220 Again, we want to compute now the number of 0,1 string of length n that does not have 136 00:14:22,220 --> 00:14:29,929 any consecutive zero and last bit is 0. 137 00:14:29,929 --> 00:14:34,550 So let us x1 to xn be a 0,1 string of length n which does not have any consecutive zero 138 00:14:34,550 --> 00:14:38,769 and the last bit is zero that means xn is 0. 139 00:14:38,769 --> 00:14:41,819 Now what happens here let us see? 140 00:14:41,819 --> 00:14:46,240 So I have xn-1 and xn. 141 00:14:46,240 --> 00:14:49,309 Now this one is 0, can xn-1 be 0? 142 00:14:49,309 --> 00:14:50,809 No it cannot be. 143 00:14:50,809 --> 00:14:59,829 Because if it was 0, then I would have got two consecutive zeros, xn-1 and xn are two 144 00:14:59,829 --> 00:15:03,559 consecutive numbers that are 0. 145 00:15:03,559 --> 00:15:13,069 So from this definition of x1 to xn, I know that xn-1 is 1. 146 00:15:13,069 --> 00:15:23,029 If xn-1 is one, now let us go to xn-2 what happens here x1 to xn-2. 147 00:15:23,029 --> 00:15:33,629 Now just like we did for the last case, we realize that x1 to xn-2 is now are a string 148 00:15:33,629 --> 00:15:41,689 of length n-2 that does not have any consecutive zeros. 149 00:15:41,689 --> 00:15:54,689 And more importantly any string of length n-2 that does not contain any consecutive 150 00:15:54,689 --> 00:16:02,829 zero, I can add a 1 and the 0 at the back to get a string that satisfies Case 2. 151 00:16:02,829 --> 00:16:20,329 So in other words, we have that xn-1 must be 1 which means that x1 to xn-2 is a string 152 00:16:20,329 --> 00:16:24,019 of length which is n-2 which does not contain any consecutive zero and hence the answer 153 00:16:24,019 --> 00:16:25,899 is T(n-2). 154 00:16:25,899 --> 00:16:33,680 Now, with this can we do something. 155 00:16:33,680 --> 00:16:38,010 Now can we combine them to get some results. 156 00:16:38,010 --> 00:16:40,709 Let us see what we have. 157 00:16:40,709 --> 00:16:52,129 We started with this value of T(n) and we wanted to split up this whole set of strings 158 00:16:52,129 --> 00:16:54,829 of length n into two cases. 159 00:16:54,829 --> 00:17:00,079 Case 1 for which we realize that now that the number of elements in Case 1 is T(n-1) 160 00:17:00,079 --> 00:17:07,419 we had Case 2 the last bit is 0 in which case the number of elements was T(n-2) and therefore 161 00:17:07,419 --> 00:17:09,940 T(n) is equals to T(n-1) + T(n-2). 162 00:17:09,940 --> 00:17:10,940 Right. 163 00:17:10,940 --> 00:17:20,819 Now this does not exactly solve the whole problem, what does it do? 164 00:17:20,819 --> 00:17:32,460 So I told that OK, if this is T(n) I have got T(1) equals to one, I have got T(2) equals 165 00:17:32,460 --> 00:17:40,000 to 2 and I also have got this recurrences that T(n) = T(n-1) +T(n-2). 166 00:17:40,000 --> 00:17:52,179 Note that by doing so I can now just keep on I mean I can find out T(n) by picking this 167 00:17:52,179 --> 00:18:00,600 process multiple times for example what is T(3), T(3) is T(2) plus T(1) which is 5. 168 00:18:00,600 --> 00:18:02,060 What is T(4)? 169 00:18:02,060 --> 00:18:07,420 T(4) is T(3) plus T (2) which is 8. 170 00:18:07,420 --> 00:18:09,110 What is T(5)? 171 00:18:09,110 --> 00:18:15,240 T(5) is T(4) plus T(3) which is 13 and so on. 172 00:18:15,240 --> 00:18:16,240 Right. 173 00:18:16,240 --> 00:18:23,680 So this way I have been able to count what the value of T(n) is-- I found out a nice 174 00:18:23,680 --> 00:18:32,970 way of computing, counting the number of elements of length n number of strings of length n 175 00:18:32,970 --> 00:18:40,669 of 0,1 which does contain any consecutive zeros. 176 00:18:40,669 --> 00:18:44,870 So this is a nice way of counting it. 177 00:18:44,870 --> 00:18:55,820 But the question one can ask is can you count with a better procedure of computing T(n) 178 00:18:55,820 --> 00:19:03,519 or do I have to keep on apply this process again and again and again n times before I 179 00:19:03,519 --> 00:19:04,519 get T(n). 180 00:19:04,519 --> 00:19:14,630 One can be get some answer like T(n) equals to 2 power n by n or something like that. 181 00:19:14,630 --> 00:19:22,190 Like we got for, the case of this distribution of n balls into k bins and so on. 182 00:19:22,190 --> 00:19:30,170 Cannot we get a closed form expression of T(n). 183 00:19:30,170 --> 00:19:40,200 So this brings us to this problem of recurrence relation. 184 00:19:40,200 --> 00:19:46,679 So recurrence relation is a subject, so according to Wikipedia this is a quotation from Wikipedia, 185 00:19:46,679 --> 00:19:51,970 “In Mathematics, a recurrence relation is an equation that recursively defines a sequence 186 00:19:51,970 --> 00:20:04,720 or multidimensional array of values, once one or more initial terms are given: each 187 00:20:04,720 --> 00:20:11,810 further term of the sequence is defined as a function of the preceding terms.” 188 00:20:11,810 --> 00:20:14,360 Let us go back and let see that this is actually a recurrence relation. 189 00:20:14,360 --> 00:20:24,820 I have been given the initial terms which are these and this is how I solved T(n) by 190 00:20:24,820 --> 00:20:36,059 looking at the preceding numbers, right. 191 00:20:36,059 --> 00:20:41,600 In fact, recurrences relation is used extensively for combinatorics, analysis of algorithms, 192 00:20:41,600 --> 00:20:49,440 computational biology, in theoretical economics and many, many other subjects. 193 00:20:49,440 --> 00:20:57,179 Recurrence Relations are used for modeling problems particularly counting problems like 194 00:20:57,179 --> 00:20:58,820 we sort today. 195 00:20:58,820 --> 00:21:08,120 And it is also a big use of the recurrence relations is that one can solve recurrence 196 00:21:08,120 --> 00:21:09,390 relations easily. 197 00:21:09,390 --> 00:21:14,240 Namely one can get some compact form for this T(n) and so on. 198 00:21:14,240 --> 00:21:21,490 In the next video lecture, we will see one more example of application of recurrence 199 00:21:21,490 --> 00:21:28,179 relations for counting and after that we will be going into how to solve recurrence relations. 200 00:21:28,179 --> 00:21:28,879 Thank you.