1 00:00:03,399 --> 00:00:20,310 Hello, welcome to another module in this massive open online course on probability and random 2 00:00:20,310 --> 00:00:22,410 variables for wireless communications. 3 00:00:22,410 --> 00:00:27,260 In the previous model we have looked at various concepts of probability. 4 00:00:27,260 --> 00:00:32,310 In this module, let us start looking at another new concept or a new result which is very 5 00:00:32,310 --> 00:00:35,010 important in the context of communication. 6 00:00:35,010 --> 00:00:37,430 This is the Bayes Theorem. 7 00:00:37,430 --> 00:00:56,080 So let us look, in this module, let us start looking at 8 00:00:56,080 --> 00:00:57,200 Bayes Theorem. 9 00:00:57,200 --> 00:01:12,250 For Bayes Theorem, what we would like to do is we would like to consider the sample space, 10 00:01:12,250 --> 00:01:16,560 S. I would like to consider 2 events. 11 00:01:16,560 --> 00:01:32,390 Consider 2 events, A0 and A1, these are basically 2 events. 12 00:01:32,390 --> 00:01:46,620 And these, both A0 and A1 belong to the sample space S such that A0 union A1 equals the entire 13 00:01:46,620 --> 00:01:54,200 sample space S. And A0, A1 are mutually exclusive. 14 00:01:54,200 --> 00:02:02,470 That is, A0 intersection A1 equals phi. 15 00:02:02,470 --> 00:02:11,370 So we are considering 2 events, A0 and A1 in our sample space S such that A0 union A1 16 00:02:11,370 --> 00:02:17,799 that is the union of these 2 events is equal to the entire sample space, S and these 2 17 00:02:17,799 --> 00:02:24,700 events A0 and A1 are mutually exclusive that is A0 intersection A1 is the null set or null 18 00:02:24,700 --> 00:02:26,459 event, phi. 19 00:02:26,459 --> 00:02:31,629 Such events, A0 and A1 are known as mutually exclusive and exhaustive. 20 00:02:31,629 --> 00:03:01,060 So these events, A0, A1 are mutually exclusive and exhaustive. 21 00:03:01,060 --> 00:03:07,709 Exhaustive meaning, their union spans the entire sample space, S and while their intersection 22 00:03:07,709 --> 00:03:09,170 is the null event. 23 00:03:09,170 --> 00:03:14,129 Therefore these events are mutually exclusive and exhaustive. 24 00:03:14,129 --> 00:03:22,889 Now let us consider another event B. Let us consider an event B in the sample space, S. 25 00:03:22,889 --> 00:03:35,709 Now you can see, this part is B intersection A0 and this part is B intersection A1. 26 00:03:35,709 --> 00:03:43,239 A0 now you can see clearly, B intersection A0 and B intersection A1 are also mutually 27 00:03:43,239 --> 00:03:44,529 exclusive. 28 00:03:44,529 --> 00:03:52,529 Further, B intersection A0 and B intersection A1 together span B or together, the union 29 00:03:52,529 --> 00:03:54,900 of these 2 is B. 30 00:03:54,900 --> 00:04:30,620 Alright ! So we can see, for any set B, B intersection A0, B intersection A1 are mutually 31 00:04:30,620 --> 00:04:38,510 exclusive events that is their intersection is phi or the null event. 32 00:04:38,510 --> 00:04:51,750 Further, B intersection A0 union B intersection A1 equals the event B. Alright ! So we are 33 00:04:51,750 --> 00:05:01,930 saying that for any event B, we have B intersection A0 and B intersection A1 are mutually exclusive. 34 00:05:01,930 --> 00:05:04,400 That is, their intersection is a null event. 35 00:05:04,400 --> 00:05:10,349 And B intersection A0 union B intersection A1 is equal to the entire event B. 36 00:05:10,349 --> 00:05:28,360 Therefore, if I now look at the probability of B, so now, B equals B intersection A0 union 37 00:05:28,360 --> 00:05:32,130 B intersection A1. 38 00:05:32,130 --> 00:05:42,940 Therefore the probability of B equals probability of B intersection A0 plus the probability 39 00:05:42,940 --> 00:05:50,370 of B intersection A1 because B intersection A0 and B intersection A1 are mutually exclusive. 40 00:05:50,370 --> 00:05:57,820 Now again we know that from the basic definition of conditional probability, we had already 41 00:05:57,820 --> 00:06:08,689 shown that probability of B intersection A0 is the probability of B given A0 times the 42 00:06:08,689 --> 00:06:10,949 probability of A0. 43 00:06:10,949 --> 00:06:21,430 Probability of B intersection A1 equals the probability of B given A1 times the probability 44 00:06:21,430 --> 00:06:23,050 of A1. 45 00:06:23,050 --> 00:06:27,310 We know from the definition, from our conditional probability module that the probability of 46 00:06:27,310 --> 00:06:31,750 B intersection A0 is the probability of B given A0 times the probability of A0. 47 00:06:31,750 --> 00:06:37,639 The probability of B intersection A1 is the probability of B given A1 times the probability 48 00:06:37,639 --> 00:06:38,640 of A1. 49 00:06:38,640 --> 00:06:46,780 Therefore, substituting these in the expression above, we have the total probability P(B) 50 00:06:46,780 --> 00:07:08,499 is the probability B given A0, probability A0 plus probability B given A1 times the probability 51 00:07:08,499 --> 00:07:09,659 of A1. 52 00:07:09,659 --> 00:07:16,129 So the probability of B is the probability B given A0 times probability A0 plus probability 53 00:07:16,129 --> 00:07:20,219 B given A1 times the probability of A1. 54 00:07:20,219 --> 00:07:28,021 Now, we also know again that the probability of B intersection A0 again from our country 55 00:07:28,021 --> 00:07:37,259 the conditional probability is the probability of B given A0 times the probability of A0 56 00:07:37,259 --> 00:07:49,750 and this is also equal to the probability of A0 given B times the probability of B. 57 00:07:49,750 --> 00:07:54,961 So the probability of B intersection A0 is the probability of B given A0 is the probability 58 00:07:54,961 --> 00:07:58,939 of B given A0 times the probability of A0 which is also the probability of A0 given 59 00:07:58,939 --> 00:08:05,629 B times the probability of B. Therefore from this, what I have is interestingly I have 60 00:08:05,629 --> 00:08:06,629 from this. 61 00:08:06,629 --> 00:08:22,611 This implies that the probability of A0 given B equals probability of B given A0 into probability 62 00:08:22,611 --> 00:08:27,289 of A0 divided by the probability of B. 63 00:08:27,289 --> 00:08:33,401 Now what I am going to do here is I am going to substitute the expression of probability 64 00:08:33,401 --> 00:08:39,460 of B from the previous page and therefore what I have is this is equal to 65 00:08:39,460 --> 00:08:53,850 Éprobability B given A0 times probability of A0 divided by probability of B given A0 66 00:08:53,850 --> 00:09:04,450 times probability of A0 + probability of B given A1 times the probability of A1. 67 00:09:04,450 --> 00:09:11,350 This is the expression for probability of A0 given B. So we have probability of A0 given 68 00:09:11,350 --> 00:09:18,690 B equals probability of B given A0 times probability of A0 divided by the probability of B given 69 00:09:18,690 --> 00:09:25,060 A0 times probability of A0 + probability of B given A1 times the probability of A1. 70 00:09:25,060 --> 00:09:26,220 Right? 71 00:09:26,220 --> 00:09:32,370 Similarly, I can derive the expression for probability of A1 given B. 72 00:09:32,370 --> 00:09:37,530 This is the expression for probability of A0 given B. The expression for the probability 73 00:09:37,530 --> 00:09:41,030 of A1 given B is similar. 74 00:09:41,030 --> 00:09:51,890 Probability of B given A1 times the probability of A1 by probability of B given A0 times probability 75 00:09:51,890 --> 00:10:03,440 of A0 plus probability of B given A1 times probability of A1 and you can also see that 76 00:10:03,440 --> 00:10:28,720 the probability of A0 given B plus the probability of A1 given B equals one because A0 and A1 77 00:10:28,720 --> 00:10:33,340 are mutually exclusive and exhaustive events. 78 00:10:33,340 --> 00:10:38,500 Therefore the probability of A0 given B plus the probability of A1 given B is equal to 79 00:10:38,500 --> 00:10:39,500 1. 80 00:10:39,500 --> 00:10:45,230 And these quantities here, these quantities that we have calculated here, so this is basically 81 00:10:45,230 --> 00:10:47,080 the Bayes theorem. 82 00:10:47,080 --> 00:10:52,900 Right ? Now these quantities here, the probabilities of A0 given B and the probabilities of A1 83 00:10:52,900 --> 00:10:58,600 given B: these quantities are very important in the context of communication. 84 00:10:58,600 --> 00:11:14,360 These are known as Aposteriori probabilities. 85 00:11:14,360 --> 00:11:18,140 These quantities are very important in the context of communication. 86 00:11:18,140 --> 00:11:27,310 The probability is P of A0 given B and P of A1 given B, these are known as Aposteriori 87 00:11:27,310 --> 00:11:28,870 probability. 88 00:11:28,870 --> 00:11:34,250 And we are going to introduce an example later which will clarify the application of these 89 00:11:34,250 --> 00:11:40,170 but these quantities are known as the Aposteriori probabilities and these Aposteriori probabilities 90 00:11:40,170 --> 00:11:43,360 can now be computed using Bayes Theorem 91 00:11:43,360 --> 00:11:49,161 and these expressions that you see for the Aposteriori probabilities that you see, this 92 00:11:49,161 --> 00:11:55,170 is nothing but this is our Bayes Theorem. 93 00:11:55,170 --> 00:12:03,260 Our Bayes theorem gives an expression for the Aposteriori probabilities. 94 00:12:03,260 --> 00:12:26,690 These quantities, P of A0, P of A1: these are known as the prior 95 00:12:26,690 --> 00:12:37,540 probabilities and these quantities, probability of B given A0 and the probability of B given 96 00:12:37,540 --> 00:12:46,080 A1, these are called the likelihoods. 97 00:12:46,080 --> 00:12:53,330 All of these are important terminologies in the context of communication or wireless communication. 98 00:12:53,330 --> 00:13:03,880 So if A0, A1 are mutually exclusive and mutually exhaustive, we have the Bayes gives us a very 99 00:13:03,880 --> 00:13:11,800 useful relation to calculate the Aposteriori probabilities, P of A0 given B and P of A1 100 00:13:11,800 --> 00:13:19,890 given B in terms of the prior probabilities P of A0 A1 and the likelihoods, P of B given 101 00:13:19,890 --> 00:13:23,370 A0 and P of B given A1. 102 00:13:23,370 --> 00:13:26,970 And this is a very important result and we are going to demonstrate an application of 103 00:13:26,970 --> 00:13:32,700 this shortly in the context of the MAP principle or the maximum Aposteriori probability receiver 104 00:13:32,700 --> 00:13:39,890 but before we do that, let us extend this Bayes Theorem to a general case with N mutually 105 00:13:39,890 --> 00:13:43,240 exclusive and exhaustive events, Ai. 106 00:13:43,240 --> 00:13:51,170 So now, let us extend this to a general version of the Bayes Theorem. 107 00:13:51,170 --> 00:14:16,760 So let us now state a generalised version of or a general version of Bayes Theorem where 108 00:14:16,760 --> 00:14:30,850 we now consider a sample space, S. Let us now consider the sample space, S which 109 00:14:30,850 --> 00:14:51,600 is divided into N mutually exclusive and exhaustive. 110 00:14:51,600 --> 00:15:19,440 So what I have over here is, I have my sample space, S and A0, A1 up to AN-1. 111 00:15:19,440 --> 00:15:36,570 These are N events which belong to S. Further, A0, A1 up to AN-1. are mutually, these are 112 00:15:36,570 --> 00:15:41,190 mutually exclusive. 113 00:15:41,190 --> 00:15:52,500 These are mutually exclusive implies Ai intersection Aj equal 114 00:15:52,500 --> 00:15:55,860 to phi for any i not equal to j. 115 00:15:55,860 --> 00:16:05,840 That is, we have the property that Ai that is I have capital N events, A0, A1 up to AN-1, 116 00:16:05,840 --> 00:16:13,160 these are mutually exclusive implying that if I take any 2 events, Ai and Aj, Ai intersection 117 00:16:13,160 --> 00:16:16,450 Aj is the null event that is phi. 118 00:16:16,450 --> 00:16:27,310 Further, if I take the union of all these events, A0 union A1 union so on until union 119 00:16:27,310 --> 00:16:36,740 AN-1 should be equal to the sample space, S. So this is the exhaustive property. 120 00:16:36,740 --> 00:16:46,730 So remember, this is the exhaustive property. 121 00:16:46,730 --> 00:16:51,300 Exhaustive implying that the union of all these events is this entire sample space and 122 00:16:51,300 --> 00:16:57,200 mutually exclusive implying that if you take any 2 events, Ai and Aj, their intersection 123 00:16:57,200 --> 00:16:58,400 is the null event. 124 00:16:58,400 --> 00:17:04,370 Basically, in terms of set theory, we say the sets of events, A0, A1 up to AN-1 are 125 00:17:04,370 --> 00:17:09,990 a partition of the entire sample space, S. That is, their unionÕs spans the entire sample 126 00:17:09,990 --> 00:17:14,250 space, S and these different events or sides are disjoint. 127 00:17:14,250 --> 00:17:18,530 That is, if I take any 2 sets, their intersection is the empty set. 128 00:17:18,530 --> 00:17:22,839 These are mutually exclusive and exhaustive events. 129 00:17:22,839 --> 00:17:30,530 And now the Bayes Theorem for the Aposteriori probabilities, Bayes theorem gives the Aposteriori 130 00:17:30,530 --> 00:17:47,900 probability similar to what we have seen before, P of Ai given B is P ofÉ remember how we 131 00:17:47,900 --> 00:18:03,030 derived the Bayes result, we first said that P of B intersection Ai equals P of B given 132 00:18:03,030 --> 00:18:18,390 Ai into P of Ai which is equal to P of Ai given B into P of B which basically implies 133 00:18:18,390 --> 00:18:39,980 that P of Ai given B equals divided by P of B, P of B given Ai times P of Ai. 134 00:18:39,980 --> 00:18:56,190 Now, if I substitute the expression for P of B, I have P of Ai given B is equal to P 135 00:18:56,190 --> 00:19:19,010 of B given Ai times P of Ai divided by summation J equal to 0 to N Ð 1, P of B given Aj times 136 00:19:19,010 --> 00:19:20,320 P of Aj. 137 00:19:20,320 --> 00:19:37,250 And this is the, this is my result for 138 00:19:37,250 --> 00:19:39,490 the Bayes Theorem. 139 00:19:39,490 --> 00:19:41,240 That is what is my Bayes Theorem? 140 00:19:41,240 --> 00:19:48,190 Bayes theorem states that P of Ai that is the Aposteriori probability of the event Ai 141 00:19:48,190 --> 00:19:56,160 given B, probability Ai given B is probability B given Ai times probability Ai divided by 142 00:19:56,160 --> 00:20:02,960 the probability B which is basically summation j equal to 0 to N - 1 probability of B given 143 00:20:02,960 --> 00:20:06,460 Aj times probability of Aj. 144 00:20:06,460 --> 00:20:15,220 And also, therefore, as we have also seen before, summation of, it is easy to see that 145 00:20:15,220 --> 00:20:23,800 the summation of j equal to 0 to N - 1 or i equal 146 00:20:23,800 --> 00:20:36,470 to 0 to N Ð 1, probability of Ai given B, this is equal to 1. 147 00:20:36,470 --> 00:20:46,300 Further, these quantityÕs, probabilities of Ai given B, these are the Aposteriori probabilities. 148 00:20:46,300 --> 00:20:58,600 Remember these are the Aposteriori probabilities. 149 00:20:58,600 --> 00:21:14,500 These probabilities of AI, these quantities, the probabilities of AI, these are the prior 150 00:21:14,500 --> 00:21:15,810 probabilities. 151 00:21:15,810 --> 00:21:26,140 And the probabilities of B given Ai, these are the likelihoods. 152 00:21:26,140 --> 00:21:28,670 Right? 153 00:21:28,670 --> 00:21:36,330 So this is a general expression for the Bayes Theorem, in terms that is expression for the 154 00:21:36,330 --> 00:21:42,010 Aposteriori probabilities in terms of the prior probabilities and the likelihoods and 155 00:21:42,010 --> 00:21:48,120 we said that this Bayes Theorem or this Bayes result is a very important principle in communications. 156 00:21:48,120 --> 00:21:54,830 This is used to construct what we call that as the maximum Aposteriori, the MAP receiver 157 00:21:54,830 --> 00:22:01,000 which is something that we are going to look at in our next module. 158 00:22:01,000 --> 00:22:16,030 So I would like to conclude this module here. 159 00:22:16,030 --> 00:22:29,850 Thank you very much.