1
00:00:03,399 --> 00:00:20,310
Hello, welcome to another module in this massive
open online course on probability and random
2
00:00:20,310 --> 00:00:22,410
variables for wireless communications.
3
00:00:22,410 --> 00:00:27,260
In the previous model we have looked at various
concepts of probability.
4
00:00:27,260 --> 00:00:32,310
In this module, let us start looking at another
new concept or a new result which is very
5
00:00:32,310 --> 00:00:35,010
important in the context of communication.
6
00:00:35,010 --> 00:00:37,430
This is the Bayes Theorem.
7
00:00:37,430 --> 00:00:56,080
So let us look, in this module, let us start
looking at
8
00:00:56,080 --> 00:00:57,200
Bayes Theorem.
9
00:00:57,200 --> 00:01:12,250
For Bayes Theorem, what we would like to do
is we would like to consider the sample space,
10
00:01:12,250 --> 00:01:16,560
S. I would like to consider 2 events.
11
00:01:16,560 --> 00:01:32,390
Consider 2 events, A0 and A1, these are basically
2 events.
12
00:01:32,390 --> 00:01:46,620
And these, both A0 and A1 belong to the sample
space S such that A0 union A1 equals the entire
13
00:01:46,620 --> 00:01:54,200
sample space S. And A0, A1 are mutually exclusive.
14
00:01:54,200 --> 00:02:02,470
That is, A0 intersection A1 equals phi.
15
00:02:02,470 --> 00:02:11,370
So we are considering 2 events, A0 and A1
in our sample space S such that A0 union A1
16
00:02:11,370 --> 00:02:17,799
that is the union of these 2 events is equal
to the entire sample space, S and these 2
17
00:02:17,799 --> 00:02:24,700
events A0 and A1 are mutually exclusive that
is A0 intersection A1 is the null set or null
18
00:02:24,700 --> 00:02:26,459
event, phi.
19
00:02:26,459 --> 00:02:31,629
Such events, A0 and A1 are known as mutually
exclusive and exhaustive.
20
00:02:31,629 --> 00:03:01,060
So these events, A0, A1 are mutually
exclusive and exhaustive.
21
00:03:01,060 --> 00:03:07,709
Exhaustive meaning, their union spans the
entire sample space, S and while their intersection
22
00:03:07,709 --> 00:03:09,170
is the null event.
23
00:03:09,170 --> 00:03:14,129
Therefore these events are mutually exclusive
and exhaustive.
24
00:03:14,129 --> 00:03:22,889
Now let us consider another event B. Let us
consider an event B in the sample space, S.
25
00:03:22,889 --> 00:03:35,709
Now you can see, this part is B intersection
A0 and this part is B intersection A1.
26
00:03:35,709 --> 00:03:43,239
A0 now you can see clearly, B intersection
A0 and B intersection A1 are also mutually
27
00:03:43,239 --> 00:03:44,529
exclusive.
28
00:03:44,529 --> 00:03:52,529
Further, B intersection A0 and B intersection
A1 together span B or together, the union
29
00:03:52,529 --> 00:03:54,900
of these 2 is B.
30
00:03:54,900 --> 00:04:30,620
Alright ! So we can see, for any set B, B
intersection A0, B intersection A1 are mutually
31
00:04:30,620 --> 00:04:38,510
exclusive events that is their intersection
is phi or the null event.
32
00:04:38,510 --> 00:04:51,750
Further, B intersection A0 union B intersection
A1 equals the event B. Alright ! So we are
33
00:04:51,750 --> 00:05:01,930
saying that for any event B, we have B intersection
A0 and B intersection A1 are mutually exclusive.
34
00:05:01,930 --> 00:05:04,400
That is, their intersection is a null event.
35
00:05:04,400 --> 00:05:10,349
And B intersection A0 union B intersection
A1 is equal to the entire event B.
36
00:05:10,349 --> 00:05:28,360
Therefore, if I now look at the probability
of B, so now, B equals B intersection A0 union
37
00:05:28,360 --> 00:05:32,130
B intersection A1.
38
00:05:32,130 --> 00:05:42,940
Therefore the probability of B equals probability
of B intersection A0 plus the probability
39
00:05:42,940 --> 00:05:50,370
of B intersection A1 because B intersection
A0 and B intersection A1 are mutually exclusive.
40
00:05:50,370 --> 00:05:57,820
Now again we know that from the basic definition
of conditional probability, we had already
41
00:05:57,820 --> 00:06:08,689
shown that probability of B intersection A0
is the probability of B given A0 times the
42
00:06:08,689 --> 00:06:10,949
probability of A0.
43
00:06:10,949 --> 00:06:21,430
Probability of B intersection A1 equals the
probability of B given A1 times the probability
44
00:06:21,430 --> 00:06:23,050
of A1.
45
00:06:23,050 --> 00:06:27,310
We know from the definition, from our conditional
probability module that the probability of
46
00:06:27,310 --> 00:06:31,750
B intersection A0 is the probability of B
given A0 times the probability of A0.
47
00:06:31,750 --> 00:06:37,639
The probability of B intersection A1 is the
probability of B given A1 times the probability
48
00:06:37,639 --> 00:06:38,640
of A1.
49
00:06:38,640 --> 00:06:46,780
Therefore, substituting these in the expression
above, we have the total probability P(B)
50
00:06:46,780 --> 00:07:08,499
is the probability B given A0, probability
A0 plus probability B given A1 times the probability
51
00:07:08,499 --> 00:07:09,659
of A1.
52
00:07:09,659 --> 00:07:16,129
So the probability of B is the probability
B given A0 times probability A0 plus probability
53
00:07:16,129 --> 00:07:20,219
B given A1 times the probability of A1.
54
00:07:20,219 --> 00:07:28,021
Now, we also know again that the probability
of B intersection A0 again from our country
55
00:07:28,021 --> 00:07:37,259
the conditional probability is the probability
of B given A0 times the probability of A0
56
00:07:37,259 --> 00:07:49,750
and this is also equal to the probability
of A0 given B times the probability of B.
57
00:07:49,750 --> 00:07:54,961
So the probability of B intersection A0 is
the probability of B given A0 is the probability
58
00:07:54,961 --> 00:07:58,939
of B given A0 times the probability of A0
which is also the probability of A0 given
59
00:07:58,939 --> 00:08:05,629
B times the probability of B. Therefore from
this, what I have is interestingly I have
60
00:08:05,629 --> 00:08:06,629
from this.
61
00:08:06,629 --> 00:08:22,611
This implies that the probability of A0 given
B equals probability of B given A0 into probability
62
00:08:22,611 --> 00:08:27,289
of A0 divided by the probability of B.
63
00:08:27,289 --> 00:08:33,401
Now what I am going to do here is I am going
to substitute the expression of probability
64
00:08:33,401 --> 00:08:39,460
of B from the previous page and therefore
what I have is this is equal to
65
00:08:39,460 --> 00:08:53,850
Éprobability B given A0 times probability
of A0 divided by probability of B given A0
66
00:08:53,850 --> 00:09:04,450
times probability of A0 + probability of B
given A1 times the probability of A1.
67
00:09:04,450 --> 00:09:11,350
This is the expression for probability of
A0 given B. So we have probability of A0 given
68
00:09:11,350 --> 00:09:18,690
B equals probability of B given A0 times probability
of A0 divided by the probability of B given
69
00:09:18,690 --> 00:09:25,060
A0 times probability of A0 + probability of
B given A1 times the probability of A1.
70
00:09:25,060 --> 00:09:26,220
Right?
71
00:09:26,220 --> 00:09:32,370
Similarly, I can derive the expression for
probability of A1 given B.
72
00:09:32,370 --> 00:09:37,530
This is the expression for probability of
A0 given B. The expression for the probability
73
00:09:37,530 --> 00:09:41,030
of A1 given B is similar.
74
00:09:41,030 --> 00:09:51,890
Probability of B given A1 times the probability
of A1 by probability of B given A0 times probability
75
00:09:51,890 --> 00:10:03,440
of A0 plus probability of B given A1 times
probability of A1 and you can also see that
76
00:10:03,440 --> 00:10:28,720
the probability of A0 given B plus the probability
of A1 given B equals one because A0 and A1
77
00:10:28,720 --> 00:10:33,340
are mutually exclusive and exhaustive events.
78
00:10:33,340 --> 00:10:38,500
Therefore the probability of A0 given B plus
the probability of A1 given B is equal to
79
00:10:38,500 --> 00:10:39,500
1.
80
00:10:39,500 --> 00:10:45,230
And these quantities here, these quantities
that we have calculated here, so this is basically
81
00:10:45,230 --> 00:10:47,080
the Bayes theorem.
82
00:10:47,080 --> 00:10:52,900
Right ? Now these quantities here, the probabilities
of A0 given B and the probabilities of A1
83
00:10:52,900 --> 00:10:58,600
given B: these quantities are very important
in the context of communication.
84
00:10:58,600 --> 00:11:14,360
These are known as Aposteriori probabilities.
85
00:11:14,360 --> 00:11:18,140
These quantities are very important in the
context of communication.
86
00:11:18,140 --> 00:11:27,310
The probability is P of A0 given B and P of
A1 given B, these are known as Aposteriori
87
00:11:27,310 --> 00:11:28,870
probability.
88
00:11:28,870 --> 00:11:34,250
And we are going to introduce an example later
which will clarify the application of these
89
00:11:34,250 --> 00:11:40,170
but these quantities are known as the Aposteriori
probabilities and these Aposteriori probabilities
90
00:11:40,170 --> 00:11:43,360
can now be computed using Bayes Theorem
91
00:11:43,360 --> 00:11:49,161
and these expressions that you see for the
Aposteriori probabilities that you see, this
92
00:11:49,161 --> 00:11:55,170
is nothing but this is our Bayes Theorem.
93
00:11:55,170 --> 00:12:03,260
Our Bayes theorem gives an expression for
the Aposteriori probabilities.
94
00:12:03,260 --> 00:12:26,690
These quantities, P of A0, P of A1: these
are known as the prior
95
00:12:26,690 --> 00:12:37,540
probabilities and these quantities, probability
of B given A0 and the probability of B given
96
00:12:37,540 --> 00:12:46,080
A1, these are called the likelihoods.
97
00:12:46,080 --> 00:12:53,330
All of these are important terminologies in
the context of communication or wireless communication.
98
00:12:53,330 --> 00:13:03,880
So if A0, A1 are mutually exclusive and mutually
exhaustive, we have the Bayes gives us a very
99
00:13:03,880 --> 00:13:11,800
useful relation to calculate the Aposteriori
probabilities, P of A0 given B and P of A1
100
00:13:11,800 --> 00:13:19,890
given B in terms of the prior probabilities
P of A0 A1 and the likelihoods, P of B given
101
00:13:19,890 --> 00:13:23,370
A0 and P of B given A1.
102
00:13:23,370 --> 00:13:26,970
And this is a very important result and we
are going to demonstrate an application of
103
00:13:26,970 --> 00:13:32,700
this shortly in the context of the MAP principle
or the maximum Aposteriori probability receiver
104
00:13:32,700 --> 00:13:39,890
but before we do that, let us extend this
Bayes Theorem to a general case with N mutually
105
00:13:39,890 --> 00:13:43,240
exclusive and exhaustive events, Ai.
106
00:13:43,240 --> 00:13:51,170
So now, let us extend this to a general version
of the Bayes Theorem.
107
00:13:51,170 --> 00:14:16,760
So let us now state a generalised version
of or a general version of Bayes Theorem where
108
00:14:16,760 --> 00:14:30,850
we now consider a sample space, S.
Let us now consider the sample space, S which
109
00:14:30,850 --> 00:14:51,600
is divided into N
mutually exclusive and exhaustive.
110
00:14:51,600 --> 00:15:19,440
So what I have over here is, I have my sample
space, S and A0, A1 up to AN-1.
111
00:15:19,440 --> 00:15:36,570
These are N events which belong to S. Further,
A0, A1 up to AN-1. are mutually, these are
112
00:15:36,570 --> 00:15:41,190
mutually exclusive.
113
00:15:41,190 --> 00:15:52,500
These are mutually
exclusive implies Ai intersection Aj equal
114
00:15:52,500 --> 00:15:55,860
to phi for any i not equal to j.
115
00:15:55,860 --> 00:16:05,840
That is, we have the property that Ai that
is I have capital N events, A0, A1 up to AN-1,
116
00:16:05,840 --> 00:16:13,160
these are mutually exclusive implying that
if I take any 2 events, Ai and Aj, Ai intersection
117
00:16:13,160 --> 00:16:16,450
Aj is the null event that is phi.
118
00:16:16,450 --> 00:16:27,310
Further, if I take the union of all these
events, A0 union A1 union so on until union
119
00:16:27,310 --> 00:16:36,740
AN-1 should be equal to the sample space,
S. So this is the exhaustive property.
120
00:16:36,740 --> 00:16:46,730
So remember, this is the exhaustive property.
121
00:16:46,730 --> 00:16:51,300
Exhaustive implying that the union of all
these events is this entire sample space and
122
00:16:51,300 --> 00:16:57,200
mutually exclusive implying that if you take
any 2 events, Ai and Aj, their intersection
123
00:16:57,200 --> 00:16:58,400
is the null event.
124
00:16:58,400 --> 00:17:04,370
Basically, in terms of set theory, we say
the sets of events, A0, A1 up to AN-1 are
125
00:17:04,370 --> 00:17:09,990
a partition of the entire sample space, S.
That is, their unionÕs spans the entire sample
126
00:17:09,990 --> 00:17:14,250
space, S and these different events or sides
are disjoint.
127
00:17:14,250 --> 00:17:18,530
That is, if I take any 2 sets, their intersection
is the empty set.
128
00:17:18,530 --> 00:17:22,839
These are mutually exclusive and exhaustive
events.
129
00:17:22,839 --> 00:17:30,530
And now the Bayes Theorem for the Aposteriori
probabilities, Bayes theorem gives the Aposteriori
130
00:17:30,530 --> 00:17:47,900
probability similar to what we have seen before,
P of Ai given B is P ofÉ remember how we
131
00:17:47,900 --> 00:18:03,030
derived the Bayes result, we first said that
P of B intersection Ai equals P of B given
132
00:18:03,030 --> 00:18:18,390
Ai into P of Ai which is equal to P of Ai
given B into P of B which basically implies
133
00:18:18,390 --> 00:18:39,980
that P of Ai given B equals divided by P of
B, P of B given Ai times P of Ai.
134
00:18:39,980 --> 00:18:56,190
Now, if I substitute the expression for P
of B, I have P of Ai given B is equal to P
135
00:18:56,190 --> 00:19:19,010
of B given Ai times P of Ai divided by summation
J equal to 0 to N Ð 1, P of B given Aj times
136
00:19:19,010 --> 00:19:20,320
P of Aj.
137
00:19:20,320 --> 00:19:37,250
And this is the, this is my result for
138
00:19:37,250 --> 00:19:39,490
the Bayes Theorem.
139
00:19:39,490 --> 00:19:41,240
That is what is my Bayes Theorem?
140
00:19:41,240 --> 00:19:48,190
Bayes theorem states that P of Ai that is
the Aposteriori probability of the event Ai
141
00:19:48,190 --> 00:19:56,160
given B, probability Ai given B is probability
B given Ai times probability Ai divided by
142
00:19:56,160 --> 00:20:02,960
the probability B which is basically summation
j equal to 0 to N - 1 probability of B given
143
00:20:02,960 --> 00:20:06,460
Aj times probability of Aj.
144
00:20:06,460 --> 00:20:15,220
And also, therefore, as we have also seen
before, summation of, it is easy to see that
145
00:20:15,220 --> 00:20:23,800
the
summation of j equal to 0 to N - 1 or i equal
146
00:20:23,800 --> 00:20:36,470
to 0 to N Ð 1, probability of Ai given B,
this is equal to 1.
147
00:20:36,470 --> 00:20:46,300
Further, these quantityÕs, probabilities
of Ai given B, these are the Aposteriori probabilities.
148
00:20:46,300 --> 00:20:58,600
Remember these are the Aposteriori probabilities.
149
00:20:58,600 --> 00:21:14,500
These probabilities of AI, these quantities,
the probabilities of AI, these are the prior
150
00:21:14,500 --> 00:21:15,810
probabilities.
151
00:21:15,810 --> 00:21:26,140
And the probabilities of B given Ai, these
are the likelihoods.
152
00:21:26,140 --> 00:21:28,670
Right?
153
00:21:28,670 --> 00:21:36,330
So this is a general expression for the Bayes
Theorem, in terms that is expression for the
154
00:21:36,330 --> 00:21:42,010
Aposteriori probabilities in terms of the
prior probabilities and the likelihoods and
155
00:21:42,010 --> 00:21:48,120
we said that this Bayes Theorem or this Bayes
result is a very important principle in communications.
156
00:21:48,120 --> 00:21:54,830
This is used to construct what we call that
as the maximum Aposteriori, the MAP receiver
157
00:21:54,830 --> 00:22:01,000
which is something that we are going to look
at in our next module.
158
00:22:01,000 --> 00:22:16,030
So I would like to conclude this module here.
159
00:22:16,030 --> 00:22:29,850
Thank you very much.