1
00:00:13,590 --> 00:00:21,270
Hello everyone, welcome to another module
in this massive open online course on principles
2
00:00:21,270 --> 00:00:25,360
of probability and random variables for wireless
communication.
3
00:00:25,360 --> 00:00:30,000
Alright, so in the previous model, we have
seen the concepts of condition the probability.
4
00:00:30,000 --> 00:00:36,710
Let us now look at another aspect, another
very important property that is known as independence.
5
00:00:36,710 --> 00:00:37,890
All right?
6
00:00:37,890 --> 00:00:59,610
So today, let us look at the concept of independent
events.
7
00:00:59,610 --> 00:01:06,200
And this is a very important aspect of probability.
8
00:01:06,200 --> 00:01:08,420
That is, let us define this.
9
00:01:08,420 --> 00:01:15,900
This is a very key concept in probability.
10
00:01:15,900 --> 00:01:41,480
We say, 2 events are statistically independent
if the probability of A given B.
11
00:01:41,480 --> 00:01:48,899
That is 2 events, A and B are statistically
independent if the probability of A given
12
00:01:48,899 --> 00:01:58,259
B equals the probability of A. That is, what
are we saying?
13
00:01:58,259 --> 00:02:09,640
That is, the probability of A given B that
is we say, 2 events A, B are statistically
14
00:02:09,640 --> 00:02:17,080
independent if the probability of A given
B or the probability of A conditioned on B
15
00:02:17,080 --> 00:02:26,489
is simply the probability of A. That is, given
that event B has occurred does not in any
16
00:02:26,489 --> 00:02:31,829
way affect the probability of occurrence of
A. We say that A and B are independent.
17
00:02:31,829 --> 00:02:38,969
If given that B has occurred does not in any
way affect the occurrence of the probability
18
00:02:38,969 --> 00:02:47,189
of A. Therefore, the probability of A conditioned
on B is simply the probability of A because
19
00:02:47,189 --> 00:02:48,189
that is unaffected.
20
00:02:48,189 --> 00:02:49,189
So let me again write this.
21
00:02:49,189 --> 00:03:09,599
This implies, the probability of occurrence
of B has no effect that is occurrence of B
22
00:03:09,599 --> 00:03:21,540
has no effect
23
00:03:21,540 --> 00:03:38,819
on probability of occurrence of A. Therefore
in that scenario we can say A and B are independent.
24
00:03:38,819 --> 00:03:40,629
Let us simplify this further.
25
00:03:40,629 --> 00:03:48,049
So we say that A and B are independent if
probability of A given B is equal to the probability
26
00:03:48,049 --> 00:03:57,150
of A. But we have also defined probability
of A given B as probability of A intersection
27
00:03:57,150 --> 00:04:05,819
B divided by the probability of B implies
probability of A intersection B divided by
28
00:04:05,819 --> 00:04:16,500
the probability of B equals probability of
A which implies the probability of A intersection
29
00:04:16,500 --> 00:04:28,651
B equals the probability of A times the probability
of B. Right, so the probability, which, if
30
00:04:28,651 --> 00:04:31,520
you can see this, this is a very important
probability.
31
00:04:31,520 --> 00:04:38,300
So for Independent events A, B, the probability
of A intersection B that is the probability
32
00:04:38,300 --> 00:04:45,400
that both events A and B occur is simply the
product of the probabilities A and B.
33
00:04:45,400 --> 00:04:58,599
So the product, so the probability
34
00:04:58,599 --> 00:05:21,229
both A, B occur is simply equal to the product
35
00:05:21,229 --> 00:05:26,900
of probabilities A, B. That is, the probability
that both A and B occur that is probability
36
00:05:26,900 --> 00:05:34,389
of A intersection B is simply the probability
of A times the probability of B if A and B
37
00:05:34,389 --> 00:05:35,389
are independent.
38
00:05:35,389 --> 00:05:40,650
That is the probability that both A and B
occur is simply the product of the probabilities
39
00:05:40,650 --> 00:05:43,699
of events A and B.
40
00:05:43,699 --> 00:05:44,860
Similarly.
41
00:05:44,860 --> 00:05:47,889
One can also define independence the other
way.
42
00:05:47,889 --> 00:05:55,509
That is the probability ofÉnow similarly
let us start with this probability for Independent
43
00:05:55,509 --> 00:06:03,241
events, we have probability of A intersection
B equals probability of A times the probability
44
00:06:03,241 --> 00:06:19,180
of B which basically implies that the probability
of A intersection B divided by the probability
45
00:06:19,180 --> 00:06:29,659
of B is equal to the divided by the probability
of A is equal to the probability of B which
46
00:06:29,659 --> 00:06:37,300
also implies that this probability of A intersection
B divided by the probability of A is nothing
47
00:06:37,300 --> 00:06:48,080
but the probability of B given A is equal
to P of B. Therefore, if A and B are independent,
48
00:06:48,080 --> 00:06:53,260
we have said that the probability of A given
B is equal to the probability of A.
49
00:06:53,260 --> 00:06:56,850
.
Similarly, we have also derived from that
50
00:06:56,850 --> 00:07:02,330
property that the probability of B given A
is equal to the probability of B. Right?
51
00:07:02,330 --> 00:07:03,419
It works both ways.
52
00:07:03,419 --> 00:07:07,919
If A is independent B then B is independent
of A.
53
00:07:07,919 --> 00:07:15,080
Therefore, we can say now, that the condition
for Independence is that the probability of
54
00:07:15,080 --> 00:07:24,860
A given B is equal to the probability of A
or both.
55
00:07:24,860 --> 00:07:35,330
And that also implies that the probability
of B given A equals probability of B.
56
00:07:35,330 --> 00:07:49,590
This is, if A, B are independent.
57
00:07:49,590 --> 00:07:57,370
If A, B are independent events, we are saying
that the probability of A given B is equal
58
00:07:57,370 --> 00:08:02,320
to the probability of A, probability of B
given A is equal to the probability of B and
59
00:08:02,320 --> 00:08:07,979
further probability of A intersection B is
simply equal to the probability of A times
60
00:08:07,979 --> 00:08:09,530
the probability of B.
61
00:08:09,530 --> 00:08:10,530
All right?
62
00:08:10,530 --> 00:08:16,469
This is a simple technical definition of independence
and intuitively we have also said that this
63
00:08:16,469 --> 00:08:24,129
means that the occurrence of event B has no
bearing on the probability of occurrence of
64
00:08:24,129 --> 00:08:30,240
A. Also, the occurrence of event A has no
bearing on the probability of occurrence of
65
00:08:30,240 --> 00:08:35,900
event B. That is when events A and B are independent.
66
00:08:35,900 --> 00:08:37,680
Ok?
67
00:08:37,680 --> 00:08:43,159
Let us look at an example.
68
00:08:43,159 --> 00:08:50,350
Let us prove an important property as an example.
69
00:08:50,350 --> 00:09:26,170
If A and B are independent, show that A, B
complement are also independent.
70
00:09:26,170 --> 00:09:32,950
That is, if 2 events A and B are independent,
then we want to show that A and B complement
71
00:09:32,950 --> 00:09:34,310
are also independent.
72
00:09:34,310 --> 00:09:36,670
That is the event A is also independent.
73
00:09:36,670 --> 00:09:41,700
That is, if event A is independent of event
B then we want to show that event A is also
74
00:09:41,700 --> 00:09:45,280
independent of the complement of event B.
75
00:09:45,280 --> 00:09:51,020
And this can be seen as follows for instance
we have already seen yesterday.
76
00:09:51,020 --> 00:09:57,670
From our pictorial, from our diagrammatic
representation, we have already seen yesterday
77
00:09:57,670 --> 00:10:33,560
that if we have 2 events A, B, this is the
region A intersection B and this is the region
78
00:10:33,560 --> 00:10:36,190
A intersection B complement.
79
00:10:36,190 --> 00:10:47,550
And you can see that A intersection B complement
and A intersection B are as we had seen yesterday,
80
00:10:47,550 --> 00:10:54,050
these are mutually exclusive.
81
00:10:54,050 --> 00:11:04,830
We can see that the events A intersection
B complement and A intersection B are mutually
82
00:11:04,830 --> 00:11:06,010
exclusive.
83
00:11:06,010 --> 00:11:09,270
That is, their intersection is the null event.
84
00:11:09,270 --> 00:11:13,890
Further, the union of these 2 events is the
event A. Right?
85
00:11:13,890 --> 00:11:27,920
So A is equal to A intersection B complement
union A intersection B. Therefore, using the
86
00:11:27,920 --> 00:11:35,560
3rd axiom of probability, we can write probability
of A equals probability of A intersection
87
00:11:35,560 --> 00:11:44,380
B complement plus the probability of A intersection
B. However, we are given that A and B are
88
00:11:44,380 --> 00:11:45,380
independent.
89
00:11:45,380 --> 00:11:56,610
Therefore, we have probability of A intersection
B is equal to the probability of A times the
90
00:11:56,610 --> 00:12:04,810
probability of B. Therefore, now using this
property, we can say, substituting this property
91
00:12:04,810 --> 00:12:13,890
over here, in place of probability of A intersection
B, we have the probability of A equals probability
92
00:12:13,890 --> 00:12:26,350
of A intersection B complement plus probability
of A times the probability of B which implies
93
00:12:26,350 --> 00:12:35,690
the probability of A intersection B complement
equals probability of A minus probability
94
00:12:35,690 --> 00:12:47,190
of A times probability of B equals probability
of A times 1 minus probability of B.
95
00:12:47,190 --> 00:12:55,751
And now you can see that basically 1 minus
probability of B is nothing but the probability
96
00:12:55,751 --> 00:13:06,490
of B complement which basically implies probability
of A intersection B complement equals the
97
00:13:06,490 --> 00:13:20,800
probability of A times the probability of
B complement.
98
00:13:20,800 --> 00:13:25,640
Therefore what we have demonstrated is that
given A and B are independent, we are able
99
00:13:25,640 --> 00:13:31,500
to show that probability of A intersection
B complement is equal to the probability of
100
00:13:31,500 --> 00:13:34,280
A times the probability of B complement.
101
00:13:34,280 --> 00:13:38,570
Therefore A is also independent of B complement.
102
00:13:38,570 --> 00:13:43,820
Remember, the condition for independence is
the probability of B the intersection or the
103
00:13:43,820 --> 00:13:47,940
probability of joint event is equal to the
product of the probabilities of the individual
104
00:13:47,940 --> 00:13:48,940
events.
105
00:13:48,940 --> 00:13:53,110
Therefore, we have shown that since probability
of probability of A intersection B complement
106
00:13:53,110 --> 00:14:00,260
is equal to probability of A times the probability
of B complement, A is independent of B complement
107
00:14:00,260 --> 00:14:03,510
as well if A is independent of B.
108
00:14:03,510 --> 00:14:22,100
So this shows, this implies A is independent
of B. So this shows that A is independent
109
00:14:22,100 --> 00:14:26,770
of B complement.
110
00:14:26,770 --> 00:14:29,600
Let us look at another example.
111
00:14:29,600 --> 00:14:38,980
Let us go back to our M-ary PAM to understand
this.
112
00:14:38,980 --> 00:14:44,830
Remember we are looking at M-ary PAM where
M is equal to 4.
113
00:14:44,830 --> 00:14:48,090
PAM stands for pulse amplitude modulation.
114
00:14:48,090 --> 00:14:59,120
Our sample space is minus 3 alpha, minus alpha,
alpha, 3 alpha.
115
00:14:59,120 --> 00:15:14,540
And I have the probabilities, 1 by 8, 1 by
8, 1 by 4 and half.
116
00:15:14,540 --> 00:15:19,240
So we are looking at M-ary PAM constellation,
pulse amplitude modulation with M is equal
117
00:15:19,240 --> 00:15:20,240
to 4.
118
00:15:20,240 --> 00:15:25,490
I have the symbols, minus 3 alpha, minus alpha,
alpha, 3 alpha.
119
00:15:25,490 --> 00:15:30,980
The probabilities are 1 over 8, 1 over 8,
1 over 4 and half respectively.
120
00:15:30,980 --> 00:15:36,080
And let us also look at the events, A which
we have already defined previously.
121
00:15:36,080 --> 00:15:47,240
A is equal to minus 3 alpha, alpha and B equals
simply the event alpha corresponding to the
122
00:15:47,240 --> 00:15:48,330
symbol alpha.
123
00:15:48,330 --> 00:16:04,000
Now, let us ask the question, are
124
00:16:04,000 --> 00:16:08,490
events A, B, are these events independent?
125
00:16:08,490 --> 00:16:10,970
So let us use our definition for independence.
126
00:16:10,970 --> 00:16:15,750
So we have defined A as the event corresponding
to theÉsimilar to what we had done before,
127
00:16:15,750 --> 00:16:17,990
A is the set.
128
00:16:17,990 --> 00:16:22,980
A is the event minus 3 alpha containing the
sample points, minus 3 alpha, alpha.
129
00:16:22,980 --> 00:16:29,570
So A basically corresponds to observing the
symbol is either minus 3 alpha or alpha.
130
00:16:29,570 --> 00:16:33,030
And B, the event B corresponds to the symbol
alpha.
131
00:16:33,030 --> 00:16:38,060
And therefore, now we are asking the question,
are these 2 events independent?
132
00:16:38,060 --> 00:16:43,740
Let us use our definition of independence
to check if these 2 are independent.
133
00:16:43,740 --> 00:16:52,600
Now we have probability of A which we have
already derived before is simply the probability
134
00:16:52,600 --> 00:17:06,839
of minus 3 alpha, alpha which is equal to
1 by 4 or which is equal to 1 by 8 plus 1
135
00:17:06,839 --> 00:17:13,820
by 4 is equal to 3 by 8.
136
00:17:13,820 --> 00:17:24,130
Probability of B is equal to the probability
of simply a single sample point alpha which
137
00:17:24,130 --> 00:17:26,250
is equal to 1 by 4.
138
00:17:26,250 --> 00:17:28,669
Therefore the probability of A intersection
B.
139
00:17:28,669 --> 00:17:40,890
Therefore the probability of A intersection
B therefore probability of A times probability
140
00:17:40,890 --> 00:17:51,080
of B is equal to 3 by 8 times 1 by 4 which
is 3 by 32.
141
00:17:51,080 --> 00:18:00,090
Now let us look at the event A intersection
B. A intersection B is simply minus 3 alpha,
142
00:18:00,090 --> 00:18:08,950
alpha intersection alpha which is basically
the single sample point alpha.
143
00:18:08,950 --> 00:18:19,330
Therefore, probability of A intersection B
equals simply the probability of alpha which
144
00:18:19,330 --> 00:18:23,029
is equal to 1 by 4.
145
00:18:23,029 --> 00:18:25,230
Now you can see the probability.
146
00:18:25,230 --> 00:18:28,330
Now you can see from both of these.
147
00:18:28,330 --> 00:18:30,730
This is probability of A times probability
of B.
148
00:18:30,730 --> 00:18:36,840
This is probability of A intersection B. Now
you can see probability of A intersection
149
00:18:36,840 --> 00:18:46,110
B is equal to 1 by 4 which is not equal to
the probability of A times the probability
150
00:18:46,110 --> 00:18:54,370
of B which is basically equal to 3 divided
by 32.
151
00:18:54,370 --> 00:18:58,809
So we are saying that the probability of A
intersection B is 1 by 4.
152
00:18:58,809 --> 00:19:03,450
Probability of A times the probability of
B is basically 3 divided by 32.
153
00:19:03,450 --> 00:19:08,419
Therefore the probability of A intersection
B that is the probability that both events
154
00:19:08,419 --> 00:19:13,120
occurring is not equal to probability of A
times the probability of B that is the product
155
00:19:13,120 --> 00:19:14,120
of probabilities.
156
00:19:14,120 --> 00:19:17,100
Therefore, A and B are not independent.
157
00:19:17,100 --> 00:19:42,009
Therefore, we are saying something interesting,
therefore
158
00:19:42,009 --> 00:19:49,700
A and B are not independent events.
159
00:19:49,700 --> 00:19:57,320
And this is also natural because A is the
symbol minus 3 alpha or alpha.
160
00:19:57,320 --> 00:20:02,950
A denotes the events observing the symbols
minus 3 alpha or alpha, B denotes observing
161
00:20:02,950 --> 00:20:04,110
the symbol alpha.
162
00:20:04,110 --> 00:20:08,549
Therefore, if one has told you that he has
observed B that is he has observed symbol
163
00:20:08,549 --> 00:20:13,879
alpha, then definitely, you can conclude that
event A has happened because because event
164
00:20:13,879 --> 00:20:17,029
A is either minus 3 alpha or alpha.
165
00:20:17,029 --> 00:20:24,470
So given B conveys a lot of information about
event A. In fact, given event B has occurred,
166
00:20:24,470 --> 00:20:28,419
one can conclude that event A has definitely
occurred.
167
00:20:28,419 --> 00:20:34,919
Therefore, observing event B has an effect
on observing, on the probability of occurrence
168
00:20:34,919 --> 00:20:38,690
of A. Therefore, B and A are not independent.
169
00:20:38,690 --> 00:20:55,929
Remember, what we are saying is B, our A equals
minus 3 alpha equals observing
170
00:20:55,929 --> 00:21:00,100
that is our symbol minus 3 alpha or alpha.
171
00:21:00,100 --> 00:21:15,889
B equals, let us put it, the set A basically
says that the symbol equals minus 3 alpha
172
00:21:15,889 --> 00:21:23,870
or alpha and B symbol is equal to basically
alpha.
173
00:21:23,870 --> 00:21:29,200
So if one has told you that B has occurred
that the symbol is Alpha, then definitely
174
00:21:29,200 --> 00:21:30,419
A has occurred.
175
00:21:30,419 --> 00:21:33,740
That is, the symbol is either minus 3 alpha
or alpha.
176
00:21:33,740 --> 00:21:40,999
And therefore, the occurrence of B has an
effect on the occurrence of A. Or B affects
177
00:21:40,999 --> 00:21:47,080
the probability of occurrence of A. Therefore,
by definition they are not independent.
178
00:21:47,080 --> 00:21:52,759
Therefore we can say that the probabilityÉand
remember, we had seen this previously probability
179
00:21:52,759 --> 00:22:07,200
of A given B equals 1 which is greater than
the probability of A which is equal to 3 by
180
00:22:07,200 --> 00:22:08,200
8.
181
00:22:08,200 --> 00:22:17,409
Therefore probability of A given B is not
equal to probability of A which basically
182
00:22:17,409 --> 00:22:35,019
also again implies A, B are not independent.
183
00:22:35,019 --> 00:22:46,860
Similarly, probability of B given A is equal
to 2 by 3rd which is greater than the probability
184
00:22:46,860 --> 00:22:51,610
of B which is equal to 1 by 4.
185
00:22:51,610 --> 00:23:01,500
Therefore probability of B given A is not
equal to the probability of B which also basically
186
00:23:01,500 --> 00:23:05,309
once again implies thatÉ
187
00:23:05,309 --> 00:23:20,749
É All of these imply the same thing, A, B
188
00:23:20,749 --> 00:23:23,340
are not independent.
189
00:23:23,340 --> 00:23:24,580
Correct?
190
00:23:24,580 --> 00:23:29,020
So this is the key aspect here.
191
00:23:29,020 --> 00:23:36,450
That is what we are saying is, these 2 events,
A, B where A denotes the event containing
192
00:23:36,450 --> 00:23:42,100
the sample points minus 3 alpha and alpha,
B denotes the event containing the sample
193
00:23:42,100 --> 00:23:46,480
point alpha, these 2 events are not independent.
194
00:23:46,480 --> 00:23:51,519
Because the occurrence of one has an effect,
has an impact on the probability of occurrence
195
00:23:51,519 --> 00:23:52,519
of the other.
196
00:23:52,519 --> 00:23:57,179
Given that B has occurred has an impact on
the probability of occurrence of A. Given
197
00:23:57,179 --> 00:24:00,940
that A has occurred has an effect impact on
the probability of occurrence of B. Therefore
198
00:24:00,940 --> 00:24:05,360
A and B are not independent.
199
00:24:05,360 --> 00:24:11,091
We have also verified this just using our
definition of independence that is the probability
200
00:24:11,091 --> 00:24:16,330
of joint event, probability of A intersection
B is not equal to probability of A times the
201
00:24:16,330 --> 00:24:20,249
probability of B. Therefore these events are
not independent.
202
00:24:20,249 --> 00:24:24,769
We will look at some other examples of independence
in subsequent modules.
203
00:24:24,769 --> 00:24:31,039
Let us stop this module here.
204
00:24:31,039 --> 00:24:51,019
Thank you very much.