1
00:00:13,780 --> 00:00:17,779
Hello, welcome to to another module in this
massive open online course on probability
2
00:00:17,779 --> 00:00:22,090
and random variables for wireless communications.
In the previous module, we have had looked
3
00:00:22,090 --> 00:00:25,660
at the experiments a lot and outcomes of the
experiment , so the experiment in and the
4
00:00:25,660 --> 00:00:34,210
concepts of sample space, events and other
things.
5
00:00:34,210 --> 00:00:51,570
Let us now start with a formal definition
of the axioms of probability. There are 3
6
00:00:51,570 --> 00:00:59,440
axioms of probability which are very important
and I’m going to list these. So, I have
7
00:00:59,440 --> 00:01:06,570
the 3 axioms of probability. I have axiom
1 which states that the probability, so what
8
00:01:06,570 --> 00:01:15,850
is the probability? So let us start with the
probability formal definition. The probability
9
00:01:15,850 --> 00:01:45,649
is a measure of any event A, the probability
measure of any set A
10
00:01:45,649 --> 00:01:56,729
satisfies the following axioms and those axioms
are as follows. 1st first, let us have, the
11
00:01:56,729 --> 00:02:05,149
first axiom, that is the probability measure
of any event A is greater than or equal to
12
00:02:05,149 --> 00:02:11,200
0. The probability measure of any set A is
greater than…so we’re saying, the probability
13
00:02:11,200 --> 00:02:18,250
is a measure which is defined for events A
of the Sample space, S.
14
00:02:18,250 --> 00:02:24,290
And it follows some basic axioms. The basic
axiom is that this probability measure is
15
00:02:24,290 --> 00:02:36,980
nonnegative. Right? This probability measure
is nonnegative. Which basically says that
16
00:02:36,980 --> 00:02:46,950
the probability of any event A is greater
than or equal to 0. Now write axiom 2 which
17
00:02:46,950 --> 00:02:54,200
is that the probability of the Sample space,
the measure of the entire sample space, the
18
00:02:54,200 --> 00:03:00,290
probability measure of the entire sample space
that is if we look at the entire sample space,
19
00:03:00,290 --> 00:03:07,290
the probability measure of the entire sample
space is unity or one. And the third axiom
20
00:03:07,290 --> 00:03:28,209
is basically that
probability of A union B equals the probability
21
00:03:28,209 --> 00:03:46,659
of A plus the probability of B if A intersection
B is the null event phi or basically A, B
22
00:03:46,659 --> 00:03:56,959
are
mutually exclusive.
23
00:03:56,959 --> 00:04:04,840
So we’re saying that if 2 sets or if 2 events,
A and B are mutually exclusive, that is A
24
00:04:04,840 --> 00:04:10,659
intersection B is the null event, A intersection
B is equal to phi which is the null event,
25
00:04:10,659 --> 00:04:15,640
then the probability of the union of these
2 events, the probability of A union B, the
26
00:04:15,640 --> 00:04:22,500
probability measure of A union B is simply
the sum of the probability measures of A and
27
00:04:22,500 --> 00:04:28,280
probability measure of B. So probability of
A union B equals probability of A plus probability
28
00:04:28,280 --> 00:04:37,280
of B if A intersection B is basically the
null event or A and B are mutually exclusive
29
00:04:37,280 --> 00:04:41,990
events. So we have 3 axioms that the probability
measure satisfies.
30
00:04:41,990 --> 00:04:47,031
First is that the probability measure is not
negative. The probability of A or probability
31
00:04:47,031 --> 00:04:53,180
of any event A, PA is greater than or equal
to 0. The probability of the sample space,
32
00:04:53,180 --> 00:04:59,310
S is equal to 1 and the probability of A union
B equals probability of A plus probability
33
00:04:59,310 --> 00:05:09,270
of B if A intersection B equals phi or A and
B are mutually exclusive events. So now, using
34
00:05:09,270 --> 00:05:13,310
these probability axioms, let us look at some
basic properties.
35
00:05:13,310 --> 00:05:20,080
First, basic properties which follow from
the probability axioms. Note that I have from
36
00:05:20,080 --> 00:05:27,560
my previous proof, I have S, as we have seen
in the previous model, the Sample space S
37
00:05:27,560 --> 00:05:41,639
is equal to any set A
union A complement equals the Sample space.
38
00:05:41,639 --> 00:05:51,009
Further, any A and A complement are also mutually
exclusive. That’s what we saw in the previous
39
00:05:51,009 --> 00:05:58,069
module. Therefore I have 1 which is equal
to the probability space, probability of the
40
00:05:58,069 --> 00:06:09,840
Sample space from the axioms which is equal
to the probability because S is equal to A
41
00:06:09,840 --> 00:06:13,940
union A complement. But A and A complement
are also mutually exclusive. Therefore from
42
00:06:13,940 --> 00:06:28,340
the third axiom, this is probability of A
plus probability of A complement. This follows
43
00:06:28,340 --> 00:06:39,039
from the third axiom. And this is therefore
also now you can see, this implies that probability
44
00:06:39,039 --> 00:06:46,650
of A equals one minus the probability of a
complement.
45
00:06:46,650 --> 00:07:00,669
But once again, from the First axiom, I have,
probability of A
46
00:07:00,669 --> 00:07:11,430
complement is greater than or equal to 0,
which implies, probability of any set A equals
47
00:07:11,430 --> 00:07:20,990
One minus probability of A complement is less
than or equal to what we are seeing is the
48
00:07:20,990 --> 00:07:25,912
following thing. We said, the probability
of A complement is greater than or equal to
49
00:07:25,912 --> 00:07:31,480
0 and the probability of the set A is One
minus the probability of A complement. Therefore
50
00:07:31,480 --> 00:07:37,129
the probability of set A is less than or equal
to 1. Therefore together with the First axiom,
51
00:07:37,129 --> 00:07:43,889
we can now say that the probability of any
set A lies between 0 and 1. So for any set
52
00:07:43,889 --> 00:08:04,520
A or for any event A, we must have 0 less
than or equal to the probability of the event
53
00:08:04,520 --> 00:08:08,581
A less than or equal to 1. Probability of
A is greater than or equal to 0 from the First
54
00:08:08,581 --> 00:08:14,220
axiom and now we have derived that the probability
of any event A is less than or equal to 1.
55
00:08:14,220 --> 00:08:18,919
If we put them both together, we have the
result that the probability of any event A
56
00:08:18,919 --> 00:08:25,949
must lie between 0 and 1. That is the First
result that we have. That is the First important
57
00:08:25,949 --> 00:08:35,849
result that we have. That is the probability
of any event A lies between 0 and 1. Let us
58
00:08:35,849 --> 00:08:40,140
now look at the next result.
59
00:08:40,140 --> 00:08:52,870
That is, I have, now the next result is as
follows. We know that the empty set. If I
60
00:08:52,870 --> 00:08:58,210
take the union of the samples Sample space
with the null event, that is again the Sample
61
00:08:58,210 --> 00:09:09,061
space. Correct? And if I take the intersection
of the Sample space and the null event, that
62
00:09:09,061 --> 00:09:15,590
gives the null event because the null event
does not contain any event. So its intersection
63
00:09:15,590 --> 00:09:22,710
with any event in particular intersection
with the Sample space is again the null event
64
00:09:22,710 --> 00:09:26,710
because the intersection is basically Sample
points which are both in the Sample space
65
00:09:26,710 --> 00:09:30,870
and the null event. And the null event does
not contain any Sample points. Therefore,
66
00:09:30,870 --> 00:09:36,830
the intersection is the empty set or the null
event.
67
00:09:36,830 --> 00:09:45,430
Now again, using the Second axiom, I have
1 equals the probability of the Sample space
68
00:09:45,430 --> 00:09:55,760
which is equal to the probability of S union
the null event. Now you can see, S intersection
69
00:09:55,760 --> 00:10:00,790
null event is the null event, which means
S and null event are disjoint which equals
70
00:10:00,790 --> 00:10:09,220
to probability of S plus the probability of
null event. And the probability of S, we said
71
00:10:09,220 --> 00:10:16,920
is equal to 1 from axiom 2. So this is equal
to 1. So I have one equal to 1plus the probability
72
00:10:16,920 --> 00:10:28,810
of null event, which means the probability
of null event equals 1 minus 1 equals 0. Therefore
73
00:10:28,810 --> 00:10:37,410
we have our Second property that we have derived
that is the probability of null event is 0
74
00:10:37,410 --> 00:10:44,360
or this is also basically an impossible event.
The probability of null event is 0 which is
75
00:10:44,360 --> 00:10:50,690
also known as an impossible event. Okay?
76
00:10:50,690 --> 00:10:55,040
za
And 3 again, some other important property
77
00:10:55,040 --> 00:11:00,030
that we’re not going to design elaborately.
3, if A and B are any events, then probability
78
00:11:00,030 --> 00:11:11,210
of A is less than probability of B if A is
a subset of B. That is, the probability is
79
00:11:11,210 --> 00:11:19,320
a measure and we’re saying, if A is a subset
of B, that is if A, the event A is the subset
80
00:11:19,320 --> 00:11:24,750
of event B, then the probability measure of
the event A is less than or equal to the probability
81
00:11:24,750 --> 00:11:27,990
measure of B.
82
00:11:27,990 --> 00:11:33,900
And 4 is another important property which
I would like to prove, which is the following
83
00:11:33,900 --> 00:11:57,490
thing. If A and B are any events, then I have
the probability of A union B equals the probability
84
00:11:57,490 --> 00:12:17,580
of A plus the probability of B
85
00:12:17,580 --> 00:12:27,700
minus the probability of A intersection B.
And this can be seen as follows. For instance,
86
00:12:27,700 --> 00:12:53,530
if I have 2 events A, B, this region of course,
if I look at this region, this region is A
87
00:12:53,530 --> 00:13:03,430
intersection B and this region is now if I
look at this region that is the region which
88
00:13:03,430 --> 00:13:13,630
is present in B. So this is my set A, this
is my set B. This is T A intersection B. This
89
00:13:13,630 --> 00:13:23,190
is the region which is present in B but not
in A and this region is therefore B intersection
90
00:13:23,190 --> 00:13:30,940
A complement. And this is the region which
is present in A but not in B. Therefore this
91
00:13:30,940 --> 00:13:38,430
is the region A intersection B complement.
Correct?
92
00:13:38,430 --> 00:13:48,180
So I have 2 sets A, B or 2 events A and B
and I can decompose these 2 events A and B
93
00:13:48,180 --> 00:13:56,040
into 3 distinct regions. One is A intersection
B that is the overlap between these 2 events.
94
00:13:56,040 --> 00:14:03,660
A intersection B complement that is all the
Sample points which are in A but not in B.
95
00:14:03,660 --> 00:14:09,460
And B intersection A complement which is all
the points which are in B but not in A.
96
00:14:09,460 --> 00:14:16,670
And you can see from this picture that these
3 events, A intersection B complement, A intersection
97
00:14:16,670 --> 00:14:23,730
B and B intersection A complement, these are
basically, you can see that these 3 events
98
00:14:23,730 --> 00:14:36,860
are mutually exclusive. That is A intersection
B complement, A intersection B and B intersection
99
00:14:36,860 --> 00:14:53,250
A complement, these, you can clearly see,
these are mutually exclusive events because
100
00:14:53,250 --> 00:14:57,440
they do not have any overlap.
101
00:14:57,440 --> 00:15:15,430
And therefore, naturally, it follows that
I have the probability of A union B
102
00:15:15,430 --> 00:15:22,490
equals and further the union of all these
is basically A union B and therefore I have
103
00:15:22,490 --> 00:15:30,730
the probability of A union B is the probability
of A intersection B complement plus the probability
104
00:15:30,730 --> 00:15:47,580
of A intersection B plus the probability of
B intersection A complement. Correct? So what
105
00:15:47,580 --> 00:15:55,930
did we say? We said that any 2 sets A and
B can be represented as 3 distinct parts.
106
00:15:55,930 --> 00:16:01,300
That is A intersection B compliment, A intersection
B and B intersection A complement. These are
107
00:16:01,300 --> 00:16:08,690
disjoint and further, their union is equal
to A union B. Therefore the probability of
108
00:16:08,690 --> 00:16:16,320
A union B is the sum of these 3 distinct mutually
exclusive components. That is, probability
109
00:16:16,320 --> 00:16:21,760
of A intersection B complement plus probability
of A intersection B plus the probability of
110
00:16:21,760 --> 00:16:24,320
B intersection A complement.
111
00:16:24,320 --> 00:16:30,680
Now let us look at this event, probability
of A intersection B complement.
112
00:16:30,680 --> 00:16:41,230
If you can see, A intersection, now if you
look at A, the event A is the union of A intersection
113
00:16:41,230 --> 00:16:43,810
B complement and A intersection B.
114
00:16:43,810 --> 00:16:56,180
Therefore I have probability of A equals probability
of A intersection B complement plus probability
115
00:16:56,180 --> 00:17:13,299
of A intersection B which implies the probability
of A intersection B complement equals probability
116
00:17:13,299 --> 00:17:23,670
of A minus the probability of A intersection
B. Further, if you look at this, similarly
117
00:17:23,670 --> 00:17:35,410
probability of B intersection A complement
equals the probability of B minus the probability
118
00:17:35,410 --> 00:17:43,240
of A intersection B. Now, substituting this
probability of A intersection B complement
119
00:17:43,240 --> 00:17:52,530
over here, and probability of B intersection
A complement over here, I have the probability
120
00:17:52,530 --> 00:18:05,410
of A union B which is equal to the probability
of A minus probability of A intersection B
121
00:18:05,410 --> 00:18:13,900
plus probability of A intersection B plus
the probability of B intersection A complement
122
00:18:13,900 --> 00:18:27,490
which is the probability of B minus the probability
of A intersection B which is therefore equal
123
00:18:27,490 --> 00:18:43,410
to the probability of A plus probability of
B minus the probability of A intersection
124
00:18:43,410 --> 00:18:47,640
B. Therefore the probability of A union B
equals probability of A plus probability of
125
00:18:47,640 --> 00:18:54,230
B minus the probability of A intersection
B. This is one of the fundamental results
126
00:18:54,230 --> 00:19:00,600
in probability. That is, the probability of
A union B, any 2 events, A and B is the probability
127
00:19:00,600 --> 00:19:06,900
of A plus the probability of B minus the probability
of the intersection of these 2 events that
128
00:19:06,900 --> 00:19:10,680
is the probability of A intersection B.
129
00:19:10,680 --> 00:19:15,011
Let us take a simple example to understand
this. It is look at a simple example. Let
130
00:19:15,011 --> 00:19:23,970
us go back to our M ary PAM. M ary pulse amplitude
modulation. Let us take a simple example to
131
00:19:23,970 --> 00:19:32,120
understand this. Let us go back to our purse
pulse amplitude, M ary pulse amplitude modulation
132
00:19:32,120 --> 00:19:46,000
which we introduced in the previous module.
This is our M ary PAM and let us say that
133
00:19:46,000 --> 00:19:51,560
we have the following probabilities. We have
the following probabilities for the Sample
134
00:19:51,560 --> 00:19:59,320
points, the various Sample points. Let us
say, the probability of this minus 3 alpha
135
00:19:59,320 --> 00:20:05,920
is basically 1 over 8, probability of minus
alpha is 1 over 8, probability of alpha is
136
00:20:05,920 --> 00:20:13,070
1 over 4, probability of 3 alpha is how half.
Therefore now if you look at the sample set,
137
00:20:13,070 --> 00:20:26,600
probability of the sample set is 1 by 8 plus
1 by 8 plus 1 by 4 plus half which is equal
138
00:20:26,600 --> 00:20:31,910
to 1. So the probability of the entire sample
space is equal to 1. So that satisfies the
139
00:20:31,910 --> 00:20:35,520
probability axiom.
140
00:20:35,520 --> 00:20:47,200
Further, let A be the event minus 3 alpha,
alpha, then the probability of event, A is
141
00:20:47,200 --> 00:20:52,960
the probability of minus 3 alpha which is
1 by 8 plus the probability of alpha which
142
00:20:52,960 --> 00:21:04,060
is 1 by 4 which is therefore equal to 3 by
8. Let B be the event, let us define another
143
00:21:04,060 --> 00:21:16,260
event, B which is equal to alpha, 3 alpha
which implies the probability of B is equal
144
00:21:16,260 --> 00:21:22,320
to probability of alpha which is equal to
1 by 4 plus plurality of 3 alpha which is
145
00:21:22,320 --> 00:21:28,900
half which is equal to 3 by 4.
146
00:21:28,900 --> 00:21:46,110
Now if you look at this, I have A union B
which is equal to from this which is minus
147
00:21:46,110 --> 00:22:00,200
3 alpha, alpha, 3 alpha implies the probability
of A union B is given as the probability A
148
00:22:00,200 --> 00:22:08,700
union B is the probability of minus 3 alpha
which is 1 by 4 plus the probability of alpha
149
00:22:08,700 --> 00:22:13,970
which is this is probability of minus 3 alpha
which is 1 by 8 plus probability of alpha
150
00:22:13,970 --> 00:22:22,810
which is 1 by 4 plus probability of 3 alpha
which is half which is equal to 7 by 8.
151
00:22:22,810 --> 00:22:34,850
Probability of A intersection B , now A intersection
B you have to look at A intersection B, A
152
00:22:34,850 --> 00:22:47,800
intersection B is basically we have minus
3 alpha, alpha intersection alpha, 3 alpha
153
00:22:47,800 --> 00:22:57,950
in event B which is equal to the only common
sample point is basically alpha. Therefore
154
00:22:57,950 --> 00:23:06,470
probability of A intersection B is simply
the probability of alpha which is equal to
155
00:23:06,470 --> 00:23:20,510
1 by 4. And now, probability of A plus probability
of B minus probability of A intersection B
156
00:23:20,510 --> 00:23:27,711
is simply probability of A as we have derived
previously that is 3 by 8 plus probability
157
00:23:27,711 --> 00:23:36,870
of B that is 3 by 4 minus probability of A
intersection B equals 1 by 4. That is 3 by
158
00:23:36,870 --> 00:23:48,270
8 plus half which is equal to 7 by 8 which
is equal to probability of A union B. And
159
00:23:48,270 --> 00:23:57,059
therefore, what we have seen from this example
is that probability indeed, probability of
160
00:23:57,059 --> 00:24:10,150
A union B equals probability of A plus probability
of B minus probability of A intersection B.
161
00:24:10,150 --> 00:24:15,100
So what we have seen is, we have considered
this simple examples example from M ary PAM,
162
00:24:15,100 --> 00:24:21,250
M ary pulse rate would modulation for a wireless
communication system and we have derived that
163
00:24:21,250 --> 00:24:25,559
the probability, we have shown simply, we
have verified the result that we had derived
164
00:24:25,559 --> 00:24:31,320
earlier that is the probability measure of
A union B is the probability measure of A
165
00:24:31,320 --> 00:24:36,651
plus the probability measure of B minus the
probability measure of A intersection B. So
166
00:24:36,651 --> 00:24:41,020
let us conclude this module with this example.
We will look at some other aspects in subsequent
167
00:24:41,020 --> 00:25:06,800
modules. Thank you very much.