1
00:00:15,150 --> 00:00:20,439
Hello, welcome to this massive open online
course on probability and random variables
2
00:00:20,439 --> 00:00:22,660
for wireless communications.
3
00:00:22,660 --> 00:00:23,660
Alright!
4
00:00:23,660 --> 00:00:27,910
So let us start our discussion with a concept
of an experiment.
5
00:00:27,910 --> 00:00:43,129
So when we perform an experiment, we frequently
perform experiments and those experiments
6
00:00:43,129 --> 00:00:53,089
result in, those experiments lead to outcomes.
7
00:00:53,089 --> 00:01:03,790
So these experiments lead to outcomes and
these outcomes are random in nature.
8
00:01:03,790 --> 00:01:22,640
These outcomes are random if they cannot be
exactly predicted.
9
00:01:22,640 --> 00:01:27,250
So the outcomes of these experiments, we say
they are random if they cannot be exactly
10
00:01:27,250 --> 00:01:28,250
predicted.
11
00:01:28,250 --> 00:01:32,640
For instance, let us take a simple example.
12
00:01:32,640 --> 00:01:36,920
Since this course is about probability for
communication systems or wireless communication
13
00:01:36,920 --> 00:01:44,670
systems, let’s take a source, an information
source.
14
00:01:44,670 --> 00:01:51,210
In communication, we talk about information
source.
15
00:01:51,210 --> 00:01:56,940
And let’s say, it is generating information
symbol 0s and 1s.
16
00:01:56,940 --> 00:02:02,870
For instance, 0100 and this is the sequence
of information.
17
00:02:02,870 --> 00:02:12,480
Now for this information source, at any instant,
we do not know exactly what is the next information
18
00:02:12,480 --> 00:02:13,480
symbol.
19
00:02:13,480 --> 00:02:17,000
It can either be 0 or it can be 1.
20
00:02:17,000 --> 00:02:22,310
So the next information symbol, there is uncertainty
about the successive information symbol.
21
00:02:22,310 --> 00:02:25,170
There is uncertainty about the next information
symbol.
22
00:02:25,170 --> 00:02:29,190
So these information symbols, these are random
in nature.
23
00:02:29,190 --> 00:02:32,950
We say, the information symbols are being
generated randomly.
24
00:02:32,950 --> 00:02:56,629
So there is uncertainty about the
25
00:02:56,629 --> 00:02:58,909
next information symbol.
26
00:02:58,909 --> 00:03:13,829
Hence, these information symbols are random.
27
00:03:13,829 --> 00:03:26,379
We say, these information symbols are
random or they are being generated randomly.
28
00:03:26,379 --> 00:03:30,030
That is a qualitative description of randomness.
29
00:03:30,030 --> 00:03:36,090
We say, the information symbols are being
generated randomly because there is uncertainty
30
00:03:36,090 --> 00:03:40,249
regarding the next information symbol that
is generated by the source.
31
00:03:40,249 --> 00:03:46,379
So let us not develop a framework to formally
characterize probability and randomness.
32
00:03:46,379 --> 00:03:48,849
So let us start with building a framework.
33
00:03:48,849 --> 00:03:57,340
The 1st thing that we would like to define
is what is known as the Sample space.
34
00:03:57,340 --> 00:04:24,400
Now, the Sample space is simply put, this
is
35
00:04:24,400 --> 00:04:28,620
the set of all possible outcomes of the experiment.
36
00:04:28,620 --> 00:04:35,370
The set of all possible outcomes of the experiment,
this is known as the Sample space.
37
00:04:35,370 --> 00:04:39,389
The Sample space is the set of all possible
outcomes of the experiment.
38
00:04:39,389 --> 00:04:44,960
For instance, let us again take a pertinent
example from communication, example from the
39
00:04:44,960 --> 00:04:52,270
context of communication for Sample space.
40
00:04:52,270 --> 00:05:04,320
Let us look at an example and in this example,
we are going to look at a digital modulation
41
00:05:04,320 --> 00:05:06,949
format.
42
00:05:06,949 --> 00:05:33,710
More specifically, we are going to consider
what is known as M ary PAM or M ary Pulse
43
00:05:33,710 --> 00:05:38,199
amplitude modulation.
44
00:05:38,199 --> 00:05:43,660
So I would like to look, illustrate this concept
of Sample space, let us look at this digital
45
00:05:43,660 --> 00:05:48,349
modulation constellation which I’m terming,
which is an M ary PAM.
46
00:05:48,349 --> 00:05:51,620
Some of you who are already familiar with
digital communication must know, this is M
47
00:05:51,620 --> 00:05:58,280
ary PAM or M ary Pulse Amplitude Modulation
where you choose from one of several amplitude
48
00:05:58,280 --> 00:06:00,919
levels to convey information.
49
00:06:00,919 --> 00:06:14,050
For instance, let us choose M is equal to
4 implies, basically that there are 4 symbols.
50
00:06:14,050 --> 00:06:18,509
These symbols are equally spaced on a line.
51
00:06:18,509 --> 00:06:28,130
So I have minus A minus 3 let’s call this
alpha,.
52
00:06:28,130 --> 00:06:34,930
Minus 3 alpha, minus alpha, alpha comma 3
alpha.
53
00:06:34,930 --> 00:06:37,800
So these are the 4 symbols.
54
00:06:37,800 --> 00:06:38,800
Right?
55
00:06:38,800 --> 00:06:46,410
So I have 4 symbols, minus 3 alpha, minus
alpha, alpha and 3 alpha spaced at intervals
56
00:06:46,410 --> 00:06:47,410
of 2 alpha.
57
00:06:47,410 --> 00:06:53,009
This is pulse amplitude modulation that is
choosing from one of 4 possible amplitude
58
00:06:53,009 --> 00:06:57,289
levels to convey information and these are
chosen randomly.
59
00:06:57,289 --> 00:07:01,120
That is the next, there is uncertainty about
the next amplitude level.
60
00:07:01,120 --> 00:07:09,400
So the Sample space clearly for this example,
my Sample space here is basically equal to,
61
00:07:09,400 --> 00:07:20,569
the Sample space is nothing but the set of
amplitude levels minus 3 alpha, minus alpha,
62
00:07:20,569 --> 00:07:22,159
alpha, 3 alpha.
63
00:07:22,159 --> 00:07:25,780
So for this M ary PAM where M is equal to
4.
64
00:07:25,780 --> 00:07:40,060
So M is equal to 4, I’m considering an M
ary PAM example and my Sample space for this
65
00:07:40,060 --> 00:07:51,729
is given as minus 3 alpha, minus alpha, alpha,
3 alpha.
66
00:07:51,729 --> 00:08:01,139
And each of these elements of this Sample
space is known as a Sample point.
67
00:08:01,139 --> 00:08:10,659
So these are known as the Sample points.
68
00:08:10,659 --> 00:08:16,770
So I have a Sample space which consists of
all the possible outcomes of the experiment
69
00:08:16,770 --> 00:08:18,460
about which there is uncertainty.
70
00:08:18,460 --> 00:08:20,650
Hence these outcomes are random.
71
00:08:20,650 --> 00:08:27,979
And each of these possible outcomes, each
of these single possible outcomes, these are
72
00:08:27,979 --> 00:08:31,029
known as Sample points.
73
00:08:31,029 --> 00:08:37,229
Now, also let us define what is known as an
Event.
74
00:08:37,229 --> 00:08:40,159
An event A in this.
75
00:08:40,159 --> 00:08:53,910
Let us define an event, the notion of an event
which is the other important thing.
76
00:08:53,910 --> 00:09:15,030
An event is any
subset of S. For example, consider A, the
77
00:09:15,030 --> 00:09:20,470
set A is equal to minus 3 alpha, alpha.
78
00:09:20,470 --> 00:09:30,970
Clearly A is, we can observe that A is the
subset of S. So, A consists of the symbols
79
00:09:30,970 --> 00:09:34,030
minus 3 alpha, alpha.
80
00:09:34,030 --> 00:09:35,030
Correct?
81
00:09:35,030 --> 00:09:40,600
So A is clearly a subset of S. Therefore A
is an event.
82
00:09:40,600 --> 00:09:43,360
A is a random event of this experiment.
83
00:09:43,360 --> 00:09:48,570
So, A is an event.
84
00:09:48,570 --> 00:09:54,000
And also from elementary set theory, you must
be familiar with definition of subset.
85
00:09:54,000 --> 00:10:16,970
A is a subset of S implies every element of
A also belongs to S. So A is a subset of S
86
00:10:16,970 --> 00:10:26,780
implies that every element of A is also present
in S. Such a set A is considered which consists
87
00:10:26,780 --> 00:10:33,090
of, which is a subset of Sample space, S.
Therefore it consists of some sample points
88
00:10:33,090 --> 00:10:38,190
in S. That is known as an event.
89
00:10:38,190 --> 00:10:44,820
Also further from elementary set theory, you
should be familiar with the concept of a complement
90
00:10:44,820 --> 00:10:48,330
of a set or the complement of an event.
91
00:10:48,330 --> 00:10:54,040
Now let us consider, to illustrate it pictorially.
92
00:10:54,040 --> 00:11:04,350
If this is my Sample space, S. If this is
my event A, all the events, all the Sample
93
00:11:04,350 --> 00:11:19,310
points that are in S but not in A, for instance,
this shaded region over here represents the
94
00:11:19,310 --> 00:11:20,310
complement.
95
00:11:20,310 --> 00:11:25,500
So
A complement, so from elementary set theory,
96
00:11:25,500 --> 00:11:44,660
A complement equals the Sample points in S
but not in A . The complement of A implies
97
00:11:44,660 --> 00:12:13,720
the Sample points A complement i.e. equals
sample points in S but not in A. So, A Complement
98
00:12:13,720 --> 00:12:22,600
is the set of sample points which are in the
Sample space, S but not in A.
99
00:12:22,600 --> 00:12:25,460
For instance, let us go back to our example.
100
00:12:25,460 --> 00:12:34,910
I have S is equal to minus 3 alpha, minus
alpha, alpha, 3 alpha.
101
00:12:34,910 --> 00:12:42,610
I have my set A which we have defined as minus
3 alpha, alpha.
102
00:12:42,610 --> 00:12:53,820
Therefore A complement is the set of elements
that are in S but not in A. So A complement
103
00:12:53,820 --> 00:13:01,240
equals minus alpha, you can see that the elements,
the Sample points in S but not in A are minus
104
00:13:01,240 --> 00:13:03,360
alpha, 3 alpha.
105
00:13:03,360 --> 00:13:12,070
Therefore A complement is minus alpha, 3 alpha
which is denoted by a bar over the set A.
106
00:13:12,070 --> 00:13:15,280
Now similarly, let us come to the next aspect.
107
00:13:15,280 --> 00:13:19,780
Again from the elementary set theory, this
is also something that you should be familiar
108
00:13:19,780 --> 00:13:27,830
with, A union B equals sample points, that
is the union of 2 sets, union of 2 events,
109
00:13:27,830 --> 00:13:37,570
A and B which is denoted by A union, so this
is the symbol for union, equals Sample points
110
00:13:37,570 --> 00:14:05,900
which are either in A or in B. And
A intersection B equals
111
00:14:05,900 --> 00:14:18,360
sample points in both A and B. So we have
2 operations, A union B and A intersection
112
00:14:18,360 --> 00:14:19,850
B of 2 events, A, B.
113
00:14:19,850 --> 00:14:24,560
This is very simple and from elementary set,
these are similar to elementary set theory.
114
00:14:24,560 --> 00:14:32,550
A union B is the set of all Sample points
which are either in A or in B. And the event
115
00:14:32,550 --> 00:14:38,570
A intersection B is the set of sample points
which are both in A and B.
116
00:14:38,570 --> 00:14:41,550
Again, let us go back, let us take a simple
example.
117
00:14:41,550 --> 00:14:49,720
We have our example in which A equals minus
3 alpha, alpha.
118
00:14:49,720 --> 00:14:57,920
Let’s say the event B equals minus alpha,
alpha.
119
00:14:57,920 --> 00:15:08,760
Then A union B equals elements which are either
in A or B that is minus 3 alpha, minus alpha,
120
00:15:08,760 --> 00:15:10,300
alpha.
121
00:15:10,300 --> 00:15:19,880
And A intersection B equals sample points
that are both in A and B, that is equal to
122
00:15:19,880 --> 00:15:21,400
alpha.
123
00:15:21,400 --> 00:15:26,910
So we have defined the union of 2 events and
the intersection of 2 events.
124
00:15:26,910 --> 00:15:32,010
And let us also define now a special event
that is also known as the null event , phi.
125
00:15:32,010 --> 00:15:39,760
Phi equals the null event.
126
00:15:39,760 --> 00:15:44,440
Also denoted by the empty set.
127
00:15:44,440 --> 00:15:46,710
This is the null event.
128
00:15:46,710 --> 00:15:50,840
This contains no Sample points.
129
00:15:50,840 --> 00:16:07,110
It does not contain any
Sample point.
130
00:16:07,110 --> 00:16:14,830
Further, we can say 2 events are mutually
exclusive if A intersection B is the null
131
00:16:14,830 --> 00:16:15,830
event.
132
00:16:15,830 --> 00:16:46,500
So 2 events, A, B are
mutually exclusive if A intersection B equals
133
00:16:46,500 --> 00:16:48,800
phi, the empty set.
134
00:16:48,800 --> 00:16:50,810
That is to say, they have no common.
135
00:16:50,810 --> 00:17:10,140
A, B have no common sample points.
136
00:17:10,140 --> 00:17:19,170
That is we can see, the 2 sets, that is the
2 events, A and B are mutually exclusive if
137
00:17:19,170 --> 00:17:23,740
A intersection B is the null event.
138
00:17:23,740 --> 00:17:28,490
That is A and B have no common sample points.
139
00:17:28,490 --> 00:17:31,690
For example, let’s take a simple example
from our MPSK.
140
00:17:31,690 --> 00:17:38,620
If A contains the symbols minus 3 alpha, alpha.
141
00:17:38,620 --> 00:17:42,380
B contains the symbol minus alpha.
142
00:17:42,380 --> 00:18:01,050
Then you can clearly see that we have A intersection
B is equal to phi where this phi is the empty
143
00:18:01,050 --> 00:18:02,050
set.
144
00:18:02,050 --> 00:18:26,640
And we say that A and B are
145
00:18:26,640 --> 00:18:28,510
mutually exclusive.
146
00:18:28,510 --> 00:18:35,140
Further, it is easy to see some properties
which can be easily clarified with a diagram.
147
00:18:35,140 --> 00:18:45,320
Let us see some properties which can be clarified
with a diagram, a picture representation.
148
00:18:45,320 --> 00:18:49,100
Let us
say, this is my sample space, S. We can clearly
149
00:18:49,100 --> 00:19:00,350
see that S complement is
equal to empty set.
150
00:19:00,350 --> 00:19:06,820
The does the complement of the Sample space,
that is the complement, remember, the complement
151
00:19:06,820 --> 00:19:13,650
of an event is all the Sample points that
are not there in the event but are there in
152
00:19:13,650 --> 00:19:15,000
the Sample space.
153
00:19:15,000 --> 00:19:19,310
Naturally, if you look at the Sample space,
the complement of the Sample space, that is
154
00:19:19,310 --> 00:19:25,630
all the events which are not in the Sample
space, which is the empty set phi.
155
00:19:25,630 --> 00:19:27,520
And similarly phi complement.
156
00:19:27,520 --> 00:19:34,420
If you look at the complement of phi, phi
complement is S. Because we’re looking at
157
00:19:34,420 --> 00:19:38,680
phi complement is the complement of empty
set.
158
00:19:38,680 --> 00:19:40,801
All the Sample points not in the empty set.
159
00:19:40,801 --> 00:19:42,990
Empty set does not contain any Sample points.
160
00:19:42,990 --> 00:19:47,240
Therefore the complement of the empty set
is basically all the Sample points in the
161
00:19:47,240 --> 00:19:50,820
Sample space which is the Sample space itself.
162
00:19:50,820 --> 00:20:13,780
Complement of empty set is basically
all Sample points not
163
00:20:13,780 --> 00:20:15,050
in phi.
164
00:20:15,050 --> 00:20:17,180
So phi complement is equal to S.
165
00:20:17,180 --> 00:20:34,260
Further, if I have any set A in this
and this is of course A complement.
166
00:20:34,260 --> 00:20:45,640
Then I have A union A complement, look at
this if I have A union A complement.
167
00:20:45,640 --> 00:20:51,030
A complement contains all the elements in
S that are not in A. Therefore naturally,
168
00:20:51,030 --> 00:21:00,990
A union A complement is the Sample space S.
Further A and A complement do not have any
169
00:21:00,990 --> 00:21:02,410
elements in common.
170
00:21:02,410 --> 00:21:11,230
Therefore, A intersection A complement is
empty set, phi.
171
00:21:11,230 --> 00:21:12,230
Correct?
172
00:21:12,230 --> 00:21:19,300
A union A complement is equal to the Sample
space, S. At the same time, A intersection
173
00:21:19,300 --> 00:21:24,010
A complement, that is basically if you look
at the intersection of A and A complement,
174
00:21:24,010 --> 00:21:30,410
that is the empty set, phi because A and A
complement do not have any sample points in
175
00:21:30,410 --> 00:21:31,410
common.
176
00:21:31,410 --> 00:21:33,970
Further, the last property which is also obvious.
177
00:21:33,970 --> 00:21:40,470
That is if I take the complement of A complement,
that this if I look at this picture, all the
178
00:21:40,470 --> 00:21:45,210
elements which are not there in A complement
but rather in the Sample space is A itself.
179
00:21:45,210 --> 00:22:03,350
So, A complement complement, so this is all
elements not in A complement but in S and
180
00:22:03,350 --> 00:22:05,180
that is A itself.
181
00:22:05,180 --> 00:22:07,580
So A complement complement is A.
182
00:22:07,580 --> 00:22:08,980
So we have some key properties.
183
00:22:08,980 --> 00:22:13,560
That is the complement of Sample space is
the empty set.
184
00:22:13,560 --> 00:22:17,380
The complement of the empty set is the Sample
space.
185
00:22:17,380 --> 00:22:23,120
A union A complement is the Sample space.
186
00:22:23,120 --> 00:22:26,460
A intersection A complement is the empty set.
187
00:22:26,460 --> 00:22:29,060
This is also known as the null event.
188
00:22:29,060 --> 00:22:33,300
Empty set in the context of an experiment
and outcomes.
189
00:22:33,300 --> 00:22:37,080
This is also known as the null event.
190
00:22:37,080 --> 00:22:41,380
So, A intersection A complement is the null
event.
191
00:22:41,380 --> 00:22:50,090
The complement event of the element event
of Ais A. These are some properties which
192
00:22:50,090 --> 00:22:56,550
I will not go into the details of and which
are fairly obvious from both, from elementary
193
00:22:56,550 --> 00:23:01,880
set theory and also from this simple diagrammatic
representation that is given here.
194
00:23:01,880 --> 00:23:09,370
So we will end this basic module here about
experiment, outcomes, events which are basically
195
00:23:09,370 --> 00:23:15,150
subset of the Sample space and the various
operations on events, that is the union, intersection,
196
00:23:15,150 --> 00:23:20,520
the complement and some properties of these
events.
197
00:23:20,520 --> 00:23:26,250
So we will end this module here.
198
00:23:26,250 --> 00:23:45,570
Thank you very much.