1
00:00:15,929 --> 00:00:35,160
Today, we discuss multivariate normal distribution
multivariate normal distribution, last class
2
00:00:35,160 --> 00:00:42,550
we have see the invariate normal distribution.
You see the formula if x is a random variable
3
00:00:42,550 --> 00:00:52,550
then 2 pi sigma square e to the power minus
half x minus mu by sigma square minus infinite
4
00:00:52,550 --> 00:01:01,780
less than x less than plus infinite, so the
p d f. Multivariate normal p d f is characterized
5
00:01:01,780 --> 00:01:19,170
by mu and sigma mu and sigma that is a population
parameter. We want the counter part of p d
6
00:01:19,170 --> 00:01:29,840
f multivariate domain when x that invariate
x is converted to X which is your X 1, X 2,
7
00:01:29,840 --> 00:01:40,600
X p and univariate mu is no longer univariate.
It will be a very mean vector mu 1, mu 2,
8
00:01:40,600 --> 00:01:45,490
mu p.
Similarly, invariate sigma square will no
9
00:01:45,490 --> 00:02:08,060
longer be univariate, it will be a multivariate
covariance matrix p cross p. So, when we want
10
00:02:08,060 --> 00:02:20,629
something by multivariate normal distribution,
we want something which is f x in terms of
11
00:02:20,629 --> 00:02:32,439
N variable number p and mu vector and covariance
matrix, how do you, how do you go about it
12
00:02:32,439 --> 00:02:38,309
and how to do it that is the discussion, today.
13
00:02:38,309 --> 00:02:45,180
Today, we will discuss bivariate normal density
function multivariate normal density function
14
00:02:45,180 --> 00:02:51,540
and properties of multivariate normal density
function. If time permits, we will go for
15
00:02:51,540 --> 00:02:59,659
statistical distance and constant density
contours.
16
00:02:59,659 --> 00:03:15,650
So, the univariate PDF is this you want to
visualize its multivariate counterpart, so
17
00:03:15,650 --> 00:03:24,059
let us consider a bivariate case. You see
this slide, in this slide you see that there
18
00:03:24,059 --> 00:03:35,609
are two variable X 1 and X 2 and probability
density that is joint density that X 1 and
19
00:03:35,609 --> 00:03:47,290
X 2 X 1 and X 2. So, this is what is given
in figure, so you see that you are getting
20
00:03:47,290 --> 00:03:52,739
a bell shape, but in three dimensional you
are getting because there first two dimension
21
00:03:52,739 --> 00:03:59,669
for the two variable values and third dimension are the density values.
22
00:03:59,669 --> 00:04:07,389
If you take one more dimension, it is difficult
you cannot visualize, suppose there are three
23
00:04:07,389 --> 00:04:17,260
variables with density, we cannot visualize
pictorially. Now, as I told you our objective
24
00:04:17,260 --> 00:04:27,400
of the first part of today’s lecture is
we want to develop, this is our objective.
25
00:04:27,400 --> 00:04:37,000
So, in order to do so, we will follow a systematic,
but simple path.
26
00:04:37,000 --> 00:04:50,569
Suppose, you think that you have two variables
X 1 and X 2 which we are saying a bivariate
27
00:04:50,569 --> 00:05:04,990
case, so our X is X 1 and X 2 that is why
my mu is that mini vector mu 1 and mu 2. Your
28
00:05:04,990 --> 00:05:16,569
covariance matrix will be 2 by 2 sigma 1 1,
sigma 1 2, sigma 1 2, sigma 2 2, I hope that
29
00:05:16,569 --> 00:05:27,590
there is no problem with you in this nomenclature.
So, we assume something here, you see the
30
00:05:27,590 --> 00:05:35,819
slide here, in the slide you see the top figure
that here you just scatter plot.
31
00:05:35,819 --> 00:05:42,900
Now, what you can say about the two variables
X 1 and X 2 seeing the scatter plot are they
32
00:05:42,900 --> 00:05:51,099
co related or there is no correlation, is
it something like a circle you are getting
33
00:05:51,099 --> 00:05:58,750
or ellipse. There is no pattern you see that
it is a ellipse type of thing, but there is
34
00:05:58,750 --> 00:06:06,759
no correlation. So, we want to simplify our
derivation without correlation, so let me
35
00:06:06,759 --> 00:06:18,280
know what it means to say our sigma 1 2 is
0 or rho 1 2 is 0. If there is no correlation,
36
00:06:18,280 --> 00:06:25,889
then what will be the joint density?
Suppose, X 1 and X 2 multiplication of the
37
00:06:25,889 --> 00:06:38,210
marginal density of the two, so X 1 cross
X 2, now all of us know that if X equal to
38
00:06:38,210 --> 00:06:48,689
1 by root over 2 pi sigma 1 square as it is
X 1 e to the power minus half X 1 minus mu
39
00:06:48,689 --> 00:06:57,139
1 by sigma 1 square. Then, similarly for X
2 also, that is a second variable, you can
40
00:06:57,139 --> 00:07:08,469
write 2 pi sigma 2 square e to the power minus
half X 2 minus mu 2 by sigma 2 square. You
41
00:07:08,469 --> 00:07:16,499
can write this and definitely here x j less
than greater than minus infinity to less than
42
00:07:16,499 --> 00:07:21,669
infinity to j equal to 1, 2 variable you have
taken.
43
00:07:21,669 --> 00:07:27,120
So, you can multiply these what you are getting,
we are getting like this 1 by, that is one
44
00:07:27,120 --> 00:07:38,439
quantity equal to 2 pi, so 2 pi you are getting
2 by 2 pi into 2 pi square root 2 by 2. Then,
45
00:07:38,439 --> 00:07:45,499
another one what you are getting, sigma 1
square and sigma 2 square also, sigma 1 square
46
00:07:45,499 --> 00:07:59,529
sigma 2 square to the power half you are getting
here. Then, I am coming to the exponent part
47
00:07:59,529 --> 00:08:05,499
e to the power minus half and all of us know
that e to the power a into e 2 power b equal
48
00:08:05,499 --> 00:08:11,539
to e to the power a plus b.
So, we can write this one like this X 1 minus
49
00:08:11,539 --> 00:08:27,669
mu 1 by sigma 1 square plus X 2 minus mu 2
by sigma 2 squares, we can write this. So,
50
00:08:27,669 --> 00:08:36,810
essentially what is happening here that when
I go for that univariate normal or bivariate
51
00:08:36,810 --> 00:08:45,320
normal with this with no dependence structure.
You are having two component in the density
52
00:08:45,320 --> 00:09:01,610
function, one is the constant part another
one is the exponent part; exponent means e
53
00:09:01,610 --> 00:09:06,810
to the power of something.
When I am making the joint distribution, here
54
00:09:06,810 --> 00:09:17,760
also you are having also two part this and
this, the general structure for the multivariate
55
00:09:17,760 --> 00:09:23,709
normal distribution invariate that remain
same what is the difference, difference will
56
00:09:23,709 --> 00:09:29,100
come in the two components and the values
will be different. So, we found out that if
57
00:09:29,100 --> 00:09:37,029
X 1 and X 2 are independent, then our structure
is like this, now let us see that we want
58
00:09:37,029 --> 00:09:48,160
to derive this constant part as well as exponent
part from the population parameter.
59
00:09:48,160 --> 00:10:04,430
We have mu 1 into mu 2 mu equal to mu 1 mu
2 and we have sigma is sigma 1 1, sigma 1
60
00:10:04,430 --> 00:10:13,410
2, sigma 1 2, sigma 2 2, this one you can
write. Now, sigma 1 1 equal to sigma 1 square
61
00:10:13,410 --> 00:10:25,880
sigma 1 2 sigma 1 2 sigma 2 square, so if
I make something like this determinant of
62
00:10:25,880 --> 00:10:38,740
sigma. Here, sigma 1 square, sigma 1 2, sigma
1 2, sigma 2 square, its determinant and you
63
00:10:38,740 --> 00:10:44,440
know that determinate will be this cross this
minus this cross this. So, this one is sigma
64
00:10:44,440 --> 00:10:56,920
1 square, sigma 2 square minus sigma 1 2 square,
now you see that what we have assumed in the
65
00:10:56,920 --> 00:11:04,399
earlier demonstration.
We say sigma 1 2 equal to 0, just for the
66
00:11:04,399 --> 00:11:11,699
sake of simplicity we have taken that sigma
1 2 is 0, so if sigma 1 2 is 0, then determinant
67
00:11:11,699 --> 00:11:23,180
of sigma is nothing but sigma 1 square sigma
2 square. So, if I make square root of these,
68
00:11:23,180 --> 00:11:30,829
then this is the determinant of square root
of the determinant and if this is the case.
69
00:11:30,829 --> 00:11:41,079
Then, the constant part what is the in case
of our independent bivariate density function,
70
00:11:41,079 --> 00:11:50,310
we found out that constant part is 2 pi 2
by 2 sigma 1 square sigma 2 square to the
71
00:11:50,310 --> 00:12:02,930
power half. Now, these I can write like these
2 pi 2 by 2 determinant of covariance matrix
72
00:12:02,930 --> 00:12:06,889
to the power half.
73
00:12:06,889 --> 00:12:20,139
Now, suppose you have one more variable that
means you have taken three variables, now
74
00:12:20,139 --> 00:12:31,069
like X is X 1 2, X 2 and X 3. You have to
consider that your sigma is sigma 1 square,
75
00:12:31,069 --> 00:12:40,819
0, 0, 0, sigma 2 squares, 0, 0, 0, sigma 3
square, we are assuming that all the variables
76
00:12:40,819 --> 00:12:55,630
are independent, then what will happen again
f x 1, x 2, x 3 will be f x 1 into f x 2 into
77
00:12:55,630 --> 00:13:00,550
f x 3.
In the similar way you multiply, ultimately
78
00:13:00,550 --> 00:13:11,350
your constant term will be 1 by 2 pi, now
three variables are there by 2, then sigma
79
00:13:11,350 --> 00:13:19,079
1 square, sigma 2 square, sigma 3 square to
the power half. You see if you take determinant
80
00:13:19,079 --> 00:13:27,660
here, what are you getting here it will be
sigma 1 square, sigma 2 square, sigma 3 square.
81
00:13:27,660 --> 00:13:36,399
So, that means determinant to the power half
is sigma 1 square sigma 2 square sigma 3 square
82
00:13:36,399 --> 00:13:45,880
to the power of half.
So, if you now increase it to p variables,
83
00:13:45,880 --> 00:13:53,610
so ultimately your dimension will change and
sigma to the power half will take care of
84
00:13:53,610 --> 00:14:00,769
one part of the constant. So, if I go by p
variable, now my constant will become like
85
00:14:00,769 --> 00:14:09,019
this one by you see that, when there are two
variables it is 2 by 2 when three variables
86
00:14:09,019 --> 00:14:12,800
2 pi to the power of 3 by 2.
So, when there are p variable, it will be
87
00:14:12,800 --> 00:14:19,029
p by 2 and whether it is two variable or three
variable, three variable case. Ultimately,
88
00:14:19,029 --> 00:14:27,339
this quantity will be replaced by determinant
of covariance matrix to power half, so with
89
00:14:27,339 --> 00:14:36,009
one assumption here that we are considering
independent variable we proved this is the
90
00:14:36,009 --> 00:14:42,810
case, now what will happen to your constant
exponent term.
91
00:14:42,810 --> 00:14:54,079
So, in two variable cases we found out that
the exponent term X 1 minus mu 1 by sigma
92
00:14:54,079 --> 00:15:15,120
1 square plus X 2 minus by mu 2 by sigma 2
squared. So, we say this is the exponent term
93
00:15:15,120 --> 00:15:23,519
this is the exponent term this portion is
exponent term, so it is x you see that X 1
94
00:15:23,519 --> 00:15:30,000
minus mu on that is been subtracted divided
by a standard deviation that is square.
95
00:15:30,000 --> 00:15:44,139
Now, create one suppose x minus mu transpose,
no I will explain from the invariate case
96
00:15:44,139 --> 00:15:52,709
that will be better, so invariate case f x
1 by root over 2 pi sigma square e to the
97
00:15:52,709 --> 00:16:08,449
power minus half x minus mu by exponent 1.
So, your exponent is minus half, I am writing
98
00:16:08,449 --> 00:16:21,699
x minus mu sigma square to the power minus
1 x minus mu is minus half x minus mu by sigma
99
00:16:21,699 --> 00:16:30,720
square. You see x minus half is there minus
half x minus mu x minus mu square divided
100
00:16:30,720 --> 00:16:34,790
by sigma square, so sigma square sigma to
the power inverse.
101
00:16:34,790 --> 00:16:45,610
Now, if you go for the multivariate case what
will happen your x is replaced by X, mu is
102
00:16:45,610 --> 00:16:52,930
replaced by bold mu, sigma square will be
replaced by sigma. Now, in matrix multiplication
103
00:16:52,930 --> 00:17:04,260
what will be the square transpose that matrix
X transpose x that is the square term.
104
00:17:04,260 --> 00:17:08,620
So, we are basically making here square, so
we want this that is why what is meant to
105
00:17:08,620 --> 00:17:16,800
say in multivariate domain, the exponent can
be written like this x minus mu transpose
106
00:17:16,800 --> 00:17:29,680
sigma square is replaced by sigma to the power
minus 1 x minus mu. From univariate normal
107
00:17:29,680 --> 00:17:38,070
distribution, we have taken the exponent part
and we are saying that if we go in same manner
108
00:17:38,070 --> 00:17:47,910
to the multivariate part our resultant quantity
will be this for the exponent is it so?
109
00:17:47,910 --> 00:17:58,130
Now, can we do like the same thing for X equal
to X 1 and X 2 variable case mu equal to mu
110
00:17:58,130 --> 00:18:05,890
1 and mu 2 and sigma equal to you have taken
already sigma 1 square, 0, 0, sigma 2 square
111
00:18:05,890 --> 00:18:12,200
because we are independent case. We want to
prove first because we know under this condition,
112
00:18:12,200 --> 00:18:18,350
what will be the distribution multivariate
normal density function that is known.
113
00:18:18,350 --> 00:18:27,400
So, then you write down minus half, so x minus
mu transpose, so that means this is X 1 minus
114
00:18:27,400 --> 00:18:38,270
mu 1 X 2 minus mu 2 because x minus mu is
X 1 minus mu 1 X 2 minus mu 2, 2 cross 1 it
115
00:18:38,270 --> 00:18:47,440
will be 1 cross 2. Now, your what do you want
sigma 1 square, 0, 0, sigma 2 square, this
116
00:18:47,440 --> 00:18:58,680
inverse then, so that is 2 cross 2 then X
1 minus mu 1, X 2 minus mu 2, this is your
117
00:18:58,680 --> 00:19:02,650
2 cross 1.
So, what is the resultant quantity 1 cross
118
00:19:02,650 --> 00:19:09,760
2, 2 cross 2, 2 cross 2, it is a 1 cross 1,
this will give you 1 cross 2, this will give
119
00:19:09,760 --> 00:19:17,690
you 1 cross 1. We say density that exponential
to the power this constant value, you will
120
00:19:17,690 --> 00:19:24,010
be getting some values density will be calculated.
Now, what is the inverse, how to calculate
121
00:19:24,010 --> 00:19:36,540
the inverse suppose if A is a matrix like
this a 1 1, a 1 2, a 2 1, a 2 2, how do you
122
00:19:36,540 --> 00:19:49,010
compute the inverse 1 by adjoint by determinant.
So, A inverse is adjoint of A by determinant
123
00:19:49,010 --> 00:20:04,970
of A, now adjoint is the transpose of the
cofactors of A divided by determinant of A.
124
00:20:04,970 --> 00:20:14,940
So, this is the case our A is nothing but
this one sigma 1 square, 0, 0, sigma 2 square,
125
00:20:14,940 --> 00:20:23,810
which is what is our sigma. Now, determinant
already we have seen the determinant is sigma
126
00:20:23,810 --> 00:20:32,360
1 square and sigma 2 square multiplied by
these two. Now, what will be the cofactor
127
00:20:32,360 --> 00:20:43,120
of this cofactor is if you if you suppose
I want to know cofactor of sigma.
128
00:20:43,120 --> 00:20:50,550
Here, suppose in a case you see cofactor means
suppose you want to see the cofactor of these
129
00:20:50,550 --> 00:20:58,820
then you have to cross this corresponding
row and column what is left that is the cofactor,
130
00:20:58,820 --> 00:21:06,960
but the sign conversion will be there. So,
that means cofactor means for a i j, the cofactor
131
00:21:06,960 --> 00:21:17,330
will be minus 1 i plus j and the remaining
portion whatever the remaining portion remaining
132
00:21:17,330 --> 00:21:22,480
part of the matrix that will be the case.
As you have take 2 by 2 cross, so ultimately
133
00:21:22,480 --> 00:21:27,100
what will happen one row and one column crossed
means only one item will be left.
134
00:21:27,100 --> 00:21:35,000
So, it is our case, then cofactor of these
we can write first one is minus 1 to the power
135
00:21:35,000 --> 00:21:43,070
1 plus 1 that is 1 plus 1 then what is remaining
here, sigma 2 squares. Suppose, you cross
136
00:21:43,070 --> 00:21:48,710
this and this sigma 2 square will be there
and see it is 0 and it will be also 0 and
137
00:21:48,710 --> 00:22:01,900
sigma 1 square will be this. So, our cofactor
is sigma 2 square, 0, 0, sigma 1 square what
138
00:22:01,900 --> 00:22:09,060
will be transpose same because these two element
are 0.
139
00:22:09,060 --> 00:22:23,340
Now, transpose of cofactor of sigma, this
is again coming the same thing because rho
140
00:22:23,340 --> 00:22:34,130
and column interchanged symmetry 1, 0, 0,
sigma 1 square. Then, what is my inverse,
141
00:22:34,130 --> 00:22:41,610
inverse is that is the cofactors mean 1 by
determinant, you write down first sigma 1
142
00:22:41,610 --> 00:22:52,370
square, and sigma 2 square that is the determinant
sigma 2 square, 0, 0, sigma 1 square.
143
00:22:52,370 --> 00:23:05,600
Now, calculate this one my calculation is
minus half, our X 1 minus mu 1 X 2 minus mu
144
00:23:05,600 --> 00:23:16,660
2. Then, inverse is coming like this 1 by
sigma 1 square sigma 2 square sigma 2 square,
145
00:23:16,660 --> 00:23:25,180
0, 0, sigma 2 square then multiplied by X
1 minus mu 1 and X 2 minus mu 2. Suppose,
146
00:23:25,180 --> 00:23:34,820
if I do this portion first, what you will
get you will get minus half X 1 minus mu 1
147
00:23:34,820 --> 00:23:44,830
X 2 minus mu 2 1 by sigma 1 square sigma 2
square this is 2 cross 2 this is 2 cross 1
148
00:23:44,830 --> 00:23:50,210
you will be getting 2 cross 1 this into this
plus this into this.
149
00:23:50,210 --> 00:23:59,440
So, it is basically sigma 2 square X 1 minus
mu 1 then this into this plus this plus 0
150
00:23:59,440 --> 00:24:15,230
then 0 again, sigma 1 square X 2 minus mu
2. Now, let me bring this one this side later
151
00:24:15,230 --> 00:24:21,180
on, we will manipulate sigma 1 square, sigma
2 square, so if you multiply this 1 cross
152
00:24:21,180 --> 00:24:27,880
2 and 2 cross 1, you will be getting 1 cross
1. So, this into this plus this into this
153
00:24:27,880 --> 00:24:34,450
you see what is happening sigma 2 square X
1 minus mu 1 square because X 1 minus mu 1
154
00:24:34,450 --> 00:24:48,560
X 1 minus mu 1 plus sigma 1 square X 2 minus
mu 2 square. So, if you divide this 2 by sigma
155
00:24:48,560 --> 00:24:52,410
1 square sigma 2 square what you will be getting?
156
00:24:52,410 --> 00:24:59,870
You will be getting like this minus half,
then it will be sigma 1 minus mu 1 by sigma
157
00:24:59,870 --> 00:25:17,830
1 square plus X 2 minus mu 2 by X 1 minus
mu 1 by sigma 1 square this one. You have
158
00:25:17,830 --> 00:25:26,420
seen that we have found out earlier also this
is f x 1 cross f x 2, this we will find out
159
00:25:26,420 --> 00:25:53,520
like
this one you found out you. Just check I showing
that earlier when we have multiplied the two
160
00:25:53,520 --> 00:26:05,030
what we got here 1 by 2 pi 2 2 pi to the power
2 by 2 sigma 1 square sigma 2 square half
161
00:26:05,030 --> 00:26:10,890
then minus 1 by 2 X 1 minus mu 1 by sigma
square this one.
162
00:26:10,890 --> 00:26:20,730
Here, what are you getting here same thing
you are getting, so what I mean to mean to
163
00:26:20,730 --> 00:26:26,350
say all though this is not a derivation this
is the other way proof that what we are saying
164
00:26:26,350 --> 00:26:40,460
that means I can write for a bivariate case.
I can write my bivariate normal distribution
165
00:26:40,460 --> 00:27:05,540
like this is the case you can write like this.
If it is true for multivariate case also then
166
00:27:05,540 --> 00:27:12,320
what will happen, ultimately multivariate
case X 1, X 2 and X p, then it will be 2 pi
167
00:27:12,320 --> 00:27:30,550
to the power p by 2 determinant of this then
e to the power minus half x minus mu transpose.
168
00:27:30,550 --> 00:27:42,090
This is the case and you have to write minus
infinite x j infinite j equal to 1 2 p, this
169
00:27:42,090 --> 00:27:51,500
is our multivariate normal distribution we
say multivariate normal density function defined.
170
00:27:51,500 --> 00:28:01,280
Now, what will be the bivariate density normal
density function when your matrix is like
171
00:28:01,280 --> 00:28:01,930
this.
172
00:28:01,930 --> 00:28:13,240
This covariance matrix is like this sigma
1 square, sigma 1 2, sigma 1 2, sigma 2 square
173
00:28:13,240 --> 00:28:33,590
that means there is covariance this the case
what will happen can you not find out this
174
00:28:33,590 --> 00:28:43,190
determinant of this is our sigma 1 square,
sigma 2 square, minus sigma 1 2 square. So,
175
00:28:43,190 --> 00:28:49,930
this can be written like this sigma 1 square,
sigma 2 square minus rho 1 2 sigma 1 sigma
176
00:28:49,930 --> 00:29:00,220
2 square covariance is correlation times the
standard deviations. So, we can write this
177
00:29:00,220 --> 00:29:13,320
one sigma 1 square sigma 2 square 1 minus
rho 1 2 square and what will be your inverse
178
00:29:13,380 --> 00:29:22,640
here now inverse will be 1 by determinant.
So, 1 by determinant, let me keep this one
179
00:29:22,640 --> 00:29:33,780
only then sigma 1 square sigma 2 square minus
sigma 1 2 square into we know that transpose
180
00:29:33,780 --> 00:29:42,140
of the cofactor. So, I will take this, so
it will be sigma 2 square then what will be
181
00:29:42,140 --> 00:29:55,870
this plus this is minus sigma 1 2 minus sigma
1 square. Then, what is my exponent part half
182
00:29:55,870 --> 00:30:08,790
x minus mu transpose X minus mu transpose
sigma inverse x minus mu. This is equal to
183
00:30:08,790 --> 00:30:24,660
minus half X 1 minus mu 1 X 2 minus mu 2 X
1 minus mu 1 mu 2 the 1 by sigma 1 square
184
00:30:24,660 --> 00:30:33,340
sigma 2 square minus sigma 1 2 square into
sigma 2 square minus sigma 1 2 minus sigma
185
00:30:33,340 --> 00:30:48,710
1 2 sigma 1 square times X 1 minus mu 1 X
2 minus mu 2 correct.
186
00:30:48,710 --> 00:31:03,090
So, if you further manipulate, what will happen
this equal to minus half X 1 minus mu 1 X
187
00:31:03,090 --> 00:31:10,950
2 minus mu 2, then I want to multiply the
last two parts. So, I am writing like this
188
00:31:10,950 --> 00:31:20,530
sigma 1 square sigma 2 square minus sigma
1 2 square into this one, you see this is
189
00:31:20,530 --> 00:31:34,510
2 cross 2 and this one is 2 cross 1. So, this multiplied by this plus this multiplied by
190
00:31:34,510 --> 00:31:41,310
this plus this multiplied by this, so if we
write like this what will happen here sigma
191
00:31:41,310 --> 00:31:53,370
2 square into X 1 minus mu 1 minus sigma 1 2 into X 2 minus mu 2.
192
00:31:53,370 --> 00:32:04,450
So, that is coming from this part row column
second one will be minus sigma 1 2 minus sigma
193
00:32:04,450 --> 00:32:20,230
1 2, we are multiplying this with this. So,
X 1 minus mu 1 an then plus sigma 1 square
194
00:32:20,230 --> 00:32:30,710
when you are saying sigma 1 X 2 minus mu 2,
so that is your matrix your this is the first
195
00:32:30,710 --> 00:32:41,110
row, this is the second row. So, it is 2 cross
1, so then minus half into sigma 1 square
196
00:32:41,110 --> 00:32:46,570
sigma 2 square minus sigma 1 2 square you
keep here. Now, you are multiplying this into
197
00:32:46,570 --> 00:32:56,220
this so multiply this is 1 cross 2, 2 cross
1 you will get 1, so this into this plus this
198
00:32:56,220 --> 00:32:59,790
into this.
So, what you are getting then you are basically
199
00:32:59,790 --> 00:33:08,580
getting sigma 2 square X 1 minus mu 1 and
X 1 minus mu 1 that is square minus what you
200
00:33:08,580 --> 00:33:17,860
are getting this into this so sigma 1 2 into
X 1 minus mu 1 and X 2 minus mu 2. So, this
201
00:33:17,860 --> 00:33:27,470
into this over second column verse here second
row, so it will be minus sigma 1 2 X 1 minus
202
00:33:27,470 --> 00:33:39,700
mu 1 X 2 minus mu 2 plus sigma 1 square X
2 minus mu 2 square that is the total. So,
203
00:33:39,700 --> 00:33:47,790
if I further manipulate this what I can write
sigma 1 square sigma 2 square minus sigma
204
00:33:47,790 --> 00:34:00,240
1 2 square then this is sigma 2 square X 1
minus mu 1 square minus 2 sigma 1 2 X 1 minus
205
00:34:00,240 --> 00:34:12,839
mu 1and X 2 minus mu 2 plus sigma 1 square
X 2 minus mu 2 square.
206
00:34:12,839 --> 00:34:19,550
If you divide this within bracket quantity
by sigma 1 square and sigma 2 square what
207
00:34:19,550 --> 00:34:28,059
will happen minus 1 by 2 1 by sigma 1 square
sigma 2 square minus sigma 1 2 square. So,
208
00:34:28,059 --> 00:34:35,950
I am dividing the entire thing by sigma 1
square and sigma 2 square I am taking common
209
00:34:35,950 --> 00:34:43,059
then what will happen this one X 1 minus mu
1 by sigma 1 see sigma 2 is already there
210
00:34:43,059 --> 00:34:51,609
sigma 1 square we have already taken sigma
1 I am keeping. Here, this minus 2 sigma 1
211
00:34:51,609 --> 00:35:01,589
2 then divided by you write sigma 1 and sigma
2 here can we not write like this, like this,
212
00:35:01,589 --> 00:35:08,900
this equal to this X 2 by this see sigma 1
square sigma 2 square.
213
00:35:08,900 --> 00:35:24,990
You have taken here plus you can write down
X 2 minus mu 2 by sigma 2 square what is what
214
00:35:24,990 --> 00:35:32,529
is this quantity sigma 1 2 by sigma 1 sigma
2 that is rho, so I can write like this minus
215
00:35:32,529 --> 00:35:40,359
half sigma 1 square sigma 2 square. You have
already seen sigma 1 2 square equal to sigma
216
00:35:40,359 --> 00:35:50,539
1 square sigma 2 square into rho 1 2 square.
So, you take common here 1 minus rho 1 2 square
217
00:35:50,539 --> 00:36:01,880
then this quantity is X 1 minus mu 1 by sigma
1 square minus 2 rho 1 2 X 1 minus mu 1 by
218
00:36:01,880 --> 00:36:14,039
sigma 1 X 2 minus mu 2 by sigma 2 plus X 2
minus mu 2 by sigma 2 square so this quantity
219
00:36:14,039 --> 00:36:26,750
this will be cancelled out.
So, if I see this verses the independent case
220
00:36:26,750 --> 00:36:41,369
you will very easily find out, now if I put
rho 1 2 equal to 0, this one will become 0,
221
00:36:41,369 --> 00:36:48,579
so then this is X 1 minus mu 1 by sigma 1
square plus X 2 minus mu 2 by sigma 2 square
222
00:36:48,579 --> 00:36:52,240
and what you have here also rho 1 2 will be 0.
223
00:36:52,240 --> 00:36:57,859
So, I remove 1 by minus half that mean the
resultant quantity will be, if I put rho 1
224
00:36:57,859 --> 00:37:07,410
2 equal to 0 my quantity is coming this 1
minus half X 1 minus mu 1 by sigma 1 square
225
00:37:07,410 --> 00:37:24,759
plus X 2 minus mu 2 by sigma 2 square. So,
this is the exponent part clear, so that means
226
00:37:24,759 --> 00:37:33,019
what I mean to say that in the reverse way
also we proved that yes this quantity is following
227
00:37:33,019 --> 00:37:40,009
the distribution equal distribution what we
have thought of. Now, question comes what
228
00:37:40,009 --> 00:38:01,579
is this is the shape of this ellipse correct,
now see this diagram this very important concept.
229
00:38:01,579 --> 00:38:10,759
Here, see this is my equation and we have
started with this we said this is the scattered
230
00:38:10,759 --> 00:38:20,369
plot of X 1 and X 2 and it resembles that
there is no dependency between the two variables
231
00:38:20,369 --> 00:38:25,730
that mean covariance is 0. We assume sigma
1 2 is equal to 0, so that mean this one is
232
00:38:25,730 --> 00:38:34,630
nothing but this ellipse what is coming here
this ellipse. So, that means if I just write
233
00:38:34,630 --> 00:38:42,490
down this one what you are getting you are
getting you see that what I will do now I
234
00:38:42,490 --> 00:38:55,079
will draw a line like this, but it will be
a curve so it is basically coming like this.
235
00:38:55,079 --> 00:39:19,529
So, when you plot this X 1 and X 2 that exponent
part you are getting an ellipse when any time
236
00:39:19,529 --> 00:39:24,670
you can get an ellipse because this one is
also a equation of ellipse please keep in
237
00:39:24,670 --> 00:39:33,500
mind this is the in two dimensional general
equation of ellipse. So, if I want to plot
238
00:39:33,500 --> 00:39:45,279
this what will happen to my figure you are
now depending on rho 1 2 yes origin is at
239
00:39:45,279 --> 00:39:54,910
mu 1 mu 2. You usually figure that is original
side mu 1 mu 2 data is given in such a manner
240
00:39:54,910 --> 00:40:05,569
that 0 is the 0, 0 is the origin mu 1 mu 2
origin is mu 1 and mu 2 this is your mu 1
241
00:40:05,569 --> 00:40:19,369
and mu 2 what will happen if you take the
general equation means this 1.
242
00:40:19,369 --> 00:40:36,849
So, depending on the rho value that rho 1
2 value is it positive is it negative is it
243
00:40:36,849 --> 00:40:46,619
0. If it is 0 this is the diagram this side
or you it may because this side see in here
244
00:40:46,619 --> 00:40:55,829
the major axis of the ellipse lies along X
2 axis the reason is the variability along
245
00:40:55,829 --> 00:41:08,519
X 2 is more they are independent. That is
why the major and the minor axis of the ellipse
246
00:41:08,519 --> 00:41:17,160
go along the original X 1 and X 2 axis and
along X 2 axis the major axis lies because
247
00:41:17,160 --> 00:41:23,029
the variability along X 2 axis is more and
variability along y 2 is less, sorry variable
248
00:41:23,029 --> 00:41:34,029
X 2 is more variable X 1 is less if variability
along X 1 is more. Then, they are independent,
249
00:41:34,029 --> 00:41:48,210
then your ellipse will become like this keep
in mind they are independent when rho 1 2
250
00:41:48,210 --> 00:41:57,769
is greater than 0, it will be so X 1 increases
X 2 increases like this so it will be like.
251
00:41:57,769 --> 00:42:09,339
This is inclined because the major and minor
axis of the ellipse is not parallel to the
252
00:42:09,339 --> 00:42:16,650
original X 1 and X 2 axis. So, as this one
is increasing this is also increasing other
253
00:42:16,650 --> 00:42:38,569
way when it is less than it will be just this
it will go to this level. In one of the slide
254
00:42:38,569 --> 00:42:48,380
I think I have shown you this picture that
my X 1 and X 2 is like this my data is like
255
00:42:48,380 --> 00:43:13,249
this it is a circle this is also a bivariate case. So, this is also independent case the question
256
00:43:13,249 --> 00:43:28,630
is this sigma 1 and sigma 2 in this case independent,
but sigma 1 square equal to sigma 2 squares,
257
00:43:28,630 --> 00:43:38,180
how do you know this axis. I know the ellipse
what is this value suppose this is the first
258
00:43:38,180 --> 00:43:47,599
of the entire direction second one is the
value how you know all this things.
259
00:43:47,599 --> 00:44:07,200
Let us see some of the slides here.
260
00:44:07,200 --> 00:44:14,630
This is multivariate, so let us see this one
first I will I will come back to this how
261
00:44:14,630 --> 00:44:21,329
to determine the axis and length all those
things. So, what I request to all of you in
262
00:44:21,329 --> 00:44:23,170
order to understand the axis.
263
00:44:23,170 --> 00:44:32,289
You have to know little bit of matrix what
is this again value Eigen value Eigen vector,
264
00:44:32,289 --> 00:44:46,740
so Eigen value and Eigen vector. I will show
you next class Eigen value Eigen vector then
265
00:44:46,740 --> 00:45:02,779
axis the all those things and now see one
example a process is characterized by two
266
00:45:02,779 --> 00:45:10,289
variables that is X process is designed to
produce laminar aluminum sheet of length X
267
00:45:10,289 --> 00:45:16,210
1 and breadth X 2.
With the following population parameters this
268
00:45:16,210 --> 00:45:23,009
and this are the population parameters obtain
its bivariate normal distribution this is
269
00:45:23,009 --> 00:45:29,609
the answer, I am sure you will be able to
find out this one from the beginning. If you
270
00:45:29,609 --> 00:45:35,430
start the way we have described if you start
in the same manner you will ultimately ending.
271
00:45:35,430 --> 00:45:50,440
With this answer, we come to properties that
multivariate normal distribution has started
272
00:45:50,440 --> 00:45:57,680
very, very useful properties, multivariate
normal distribution that we will denote.
273
00:45:57,680 --> 00:46:06,230
Next, hence proved that is MND multivariate
normal distribution which is we say N p mu
274
00:46:06,230 --> 00:46:17,249
and sigma it has many useful properties some
of the useful properties I am describing.
275
00:46:17,249 --> 00:46:29,050
Now, you see the first property if X is multivariate
normal then all the variables individually
276
00:46:29,059 --> 00:46:39,440
are invariate normal obvious that when x is
there are X 1 to X p. They simultaneously
277
00:46:39,440 --> 00:46:46,069
multivariate normal then X 1 is also invariate
normal X 2 is invariate normal x p is also
278
00:46:46,069 --> 00:46:55,920
invariate normal that means what I mean to
say that this one where mu is mu 1 mu 2 mu
279
00:46:55,920 --> 00:47:00,910
p and sigma.
I am writing sigma 1 square sigma 2 square
280
00:47:00,910 --> 00:47:10,569
sigma p square and these components are also
there any one. If say x j this will be your
281
00:47:10,569 --> 00:47:21,039
invariate normal with mu j and sigma j square
j equal to 1 2 p, so that mean that sigma
282
00:47:21,039 --> 00:47:25,769
j will be coming from here that sigma j square.
283
00:47:25,769 --> 00:47:37,739
This is your first property what is the second
property, if x is multivariate normal, then
284
00:47:37,739 --> 00:47:48,470
any subset you take that will be multivariate
normal by this what do you mean.
285
00:47:48,470 --> 00:48:05,989
Suppose, my X is this X 1 X 2 X q X p, I will
create two subsets that is X 1 X 2 that is
286
00:48:05,989 --> 00:48:18,999
big X 1 X 2, so then this will be this X 1
is q cross 1 and X 2 is p minus q cross 1
287
00:48:18,999 --> 00:48:28,970
vector then what I want to say we want to
say that X 1 is q variable vector. So, it
288
00:48:28,970 --> 00:48:39,849
will be multivariate with q dimensions and
what will be the mu that is mu and sigma what
289
00:48:39,849 --> 00:48:47,140
you will do I am writing mu q and sigma q.
If I write what will be the mu that first
290
00:48:47,140 --> 00:48:55,819
mu because that first mu variable you have
taken what will be the sigma q sigma. Now,
291
00:48:55,819 --> 00:49:04,609
sigma 1 1, sigma 1 2 like sigma 1 q, sigma
1 p, similarly this will be sigma 1 q then
292
00:49:04,609 --> 00:49:12,200
somewhere sigma 1 q then sigma q p then sigma
1 p that sigma p p.
293
00:49:12,200 --> 00:49:20,809
So, you have created a subset with q variables,
so that means what is happening here, now
294
00:49:20,809 --> 00:49:46,239
this is your sigma q and mu case is mu 1 mu
2 mu q mu p, so this is your mu q. So, that
295
00:49:46,239 --> 00:49:52,180
means if you take a subset and you know the
parameters for those the subset of parameters
296
00:49:52,180 --> 00:49:57,739
you consider and find out it distribution
that will be multivariate normal distribution
297
00:49:57,739 --> 00:50:05,430
the third distribution is very, very useful.
The third property is very, very useful, you
298
00:50:05,430 --> 00:50:18,430
see what is written if X is if x is multivariate
normal linear combination of x j is invariate
299
00:50:18,430 --> 00:50:29,839
normal this property can be exploited like
anything in your research what is what does
300
00:50:29,839 --> 00:50:31,079
it mean?
301
00:50:31,079 --> 00:50:43,019
It means that suppose I will first create
a vector like this a 1 all constant a p some
302
00:50:43,019 --> 00:51:00,119
I is my X is X 1 X 2 x p then this one is
1 cross p this is p cross 1 so what is the
303
00:51:00,119 --> 00:51:21,710
linear combination, linear combination. Obviously,
this one gives you a 1 X 1 plus a 2 X 2 like
304
00:51:21,710 --> 00:51:30,890
a p x p what it is say is that the this property
says that what will be the expected value
305
00:51:30,890 --> 00:51:42,239
of a T x it will basically a T. Expected value
of x will be a T and mu this is nothing but
306
00:51:42,239 --> 00:52:03,170
a 1 mu 1 a 2 mu 2 a p mu p and what will be
your variance of this a transpose x.
307
00:52:03,170 --> 00:52:15,420
This will be you are a transpose sigma a,
you can prove it also writing like this, so
308
00:52:15,420 --> 00:52:22,509
a transpose sigma. You see it is 1 cross p
p cross p p cross 1 resultant is 1 cross 1.
309
00:52:22,509 --> 00:52:32,589
So, then the linear combination will follow
univariate normal with a transpose mu a transpose
310
00:52:32,589 --> 00:52:49,690
sigma a that is our variance spot. Now, the
fourth property fourth property says that
311
00:52:49,690 --> 00:52:56,299
instead of one linear combination if you make
two linear combinations, what is happening
312
00:52:56,299 --> 00:52:59,039
here, you just see in one linear combination.
313
00:52:59,039 --> 00:53:10,019
We have taken a 1 a 2 we have taken a p this
is the instead of this I am creating another
314
00:53:10,019 --> 00:53:27,450
one like this a 1 1, a 1 2, a 1 p, a 2 1,
a 2 2, a 2 p. Suppose, a q 1, a q 2 dot dot
315
00:53:27,450 --> 00:53:37,269
dot a q p what is happening now? If I find
out, so this one is my 1 2 p, so this is p
316
00:53:37,269 --> 00:53:50,739
cross q this one is p cross 1. Now, if I make
like this A transpose x what will happen then
317
00:53:50,739 --> 00:53:58,440
this will be your q cross p and this will
be your p cross 1. So, you will be getting
318
00:53:58,440 --> 00:54:06,680
something called q cross 1 where as in one
linear combination a cross t is basically
319
00:54:06,680 --> 00:54:16,519
1 cross, so that means q cross 1. This means
you are ultimately creating this one q cross
320
00:54:16,519 --> 00:54:33,119
1 a 1 1 X 1 a 1 2 X 2 like this a 1 p x p
for this a 2 1 X 1 plus a 2 2 X 2 plus a 2
321
00:54:33,119 --> 00:54:46,119
p x p.
So, like this a q 1 X 1 a q 2 X 2 plus q p
322
00:54:46,119 --> 00:54:57,009
x p so if q 1 will be this if I take one combination
that is invariate normal take this one second,
323
00:54:57,009 --> 00:55:05,589
so all collectively what you are saying collectively
they will be multivariate normal. So, that
324
00:55:05,589 --> 00:55:17,381
means this quantity will be as q linear combination
you have made this into definitely what will
325
00:55:17,390 --> 00:55:30,839
happen a sorry A transpose mu. Then, A just
check this transpose part you have to check
326
00:55:30,839 --> 00:55:39,999
what is A here a is p cross q and this one
is q, so that means what do you want this
327
00:55:39,999 --> 00:55:49,019
will be cross q if I write like this. I think
in books may they have written in the other
328
00:55:49,019 --> 00:55:58,089
way round that part you check ultimate aim
is as it is q variable that a transpose x
329
00:55:58,089 --> 00:56:03,920
is q variable vector.
So, the variance component will be order of
330
00:56:03,920 --> 00:56:14,759
q cross q and mean component be order of q
cross 1 column vector definitely. So, this
331
00:56:14,759 --> 00:56:21,759
four properties are important and you will
you have you see that we you calculate x bar
332
00:56:21,759 --> 00:56:29,680
in invariate case. When you calculate x bar
that is what that is linear combination of
333
00:56:29,680 --> 00:56:39,910
multivariate observations n observations are
there 1 by n into x or so equal. Now, then
334
00:56:39,910 --> 00:56:47,170
that then what will be the distribution of
x bar, although it is invariate normal that
335
00:56:47,170 --> 00:56:54,749
is why the sigma square by n is coming there,
so all those things. Here, we will be seeing
336
00:56:54,749 --> 00:57:15,809
not that x bar only it will be a big x bar
that means mean vector for, so next class
337
00:57:15,809 --> 00:57:22,710
I will explain you that statistical distance.
Thank you very much.