1
00:00:14,219 --> 00:00:21,820
Welcome to the course on Coding Theory Today
in this lecture we are going to describe what
2
00:00:21,820 --> 00:00:33,510
we mean by generator matrix and parity check
matrix So we will continue our discussion
3
00:00:33,510 --> 00:00:39,960
with introduction to linear block codes We
will first describe what is a generator matrix
4
00:00:39,960 --> 00:00:44,160
and what is a parity check matrix and how
are they related So as we described in the
5
00:00:44,160 --> 00:00:53,760
last class an encoder for a linear block codes
what it does it takes a block of k-bits and
6
00:00:53,760 --> 00:01:02,970
maps it to the to n-bit Now the matrix we
can use a k cross n matrix to define this
7
00:01:02,970 --> 00:01:13,500
mapping from k information bits to n coded
bits and this matrix is basically our generator
8
00:01:13,500 --> 00:01:22,830
matrix for a so for a n-k linear block code
the mapping of k information bits to n-coded
9
00:01:22,830 --> 00:01:32,850
bits is defined by this generator matrix G
which is of rank k so if we denote information
10
00:01:32,850 --> 00:01:45,020
bits by u and we denote our coded bits by
v then we can write v as u times g where our
11
00:01:45,020 --> 00:01:56,460
u is 1 cross k vector and this is generator
matrix is k cross n matrix and our output
12
00:01:56,460 --> 00:02:00,479
coded bit is 1 cross n vector
13
00:02:00,479 --> 00:02:09,399
So as a name suggests basically generator
matrix is used to generate our codeword So
14
00:02:09,399 --> 00:02:16,819
we generate our codewords using this generator
matrix and this generator matrix gives the
15
00:02:16,819 --> 00:02:25,980
mapping between the information bits u to
coded bits v So how do we find codewords We
16
00:02:25,980 --> 00:02:34,170
find codewords by taking linear combinations
of rows of this generator matrix In case of
17
00:02:34,170 --> 00:02:43,810
binary codes so then these entries in the
generator matrix are either 0 or 1 depending
18
00:02:43,810 --> 00:02:53,560
upon which bits are used to generate a particular
coded sequence So we form a set of 2 k codewords
19
00:02:53,560 --> 00:03:03,290
by taking linear combinations of rows of these
generator matrix So we can as I said we can
20
00:03:03,290 --> 00:03:14,100
write our coded sequence as q times p which
is basically linear combination of rows of
21
00:03:14,100 --> 00:03:23,790
the generator matrix So these are basically
linearly independent k rows and the rank of
22
00:03:23,790 --> 00:03:30,860
this generator matrix is k Since we are without
loss of generality since we are talking about
23
00:03:30,860 --> 00:03:39,300
binary linear block codes so we will be doing
this addition modulus 2 So what are the properties
24
00:03:39,300 --> 00:03:46,940
of linear block code Sum of any two codewords
in a linear code is also a valid codeword
25
00:03:46,940 --> 00:04:01,459
So if v 1 and v 2 are valid codewords then
v 1 plus v 2 will also be a valid codeword
26
00:04:01,459 --> 00:04:11,159
Also an all zero codeword is a valid codeword
in any linear block codes So we can define
27
00:04:11,159 --> 00:04:20,629
a linear block code n k linear block code
as k dimensional space of vector space v n
28
00:04:20,629 --> 00:04:28,220
of all binary n-tuples So we can define a
linear binary block codes as a k dimensional
29
00:04:28,220 --> 00:04:35,840
subspace of vector space v n of all binary
n tuples
30
00:04:35,840 --> 00:04:44,500
Now let us take an example to illustrate what
is a generator matrix So in this example we
31
00:04:44,500 --> 00:04:55,520
have considered 3 information bits and 6 coded
bits And in this table I have given you the
32
00:04:55,520 --> 00:05:06,490
set of 8 information sequences and their corresponding
codewords So how do we find the generator
33
00:05:06,490 --> 00:05:14,060
matrix in this case So we will have to look
at each of this code bits and see how are
34
00:05:14,060 --> 00:05:24,150
we generating these code bits in terms of
message bits u 0 u 1 and u 2 So first thing
35
00:05:24,150 --> 00:05:32,770
we are going to do is look at each of these
code bits v 0 v 1 v 2 v 3 v 4 v 5 and write
36
00:05:32,770 --> 00:05:45,389
them in terms of u 0 u 1 u 2 Ok So let’s
look at each of these So v 0 is u 1 plus u
37
00:05:45,389 --> 00:05:55,490
2 we can see easily v 0 is this column and
we can see this is same as u 1 plus u 2 So
38
00:05:55,490 --> 00:06:03,229
u 1 plus u 2 in this case is 0 u 1 plus u
2 is 0 1 plus 0 is 1 1 plus 0 plus 1 is 1
39
00:06:03,229 --> 00:06:11,860
1 plus 1 is 0 modular 2 and 1 plus 1 is 0
modular 2 So this v 0 is basically given by
40
00:06:11,860 --> 00:06:23,370
u 1 plus u 2 Similarly we can see v 1 is given
by u 0 plus u 2 and v 2 is given by u 0 plus
41
00:06:23,370 --> 00:06:34,370
u 1 So let's just check let's say v 2 v 2
you can see is given by v 2 is given by u
42
00:06:34,370 --> 00:06:44,170
0 plus u 1 You can check v 2 is given by so
u 0 plus u 1 0 plus 0 is 0 1 plus 0 is 1 0
43
00:06:44,170 --> 00:06:55,120
plus 1 is 1 1 plus 1 is 0 0 plus 0 is 0 1
plus 0 is 1 0 plus 1 is 1 and 1 plus 1 is
44
00:06:55,120 --> 00:07:09,290
0 Similarly we notice that v 3 v 4 v 5 are
nothing but information bits u 0 u 1 and u
45
00:07:09,290 --> 00:07:17,860
2 respectively So let's go back v 3 is this
column and we can see this is same as u 0
46
00:07:17,860 --> 00:07:31,419
0 1 0 1 0 1 0 1 similarly v 4 is equal to
u 1 and v 5 is same as u 2
47
00:07:31,419 --> 00:07:44,419
So now we have written our coded bits in terms
of our information bits This set of 6 equations
48
00:07:44,419 --> 00:07:53,020
I can write it in a matrix form So I can write
my coded bits in terms of information bits
49
00:07:53,020 --> 00:08:02,150
and this matrix G which is our generator matrix
will tell us how are we generating each of
50
00:08:02,150 --> 00:08:10,819
these coded bits as a linear combination of
these information bits So if we compare each
51
00:08:10,819 --> 00:08:24,050
equation let’s look at v 0 So what is v
0 v 0 is u 0 g 0 0 plus u 1 g 1 0 plus u 2
52
00:08:24,050 --> 00:08:33,320
g 2 0 and what do we see here v 0 is u 1 plus
u 2 So that means g 0 0 is 0 because there
53
00:08:33,320 --> 00:08:42,919
is no u 0 term here g 1 0 is 1 because there
is a u 1 term here and g 2 0 is 1 because
54
00:08:42,919 --> 00:08:57,279
there is a u 2 term here So this will be 0
1 1 Similarly look at v 1 v 1 is u 0 g 0 1
55
00:08:57,279 --> 00:09:11,589
plus u 1 g 1 1 plus u 2 g 2 1 And if we compare
it with v 1 here we see v 1 is u 0 plus u
56
00:09:11,589 --> 00:09:22,319
2 that means this g 0 1 should be 1 g 1 1
should be 0 and this should be 1 Likewise
57
00:09:22,319 --> 00:09:30,179
we build up the other columns of the matrix
So if we do that what we get is something
58
00:09:30,179 --> 00:09:40,730
like this We can verify basically Let's take
second last column second last column so what
59
00:09:40,730 --> 00:09:53,350
is v 4 v 4 is u 0 times g 0 4 plus u 1 times
g 1 4 plus u 2 times g 2 4 and what is v 4
60
00:09:53,350 --> 00:09:59,949
v 4 is u 1 so then this should be 1 and this
should be 0 and this should be 0 and this
61
00:09:59,949 --> 00:10:10,610
is what we have 0 1 0 So now we can basically
find out the generator matrix so linear block
62
00:10:10,610 --> 00:10:19,329
code is completely described by its generator
matrix And we said we can use the generator
63
00:10:19,329 --> 00:10:23,019
matrix to generate our codewords
64
00:10:23,019 --> 00:10:33,040
For example if my information sequence is
1 0 1 what should be my corresponding coded
65
00:10:33,040 --> 00:10:40,420
bits for the information sequence 1 0 1 How
do I find that So as I know my output codeword
66
00:10:40,420 --> 00:10:48,279
is basically u times G so I will take linear
combinations of rows of my generator matrix
67
00:10:48,279 --> 00:10:55,209
What are the rows of my generator matrix These
are the 3 rows of my generator matrix So my
68
00:10:55,209 --> 00:11:03,420
coded bit corresponding to this information
sequence would be 1 times G 0 plus 0 times
69
00:11:03,420 --> 00:11:14,399
G 1 plus 1 times G 2 So that's what I have
written here 1 times G 0 0 times G 1 plus
70
00:11:14,399 --> 00:11:24,639
1 times G 2 So this is basically 0 So what
I have is then this plus this right So let's
71
00:11:24,639 --> 00:11:35,269
look at 0 plus 1 would be 1 1 plus 1 would
be 0 1 plus 0 is 1 1 plus 0 is 1 0 plus 0
72
00:11:35,269 --> 00:11:45,399
is 0 and 0 plus 1 is 1 So my codeword corresponding
to this information message bits this information
73
00:11:45,399 --> 00:11:52,720
bit is given by 1 0 1 1 0 1 Ok
74
00:11:52,720 --> 00:12:04,230
Now what do we mean by a linear code in systematic
form Now if we are able to among the coded
75
00:12:04,230 --> 00:12:12,369
bits if we are able to separate them out into
if the message bits appear directly in the
76
00:12:12,369 --> 00:12:18,509
coded bit sequence then we can separate out
the message bits from the parity bits For
77
00:12:18,509 --> 00:12:30,009
example go back to this example What do we
have here We have 3 of these coded bits exactly
78
00:12:30,009 --> 00:12:36,949
same as information bits and the other 3 bits
parity bits are linear combination of this
79
00:12:36,949 --> 00:12:44,959
message bits So from the output codeword we
we can clearly separate out the information
80
00:12:44,959 --> 00:12:57,069
sequence which is in this case given by v
3 v 4 and v 5 So in this case v 1 v 2 v 3
81
00:12:57,069 --> 00:13:05,949
are these n minus k parity bits and v 3 v
4 v 5 are my information bits So in this particular
82
00:13:05,949 --> 00:13:14,529
example we can see that we are able to separate
out information bits directly from the coded
83
00:13:14,529 --> 00:13:23,040
bits So in a systematic a block code in a
systematic form we are able to separate out
84
00:13:23,040 --> 00:13:33,350
the information bit part from the coded bits
So a generator matrix for a linear block code
85
00:13:33,350 --> 00:13:45,730
in systematic form will be of the form like
this or it would be basically i times i k
86
00:13:45,730 --> 00:13:58,179
times some some either of this form Now why
do we say that So only when we have our part
87
00:13:58,179 --> 00:14:03,519
of our generator matrix of the form of identity
then what is going to happen
88
00:14:03,519 --> 00:14:08,350
When we multiply our information sequence
with this sort of generator matrix you will
89
00:14:08,350 --> 00:14:16,639
see part of my coded bits will just depend
on one particular information bit sequence
90
00:14:16,639 --> 00:14:25,970
So if I have write down the corresponding
equations for coded sequence what you will
91
00:14:25,970 --> 00:14:36,509
see that some coded bits directly depend on
the message bits and then rest are which are
92
00:14:36,509 --> 00:14:43,350
parity bits are linear combination of these
message bits So in a systematic form basically
93
00:14:43,350 --> 00:14:51,980
we can separate out the message part from
the parity bit part So as I said for a systematic
94
00:14:51,980 --> 00:15:01,819
linear block codes the message part will consist
of the k information bits and the remaining
95
00:15:01,819 --> 00:15:10,149
n minus k bits which are the parity bits basically
they will be linear combination of these message
96
00:15:10,149 --> 00:15:17,709
bits So we can write down the encoding equations
for these matrices for these for systematic
97
00:15:17,709 --> 00:15:29,170
code so if you look at what is our encoding
equation Our v is u times G where G is of
98
00:15:29,170 --> 00:15:43,359
form like this Ok So if we write u which is
basically u 0 u 1 to u k minus 1 times this
99
00:15:43,359 --> 00:15:53,779
G matrix what we will get is a form like this
So you will have n minus k parity equations
100
00:15:53,779 --> 00:16:00,100
parity check equations which are given by
this expression and then you will have remaining
101
00:16:00,100 --> 00:16:10,679
k unaltered message bits So for a linear block
code in a systematic form the encoding equations
102
00:16:10,679 --> 00:16:17,649
will be of this form And as I said since we
are restricting ourselves without any loss
103
00:16:17,649 --> 00:16:25,339
of generality to binary codewords this addition
is basically done modulo 2
104
00:16:25,339 --> 00:16:33,410
So what we have seen so far is we can describe
a linear block code by its generator matrix
105
00:16:33,410 --> 00:16:40,910
which is a k cross n matrix and we can use
this generator matrix to generate our set
106
00:16:40,910 --> 00:16:49,060
of codewords Now there is another matrix which
we call parity check matrix which is related
107
00:16:49,060 --> 00:16:57,279
to our generator matrix we will show which
can also be used to completely describe a
108
00:16:57,279 --> 00:17:09,689
linear block code So for a n k linear block
code can be specified by a n minus k cross
109
00:17:09,689 --> 00:17:17,660
n parity check matrix which we denote by G
H the generator matrix we denote by G and
110
00:17:17,660 --> 00:17:25,110
the parity check matrix we denote by H Now
this parity check matrix has this property
111
00:17:25,110 --> 00:17:37,250
that if v is your valid codeword if and only
if v H transpose is going to be 0 So if v
112
00:17:37,250 --> 00:17:45,399
is a valid codeword v H transpose will be
0 So let us see how we can derive our parity
113
00:17:45,399 --> 00:17:52,350
check matrix from a generator matrix and what's
a relation of the generation matrix with the
114
00:17:52,350 --> 00:18:02,670
parity check matrix So we will take an example
of a 7 4 systematic linear block code whose
115
00:18:02,670 --> 00:18:10,230
generator matrix is given by this So since
this is a systematic code we can write it
116
00:18:10,230 --> 00:18:21,429
of the form p times this i k This generator
matrix can be written of this form Ok Now
117
00:18:21,429 --> 00:18:29,779
from this generator equation we can write
our coded bits in terms of our message bits
118
00:18:29,779 --> 00:18:38,970
So let's do that So the encoding equation
this was our generator matrix G this is our
119
00:18:38,970 --> 00:18:46,289
information bits message bits u and this is
our coded bits v So we can write v as u times
120
00:18:46,289 --> 00:18:57,200
G So it’s a 7 4 code So there are 4 information
bits I denote them by u 0 u 1 u 2 u 3 And
121
00:18:57,200 --> 00:19:04,159
there are 7 coded bits I denote them by v
0 v 1 v 2 to v 6 And this generator matrix
122
00:19:04,159 --> 00:19:12,080
I have already given you this So I can write
down the encoding equations We do that You
123
00:19:12,080 --> 00:19:20,720
can see from this what is v 0 It is u 0 plus
u 2 plus u 3
124
00:19:20,720 --> 00:19:31,659
There is a typing mistake here This should
have been u 2 So v 0 is u 0 times 1 plus 0
125
00:19:31,659 --> 00:19:44,140
times u 1 plus 1 times u 2 plus 1 times u
3 So this v 0 is given by u 0 plus u 2 plus
126
00:19:44,140 --> 00:19:58,360
u 3 Similarly what is v 1 v 1 is given by
u 0 plus u 1 plus u 2 v 2 is given by u 1
127
00:19:58,360 --> 00:20:10,269
u 0 into 0 u 1 into 1 u 2 plus u 2 into 1
plus u 3 into 1 So v 2 is given by u 1 plus
128
00:20:10,269 --> 00:20:20,289
u 2 plus u 3 So that's what I have here What
is v 3 v 3 is given by u 0 into 1 and rest
129
00:20:20,289 --> 00:20:38,169
are all 0s So v 3 is nothing but u 0 Similarly
v 4 is u 1 v 5 is u 2 and v 6 is u 3 So I
130
00:20:38,169 --> 00:20:49,039
now have set of 7 coded bits and this shows
the relation between the coded bits and information
131
00:20:49,039 --> 00:20:55,389
bits Now we are since we are restricting ourselves
to binary codes we can even write this equation
132
00:20:55,389 --> 00:21:12,139
like this v 0 plus u 0 plus u 2 plus u 3 is
equal to 0 correct Because this v 0 v 1 is
133
00:21:12,139 --> 00:21:17,490
nothing but parity bit which is basically
nothing but like 1 or 0 So we add this to
134
00:21:17,490 --> 00:21:30,290
this Modulo 2 sum will be 0 So this similarly
we can write as v 1 plus u 0 plus u 1 plus
135
00:21:30,290 --> 00:21:43,860
u 2 equal to 0 and this can be written as
v 2 plus u 1 plus u 2 plus u 3 is equal to
136
00:21:43,860 --> 00:21:54,159
0 The next what we would try to do is we would
try to write these parity check equations
137
00:21:54,159 --> 00:22:03,879
in terms of other coded bits So we can see
here u 0 is nothing but v 3 So wherever u
138
00:22:03,879 --> 00:22:14,580
0 appears we can replace it by v 3 Similarly
u 1 is equal to v 4 So wherever u 1 appears
139
00:22:14,580 --> 00:22:21,960
we can replace it by v 4 u 2 is equal to v
5 so we can replace u 2 by v 5 And u 3 is
140
00:22:21,960 --> 00:22:31,120
equal to v 6 We can replace u 3 in terms of
v 6 By doing this what we will get is set
141
00:22:31,120 --> 00:22:40,590
of equations which basically are dependent
on these coded bits If we do that what we
142
00:22:40,590 --> 00:22:51,789
get is something like this The first expression
basically which was v 0 plus u 0 the u 2 plus
143
00:22:51,789 --> 00:22:52,879
u 3
144
00:22:52,879 --> 00:23:04,379
Now this can be re-written as v 0 plus what
is u 0 u 0 is v 3 v 3 plus what is u 2 u 2
145
00:23:04,379 --> 00:23:16,680
is v 5 v 5 plus what is u 3 it is v 6 So v
0 plus v 3 plus v 5 plus v 6 is 0 and that
146
00:23:16,680 --> 00:23:28,529
is what we have here v 0 plus v 3 plus v 5
plus v 6 is equal to 0 Similarly we can write
147
00:23:28,529 --> 00:23:37,940
the other equations as well Here also we will
replace u 0 u 1 u 2 by v 3 v 4 v 5 and what
148
00:23:37,940 --> 00:23:47,049
we will get is v 1 plus v 3 plus v 4 plus
v 5 is equal to 0 and similarly the last parity
149
00:23:47,049 --> 00:23:57,950
check equation can be written as v 2 plus
v 2 plus u 1 is v 4 u 2 is v 5 plus u 3 is
150
00:23:57,950 --> 00:24:06,480
v 6 So that's what we have here u 2 v 2 plus
v 4 plus v 5 plus v 6 is equal to 0 So now
151
00:24:06,480 --> 00:24:14,019
we have set of encode equations in terms of
coded bits Next the same thing we can write
152
00:24:14,019 --> 00:24:25,730
it in a matrix form So I have my coded bits
v 0 to v 6 I have 3 sets of parity check equations
153
00:24:25,730 --> 00:24:33,240
this this and this And the same thing I can
write it in a matrix form like this Now you
154
00:24:33,240 --> 00:24:40,380
can see these are equivalent So look let's
look at first equation This is v 0 plus v
155
00:24:40,380 --> 00:24:49,299
3 plus v 5 plus v 6 is 0 You can see which
are the elements which are so v 0 times 1
156
00:24:49,299 --> 00:24:58,629
this is v 3 times 1 plus v 5 times 1 plus
v 6 times 1 So that's what is defined in this
157
00:24:58,629 --> 00:25:08,490
equation Similarly we can see this equation
This v 1 plus v 3 plus v 4 plus v 5 is equal
158
00:25:08,490 --> 00:25:17,850
to 0 and this last equation this is v 2 plus
v 4 plus v 5 plus v 6 is 0 And what we did
159
00:25:17,850 --> 00:25:22,149
we say about parity check matrix
160
00:25:22,149 --> 00:25:31,600
We said that if H is a parity check matrix
it is n minus k cross matrix and it has this
161
00:25:31,600 --> 00:25:42,549
property that v H transpose is 0 So we have
we can write this as this is my v this is
162
00:25:42,549 --> 00:25:51,419
my H transpose v H transpose is 0 so then
what is my H matrix H matrix is a transpose
163
00:25:51,419 --> 00:26:11,269
of this matrix so this will be 1 0 0 0 1 0
0 0 1 1 1 0 0 1 1 1 1 1 and 1 0 1 This is
164
00:26:11,269 --> 00:26:21,999
my so for the 7 4 code 7 4 code this is basically
3 cross 7 As I said n minus k cross n matrix
165
00:26:21,999 --> 00:26:34,399
this is my parity check matrix corresponding
to this same code which is generated by this
166
00:26:34,399 --> 00:26:39,010
Another interesting property which you can
generally see is so v H transpose is 0 I can
167
00:26:39,010 --> 00:26:49,070
write this u times v H transpose is equal
to 0 In other words v H transpose is 0 so
168
00:26:49,070 --> 00:26:59,350
what does that mean The rows of G matrix and
rows of H matrix are orthogonal to each other
169
00:26:59,350 --> 00:27:08,950
So the H lies in the null space of G so as
we can see from this that generator matrix
170
00:27:08,950 --> 00:27:18,340
and parity check matrix are related to each
other And they have this property that rows
171
00:27:18,340 --> 00:27:26,470
of G matrix and H matrix are basically orthogonal
to each other So if you have a systematic
172
00:27:26,470 --> 00:27:34,399
code whose generator matrix can be written
in this form because H lies in the null space
173
00:27:34,399 --> 00:27:41,750
of G we can write down its corresponding H
matrix very easily And this is basically given
174
00:27:41,750 --> 00:27:48,169
by so if a generator matrix can be written
of the form P and identity matrix we can write
175
00:27:48,169 --> 00:27:55,019
its parity check matrix as identity matrix
and P transpose
176
00:27:55,019 --> 00:28:01,350
So let's take an example of a generator matrix
of a systematic code This is systematic code
177
00:28:01,350 --> 00:28:07,330
we can see we can separate out this generator
matrix as some matrix P and some identity
178
00:28:07,330 --> 00:28:14,440
matrix So this we can write as H matrix we
can write as identity matrix and P transpose
179
00:28:14,440 --> 00:28:24,159
So then this can be written as 1 1 0 is 1
1 0 0 1 1 0 1 1 1 1 1 1 1 1 0 1 0 so this
180
00:28:24,159 --> 00:28:33,299
is my h matrix corresponding to this So whether
you are given a generator matrix or a parity
181
00:28:33,299 --> 00:28:40,460
check matrix your linear block code is completely
specified by either of them And as I said
182
00:28:40,460 --> 00:28:46,119
we use the generator matrix to generate our
code set of codewords where as parity check
183
00:28:46,119 --> 00:28:53,570
matrix as the name suggests is used to check
whether the parity check constraints are satisfied
184
00:28:53,570 --> 00:29:00,190
As we said basically parity check matrix has
this property that if v is a valid codeword
185
00:29:00,190 --> 00:29:07,020
if and only if v H transpose is zero and we
use this property in decoding so that's why
186
00:29:07,020 --> 00:29:12,549
you see the name parity check matrix because
this matrix H is essentially used to in some
187
00:29:12,549 --> 00:29:20,299
sense check whether the parity check constraints
of the code are satisfied or not Thank you