1
00:00:13,889 --> 00:00:22,640
Before we discuss decoding of linear block
codes let us solve some problems today So
2
00:00:22,640 --> 00:00:29,070
first question we are going to look at is
consider a linear block code C whose parity
3
00:00:29,070 --> 00:00:36,900
check matrix is given by this And you are
asked what are the code parameters n and k
4
00:00:36,900 --> 00:00:42,940
n which is a block length codeword length
and k is the size it’s the dimension of
5
00:00:42,940 --> 00:00:51,420
the basically information sequence length
s k Now how do we solve it We know we will
6
00:00:51,420 --> 00:01:01,210
first find out what is the rank of this matrix
H Now you can see this is a 4 cross 7 matrix
7
00:01:01,210 --> 00:01:09,750
right So the maximum rank possible is 4 Let's
see whether it has rank 4 Now if you add row
8
00:01:09,750 --> 00:01:27,990
1 2 and 3 what do you get 1 1 1 1 0 1 0 sorry
1 1 1 0 this is 0 0 1 0 This is what you get
9
00:01:27,990 --> 00:01:41,580
you can see this is 1 this is 1 this is 1
this is 0 this is 0 this is 1and this is 0
10
00:01:41,580 --> 00:01:44,899
And what is row number 4
11
00:01:44,899 --> 00:01:54,891
It’s exactly same as this So you can see
row 1 row 2 row 3 and row 4 add up to 0 That
12
00:01:54,891 --> 00:02:05,460
means it does not have rank 4So maximum rank
possible is 3 So let's see if any 3 rows combination
13
00:02:05,460 --> 00:02:13,480
add up to 0 So let's see let's see if we can
consider sum of these two rows This is what
14
00:02:13,480 --> 00:02:27,230
1 1 0 0 1 0 1 Now none of the rows are equal
to this you can see If we consider this row
15
00:02:27,230 --> 00:02:42,550
and this row we add these two rows Let's see
What do we get is 1 0 1 1 1 0 0 Now note none
16
00:02:42,550 --> 00:02:50,050
of these rows r 2 and r 4 is equal to this
So these set of 3 rows basically they are
17
00:02:50,050 --> 00:03:01,300
independent Let's try adding up this and this
If we add first row and fourth row what do
18
00:03:01,300 --> 00:03:19,239
we get 0 1 1 1 0 0 and 1 and note row number
3 and 2 are not same as this Like that we
19
00:03:19,239 --> 00:03:27,270
can check we can check for example row 2 and
4 we add up row 2 and 4 what do we get 1 0
20
00:03:27,270 --> 00:03:41,599
1 1 1 0 0 now note row number 3 and row number
1 are not same as this So we can see that
21
00:03:41,599 --> 00:03:50,980
any 3 rows do not add up to 0 So the rank
of this matrix H is 3 So rank of this matrix
22
00:03:50,980 --> 00:04:02,450
is 3 Now we know parity check matrix is n
minus k cross n So n minus k is in our case
23
00:04:02,450 --> 00:04:19,400
equal to 3 and what is n number of columns
of this So that's 1 2 3 4 5 6 7 So n is 7
24
00:04:19,400 --> 00:04:31,570
So that would then give us k equal to 4 So
this is an example parity check matrix for
25
00:04:31,570 --> 00:04:35,969
a 7 4 linear block code Ok
26
00:04:35,969 --> 00:04:42,159
Now let's look at another problem You are
given a set of codewords And what are these
27
00:04:42,159 --> 00:04:53,510
codewords These are binary codewords So this
is all zero 1 1 0 0 1 1 0 1 1 1 0 1 and 1
28
00:04:53,510 --> 00:05:04,000
1 all 1 sequence And the question that has
been asked is is this a linear code Is this
29
00:05:04,000 --> 00:05:10,570
a linear code Now what do we know about linear
code A linear code should have all 0 codewords
30
00:05:10,570 --> 00:05:18,540
which this codeword has And sum of any two
codewords is also a valid codeword So let's
31
00:05:18,540 --> 00:05:28,050
see So let's see if sum of all codewords is
already a valid codeword So let's call this
32
00:05:28,050 --> 00:05:40,050
v 0 v 1 v 2 and v 3 So what we want is all
possible combinations of v 0 v 1 v 2 v 3 should
33
00:05:40,050 --> 00:05:48,430
also be a valid codeword They should be in
C So let's see So as I said we take v 0 to
34
00:05:48,430 --> 00:05:58,320
means all zero codewords v 1 is given by this
this is v 1 this is v 2 and this is v 3 Now
35
00:05:58,320 --> 00:06:04,880
let's see all possible combinations of v 1
v 2 v 3 the non-zero codewords So we consider
36
00:06:04,880 --> 00:06:18,960
v 1 plus v 2 What is v 1 plus v 2 v 1 plus
v 2 is you can see this is 1 0 1 1 1 0 it's
37
00:06:18,960 --> 00:06:31,060
given by this Now is this codeword in C We
don't see any codeword which is 1 0 1 1 1
38
00:06:31,060 --> 00:06:43,880
0 listed here That means this C is not a linear
code Why it's not a linear code because sum
39
00:06:43,880 --> 00:06:52,709
of any two codewords is also a valid codeword
Now v 1 and v 2 are valid codewords in C So
40
00:06:52,709 --> 00:06:56,399
sum of v 1 plus v 2 should also be in C
41
00:06:56,399 --> 00:07:04,470
But we notice that 1 0 1 1 1 0 which is sum
of v 1 plus v 2 is not there in C And that's
42
00:07:04,470 --> 00:07:15,620
why we say that C is not a linear block code
Now my next question is can we add additional
43
00:07:15,620 --> 00:07:24,089
codewords here such that C becomes a linear
block code Now how do we do that To do that
44
00:07:24,089 --> 00:07:30,570
we will have to ensure all possible combinations
of these codewords is also there in C So let's
45
00:07:30,570 --> 00:07:39,080
compute v 1 plus v 3 which is basically given
by 0 0 1 1 0 0 Let's look at v 2 plus v 3
46
00:07:39,080 --> 00:07:48,280
which is given by 1 0 0 1 0 And let's look
at v 1 plus v 2 plus v 3 is basically given
47
00:07:48,280 --> 00:08:00,520
by 0 1 0 0 0 1 So note that I have listed
all possible combinations of these codewords
48
00:08:00,520 --> 00:08:09,310
here Now none of these sums are there in this
linear block code So if we add them in this
49
00:08:09,310 --> 00:08:19,000
set of C set of codewords then we our block
code C will become a linear block code So
50
00:08:19,000 --> 00:08:25,830
if we want to make it a linear block code
what do we need to do In this set of 4 codewords
51
00:08:25,830 --> 00:08:37,710
v 0 v 1 v 2 and v 3 we need to add these set
of codewords which was basically v 1 plus
52
00:08:37,710 --> 00:08:53,580
v 2 This is v 1 plus v 2 This is v 1 plus
v 3 v 1 plus v 3 Then this one is v 2 plus
53
00:08:53,580 --> 00:09:09,370
v 3 v 2 plus v 3 and this one was v 1 plus
v 2 plus v 3 So let's look at these 2 codewords
54
00:09:09,370 --> 00:09:17,220
This is v 1 plus v 2 and this is v 1 plus
v 2 plus v 3 So if we add these two what we
55
00:09:17,220 --> 00:09:20,350
will get is v 3 We can double check
56
00:09:20,350 --> 00:09:27,950
So if we consider add these two the first
bit will be 1 this 0 plus 1 will be 1 then
57
00:09:27,950 --> 00:09:38,360
1 plus 0 will be 1 then 1 plus 0 will be 1
then 1 plus 0 will be 1 and 0 plus 1 will
58
00:09:38,360 --> 00:09:45,010
be 1 And this is already there in this set
of codewords This is v 3 Ok Similarly take
59
00:09:45,010 --> 00:09:53,380
these two This one is v 1 plus v 2 and this
is v 2 plus v 3 If we add them what we get
60
00:09:53,380 --> 00:10:01,550
is v 1 plus v 3 We will get this If we consider
these two we will get v 2 We will consider
61
00:10:01,550 --> 00:10:08,570
this we will get v 3 If we consider these
two sum of these two we will get v 3 If we
62
00:10:08,570 --> 00:10:17,130
consider sum of these three what we will get
we will get v 3 So you can see basically linear
63
00:10:17,130 --> 00:10:25,650
combinations of all these codewords is already
there in this C So this C which contains the
64
00:10:25,650 --> 00:10:40,190
set of 8 codewords is a linear code And what
are the parameters n and k Now the length
65
00:10:40,190 --> 00:10:48,141
of the codewords is 6 Each of these codewords
are 6 bits So that's why n is 6 And there
66
00:10:48,141 --> 00:10:49,810
are total 2 k codewords
67
00:10:49,810 --> 00:11:04,010
And in our case 2 k is basically 8 So k is
3 This is basically a 6 3 linear binary code
68
00:11:04,010 --> 00:11:11,240
Now if I ask you tell me what is a generator
matrix that will generate this set of codewords
69
00:11:11,240 --> 00:11:19,980
Now how can you do that So we know the generator
matrix It's basically a k cross n matrix right
70
00:11:19,980 --> 00:11:27,580
So if you take basically 3 k in this case
is 3 if you take 3 codewords which are linearly
71
00:11:27,580 --> 00:11:34,760
independent basically if you take them and
form them as rows of your generator matrix
72
00:11:34,760 --> 00:11:44,220
then you get your generator matrix So I just
took this v 1 v 2 and v 3 and you can verify
73
00:11:44,220 --> 00:11:54,530
that rank of this matrix G is 3 It's full
rank Ok So then this G will be able to this
74
00:11:54,530 --> 00:12:05,750
generator matrix will be able to generate
this set of codewords Now can we put this
75
00:12:05,750 --> 00:12:13,440
is this generator matrix in systematic form
The answer is no Because to get it in systematic
76
00:12:13,440 --> 00:12:22,180
form what we need is our generator matrix
should be of the form like this or something
77
00:12:22,180 --> 00:12:31,480
like this Ok But this is not in this particular
form So we will have to get some identity
78
00:12:31,480 --> 00:12:41,130
matrix and some matrix P Now by doing elementary
row operation we can put this in systematic
79
00:12:41,130 --> 00:12:50,370
form So let's do that So note if we want to
get let's say this in the form of identity
80
00:12:50,370 --> 00:12:58,290
what do we need We would need basically here
we would need a 0 here we would need a 0 here
81
00:12:58,290 --> 00:13:08,060
we would need a 0 here we would need a 0 right
So first let's try to get this 1 to 0 Now
82
00:13:08,060 --> 00:13:16,620
how can we make this 0 So if we do this transformation
that row 3 is row 3 plus row 1 So row 3 is
83
00:13:16,620 --> 00:13:25,670
row 3 plus row 1 if we do that then 1 plus
1 this will be 0 1 plus 1 this is 0 0 plus
84
00:13:25,670 --> 00:13:38,260
1 this is 1 0 plus 1 this is 1 1 plus 1 this
is 0 and 1 plus 1 this is 0 Ok So we got a
85
00:13:38,260 --> 00:13:42,230
0 here right
86
00:13:42,230 --> 00:13:56,960
Next we want a 0 here We want this we want
to make this 0 So how can we do that We do
87
00:13:56,960 --> 00:14:07,070
this transformation that row 2 is row 3 plus
row 1 row 2 So if row 2 is row 2 plus row
88
00:14:07,070 --> 00:14:11,910
3 then what's going to happen This will remain
0 this will remain 1 but this 1 will become
89
00:14:11,910 --> 00:14:20,910
0 So let's do that So this is 0 plus 0 is
0 1 plus 0 is 1 1 plus 1 is 0 1 plus 1 is
90
00:14:20,910 --> 00:14:33,790
0 0 plus 0 is 0 and 1 plus 0 is 1 Ok So we
got these 0's we got this 0 Ok now what do
91
00:14:33,790 --> 00:14:40,860
we have to do We will have to get this here
we will have to get a 0 So how can we get
92
00:14:40,860 --> 00:14:51,500
a 0 here We will do this transformation We
will add row 1 and row 2 and replace row 1
93
00:14:51,500 --> 00:14:59,660
by this So we are going to add these 2 rows
If we add these 2 rows what's going to happen
94
00:14:59,660 --> 00:15:06,080
This 1 will remain 1 1 plus 1 this will become
0 and this will remain 0 This will be 0 this
95
00:15:06,080 --> 00:15:15,410
will be 1 and this will be 0 So if we do this
transformation what we get is this Now note
96
00:15:15,410 --> 00:15:24,730
that this is our identity matrix This is 3
cross 3 identity matrix and then this is your
97
00:15:24,730 --> 00:15:34,000
another matrix P Ok So by doing elementary
row operation we are able to get our generator
98
00:15:34,000 --> 00:15:41,430
matrix in a systematic form And if we have
a generator matrix in a systematic form we
99
00:15:41,430 --> 00:15:50,190
can very easily find the parity check matrix
in systematic form So this is like I k P then
100
00:15:50,190 --> 00:16:02,750
this H matrix will be P transpose I n minus
k So this this is basically your P transpose
101
00:16:02,750 --> 00:16:14,180
This is 0 1 0 this will come here 0 1 0 0
0 1 this is 0 0 1 And 1 0 0 is this 1 0 0
102
00:16:14,180 --> 00:16:23,920
And then you have this identity matrix which
is here Ok
103
00:16:23,920 --> 00:16:35,270
Next we are given a parity check matrix H
of a linear block code with parameter n and
104
00:16:35,270 --> 00:16:44,830
k And it is given that this code C has both
odd weight codewords and even weight codewords
105
00:16:44,830 --> 00:16:52,370
In other words the number of 1's in the codewords
it contains both odd number of 1's as well
106
00:16:52,370 --> 00:17:01,371
as even number of 1's And we are constructing
a new code that we are calling as C 1 and
107
00:17:01,371 --> 00:17:12,909
the parity check matrix of the new code C
1 is given by this So how do we find this
108
00:17:12,909 --> 00:17:22,360
new matrix parity check matrix H 1 We are
adding a new column which is 0 in the initial
109
00:17:22,360 --> 00:17:32,650
rows except in the last row which is a 1 and
here we have put our original n minus k cross
110
00:17:32,650 --> 00:17:45,160
n matrix And the last row is basically all
1's Ok So the dimension of this matrix is
111
00:17:45,160 --> 00:17:59,620
so number of rows is n minus k plus 1 and
number of columns are n plus 1 Now you are
112
00:17:59,620 --> 00:18:11,130
asked to show that the code generated by this
parity check matrix H 1 is a linear code with
113
00:18:11,130 --> 00:18:23,080
parameters n plus 1 and k Second thing you
are asked to prove is that all the codewords
114
00:18:23,080 --> 00:18:29,780
of this new code C 1 will have even weight
That means they will have even number of 1's
115
00:18:29,780 --> 00:18:39,380
in them The third thing you have to prove
is this new code C 1 is obtained from old
116
00:18:39,380 --> 00:18:46,100
code C by adding an additional parity bit
which we are denoting by v infinity to the
117
00:18:46,100 --> 00:18:53,100
left of this codeword and how do you select
this parity bit v infinity If the original
118
00:18:53,100 --> 00:19:01,840
codeword has odd weight then you put v infinity
as 1 otherwise if the original codeword has
119
00:19:01,840 --> 00:19:06,900
even weight then you put this v infinity as
0
120
00:19:06,900 --> 00:19:14,700
So let's prove one by one Let's first prove
this that code generated by this new parity
121
00:19:14,700 --> 00:19:25,330
check matrix is basically a new code with
n given by n plus 1 and k given by k So as
122
00:19:25,330 --> 00:19:32,770
we know that this H matrix has these dimensions
because we are adding a new column and we
123
00:19:32,770 --> 00:19:43,720
are adding a new row Next now what is the
rank of the original matrix H The rank of
124
00:19:43,720 --> 00:19:52,510
the original matrix H is n minus k That means
the n minus k rows of the original parity
125
00:19:52,510 --> 00:20:02,370
check matrix H are linearly independent Ok
Now go back and look at the new construction
126
00:20:02,370 --> 00:20:11,240
So these n minus k rows are linearly independent
And what have we added here We have added
127
00:20:11,240 --> 00:20:25,610
0 here So these new rows these new n minus
k rows will also be linearly independent So
128
00:20:25,610 --> 00:20:32,600
that's what we are saying that since n minus
k rows of the original parity check matrix
129
00:20:32,600 --> 00:20:41,010
H are linearly independent so the first n
minus k rows of the original parity check
130
00:20:41,010 --> 00:20:54,799
matrix H 1 will also be linearly independent
Now let's look at the last row of this new
131
00:20:54,799 --> 00:21:05,809
parity check matrix H 1 Note that we have
1 here And these are all 1's here Whereas
132
00:21:05,809 --> 00:21:14,630
here all of these are 0's So this new row
will also be linearly independent from any
133
00:21:14,630 --> 00:21:23,760
of the other rows of this parity check matrix
H 1 So any linear combination including the
134
00:21:23,760 --> 00:21:32,160
last row of H 1 will never result in a all
zero vector So what does it mean It means
135
00:21:32,160 --> 00:21:45,630
that n minus k plus one rows of our new parity
check matrix H one are linearly independent
136
00:21:45,630 --> 00:21:59,429
Hence the dimension of H 1 is n minus k plus
1 Now how do we find the dimension of basically
137
00:21:59,429 --> 00:22:05,600
the null space of this parity check matrix
H 1 It is given by so number of columns is
138
00:22:05,600 --> 00:22:13,799
n plus 1 The dimension of H 1 is given by
this So this is the dimension of the null
139
00:22:13,799 --> 00:22:20,860
space of this parity check matrix So then
basically number of information bits is then
140
00:22:20,860 --> 00:22:31,990
k and number of coded bits is n plus 1 So
this proves that C 1 is an n plus 1 k linear
141
00:22:31,990 --> 00:22:34,870
code
142
00:22:34,870 --> 00:22:43,340
Next we are going to show is every codeword
of C 1 has even weight So how do we prove
143
00:22:43,340 --> 00:22:52,990
this Please note that the last row of this
parity check matrix H 1 contain all 1 vector
144
00:22:52,990 --> 00:23:02,960
If you go back recall the last row of this
parity check matrix has all 1's And if v is
145
00:23:02,960 --> 00:23:13,830
a valid codeword what property does it satisfy
If v is a valid codeword then v H transpose
146
00:23:13,830 --> 00:23:22,820
should be 0 Now let us take a codeword let's
say there exists a codeword with odd weight
147
00:23:22,820 --> 00:23:33,120
which is generated by described by this parity
check matrix H 1 Now if we do v H transpose
148
00:23:33,120 --> 00:23:43,250
so when you are going to take the inner product
of this code vector v with the last row of
149
00:23:43,250 --> 00:23:53,540
this parity check matrix what will you get
You will essentially get some of so basically
150
00:23:53,540 --> 00:24:02,230
if you do v H transpose so your H is of the
so v H transpose where H is H 1 transpose
151
00:24:02,230 --> 00:24:10,380
where H 1 is given like this This is all 0's
You have your parity check matrix H and you
152
00:24:10,380 --> 00:24:22,430
have here all 1 matrix sorry you have your
this is 1 this is 1 and this is all 1 vector
153
00:24:22,430 --> 00:24:30,960
Now when you do v H transpose so let's say
v is your v 0 to v n minus 1 When you do v
154
00:24:30,960 --> 00:24:41,270
H 1 transpose what you will get is v 0 plus
v 1 plus v 2 up to v n minus 1 is going to
155
00:24:41,270 --> 00:24:55,559
be 0 Now if this v has odd number of 1's this
sum cannot be 0 right Hence we prove that
156
00:24:55,559 --> 00:25:04,549
v has to have even number of 1's because we
know if v is a valid codeword then v H 1 transpose
157
00:25:04,549 --> 00:25:12,930
should be 0 So if we do v H transpose because
the last row of this parity check matrix H
158
00:25:12,930 --> 00:25:18,640
1 is all 1 the condition that we will get
is the individual components of this parity
159
00:25:18,640 --> 00:25:28,000
code vector v v 0 plus v 1 plus v 2 plus v
3 up to v n minus 1 basically v n minus 1
160
00:25:28,000 --> 00:25:39,070
they should all add up to 0 Hence we cannot
have an odd weight vector which will give
161
00:25:39,070 --> 00:25:52,640
v H transpose v H 1 transpose to be 0 Hence
every codeword in C 1 has even weight
162
00:25:52,640 --> 00:26:00,130
Next we are going to show prove how we can
generate this new code C 1 from the original
163
00:26:00,130 --> 00:26:08,360
code C and what did we mention We mentioned
that this new code C 1 can be obtained from
164
00:26:08,360 --> 00:26:16,440
the original code C by adding an extra parity
bit which we are denoting by v infinity to
165
00:26:16,440 --> 00:26:26,570
the left of the original codeword v in this
fashion If v has odd weight then v infinity
166
00:26:26,570 --> 00:26:34,830
is odd parity and if v has even weight then
v infinity is zero parity is zero is even
167
00:26:34,830 --> 00:26:35,830
parity
168
00:26:35,830 --> 00:26:44,780
So let's prove this So let's see let v be
a codeword in C Then v H transpose will be
169
00:26:44,780 --> 00:26:52,590
0 Now we are extending this original code
v by adding a bit v infinity to its left So
170
00:26:52,590 --> 00:27:00,000
we are defining a new codeword of length n
plus 1 which is defined as follows So this
171
00:27:00,000 --> 00:27:07,200
is your original codeword v which is basically
v 0 to v n minus 1 and then this is the additional
172
00:27:07,200 --> 00:27:16,990
parity bit that you added to the left Now
if v 1 is a valid codeword then it should
173
00:27:16,990 --> 00:27:27,790
satisfy the property that v 1 H 1 transpose
should be 0 And what is our H 1 Again please
174
00:27:27,790 --> 00:27:36,900
recall our H 1 is of form like this So the
first column here is 0 then you have here
175
00:27:36,900 --> 00:27:46,470
the original H matrix H and this is all 1
vector So when we do v H transpose so when
176
00:27:46,470 --> 00:27:58,560
v 1 will be multiplied by this last row what
we will get is condition of this form v infinity
177
00:27:58,560 --> 00:28:08,650
plus v 0 plus v 1 plus v 2 plus v n minus
1 that's basically should be equal to 0 Ok
178
00:28:08,650 --> 00:28:17,070
Now how are we getting this condition again
We are making use of the fact that v 1 H 1
179
00:28:17,070 --> 00:28:27,490
transpose is 0 and H 1 is of form like this
So when we do v 1 H 1 transpose the last row
180
00:28:27,490 --> 00:28:34,360
which will be H transpose will be last column
If we multiply v with that H 1 transpose column
181
00:28:34,360 --> 00:28:42,039
what we would get is something of this form
Now this should be equal to 0 if v 1 is a
182
00:28:42,039 --> 00:28:53,299
valid codeword right So if the sum has to
be 0 what do we need See the original codeword
183
00:28:53,299 --> 00:29:00,720
is odd weight codeword We need v infinity
to be 1 And if the original codeword is even
184
00:29:00,720 --> 00:29:12,000
parity then this new parity bit should be
0 and that's basically the proof how we can
185
00:29:12,000 --> 00:29:20,870
extend the original code to construct a new
code And this is basically if this is equal
186
00:29:20,870 --> 00:29:33,320
to 0 we know that v H transpose v 1 H 1 transpose
is 0 so v 1 is the valid codeword in C 1 And
187
00:29:33,320 --> 00:29:39,680
total there are 2 k codewords This we have
already proved in the first part that there
188
00:29:39,680 --> 00:29:49,740
are total 2 k codewords of length n plus 1
Ok So with this I will conclude this lecture
189
00:29:49,740 --> 00:29:50,120
Thank you