1
00:00:14,250 --> 00:00:22,300
Welcome to the course on Coding Theory In
this lecture we are going to describe how
2
00:00:22,300 --> 00:00:27,840
we can use error correcting codes for error
detection and error correction So we will
3
00:00:27,840 --> 00:00:37,300
first describe what we mean by Syndrome and
then we will show how we will use this Syndrome
4
00:00:37,300 --> 00:00:45,530
to do error correction and error detection
So this lecture is about Syndrome and error
5
00:00:45,530 --> 00:00:50,530
correction and error detection We will first
talk about what is a Syndrome and how we can
6
00:00:50,530 --> 00:00:57,280
use it for error detection And then we will
talk about how the Syndrome can be used for
7
00:00:57,280 --> 00:01:06,520
error correction So we know so far that if
we have an n k linear block codes so that
8
00:01:06,520 --> 00:01:15,000
means we have k information bits and n coded
bits and this n k linear block code is completely
9
00:01:15,000 --> 00:01:29,310
described by a generator matrix and a parity
check matrix So let us assume that we have
10
00:01:29,310 --> 00:01:39,689
codeword which is encoded using a linear n
k block encoder So we have an output of a
11
00:01:39,689 --> 00:01:48,979
encoder which is of coded bits of length n
Now we want to transmit this codeword over
12
00:01:48,979 --> 00:01:55,909
a communication channel For simplicity let's
consider a binary symmetric channel Again
13
00:01:55,909 --> 00:02:01,009
let's recall what is a binary symmetric channel
So in a binary symmetric channel we have 2
14
00:02:01,009 --> 00:02:09,729
inputs and 2 outputs so binary inputs 0s and
1 binary output 0s and 1 and with probability
15
00:02:09,729 --> 00:02:17,000
let's say 1 minus probability 1 minus p we
receive the bits correctly and there is a
16
00:02:17,000 --> 00:02:21,090
crossover probability of p that the bits get
flipped
17
00:02:21,090 --> 00:02:30,120
So this is a binary symmetric channel Let's
denote by r our received sequence received
18
00:02:30,120 --> 00:02:38,950
codeword which we sent over this binary symmetric
channel So the types of as I said the outputs
19
00:02:38,950 --> 00:02:47,930
of binary symmetric channel is also 0 and
1 so we can describe the output of a binary
20
00:02:47,930 --> 00:02:57,700
symmetric channel r as our transmitted codeword
plus some error vector So our transmitted
21
00:02:57,700 --> 00:03:05,600
codeword is an n-bit tuple Similarly our error
vector is also an n-bit So we describe our
22
00:03:05,600 --> 00:03:16,489
error vector by e 0 e 1 e 2 e n minus 1 and
whenever e i term is 1 that means that particular
23
00:03:16,489 --> 00:03:24,900
bit was not received correctly So we can write
an output of a binary symmetric channel So
24
00:03:24,900 --> 00:03:31,850
the first bit that we would receive is basically
v 0 plus e 0 v 1 plus second bit will be v
25
00:03:31,850 --> 00:03:38,370
1 plus e 1 similarly v 2 plus e 2 and last
finally we will get v n minus 1 and e n minus
26
00:03:38,370 --> 00:03:51,390
1 where this e 0 e 1 e 2 e n minus 1 is my
error pattern Now when does an error occur
27
00:03:51,390 --> 00:03:57,569
If my received sequence is not same as my
transmitted codeword then there is an error
28
00:03:57,569 --> 00:04:06,970
So the error is 1 only if my received bit
is not same as my transmitted bit If the received
29
00:04:06,970 --> 00:04:14,060
bit is same as my transmitted bit there is
no error That I will keep that bit e i as
30
00:04:14,060 --> 00:04:25,910
0 Ok now if a particular bit e i is 1 what
does it denote It denotes that that i’th
31
00:04:25,910 --> 00:04:36,080
bit is in error Now when we are sending this
codeword over a communication channel what
32
00:04:36,080 --> 00:04:43,630
are we interested in We are interested to
find out whether any error has occurred If
33
00:04:43,630 --> 00:04:49,460
any error has occurred we are interested to
find out the location where the error has
34
00:04:49,460 --> 00:04:59,840
occurred so that we can correct those errors
Ok So first we are going to show how we can
35
00:04:59,840 --> 00:05:01,720
detect error
36
00:05:01,720 --> 00:05:10,900
So we define a term which we call as Syndrome
What is a Syndrome Syndrome is computed by
37
00:05:10,900 --> 00:05:20,150
computing this r H transpose So r is basically
1 cross n vector H is n minus k cross n vector
38
00:05:20,150 --> 00:05:31,720
so H transpose is n into n minus k matrix
So this term r H transfer is known as Syndrome
39
00:05:31,720 --> 00:05:43,510
and this is when this is non zero it indicates
there is an error So r is a codeword if and
40
00:05:43,510 --> 00:05:53,639
only if the Syndrome is 0 So whenever the
Syndrome is 0 basically if the Syndrome is
41
00:05:53,639 --> 00:06:03,490
0 then r is a codeword if and only if Syndrome
is 0 and this is easy to check because we
42
00:06:03,490 --> 00:06:13,300
know that if v is a valid codeword then v
H transpose will be 0 So if there is no error
43
00:06:13,300 --> 00:06:22,410
my received sequence r would be just equal
to v And then the Syndrome would be v H transpose
44
00:06:22,410 --> 00:06:35,509
which is 0 And if Syndrome is not equal to
0 then it means there is an error Now if the
45
00:06:35,509 --> 00:06:46,110
Syndrome is 0 does it always mean that there
is no error No There is a category of error
46
00:06:46,110 --> 00:06:56,720
which we call as undetected error Now when
does an undetected error happen If your s
47
00:06:56,720 --> 00:07:05,840
is 0 and your received sequence r is not the
codeword that you transmitted but some other
48
00:07:05,840 --> 00:07:16,440
codeword let's say I transmitted a codeword
v 1 and my error e was such that that it transformed
49
00:07:16,440 --> 00:07:26,060
it into another codeword v 2 So received sequence
is v 2 Now if I compute Syndrome because v
50
00:07:26,060 --> 00:07:36,449
2 is a valid codeword then v H transpose will
be 0 So an undetected error happens when your
51
00:07:36,449 --> 00:07:43,080
error pattern is such that it transforms your
one codeword to another codeword And that's
52
00:07:43,080 --> 00:07:44,580
what we have written here
53
00:07:44,580 --> 00:07:55,289
So if s is 0 and r is a codeword that means
no errors are detected However if r is a codeword
54
00:07:55,289 --> 00:08:02,819
other than the actual codeword transmitted
then an undetected error has happened And
55
00:08:02,819 --> 00:08:10,889
this happens when an error pattern is a non-zero
codeword because we have said a property of
56
00:08:10,889 --> 00:08:17,319
linear block code that sum of 2 codewords
is also a codeword So for this scenario to
57
00:08:17,319 --> 00:08:27,760
happen this e has to be a valid non-zero codeword
So as I said we compute the Syndrome from
58
00:08:27,760 --> 00:08:37,500
the received sequence r the interesting part
is Syndrome depends only on the error pattern
59
00:08:37,500 --> 00:08:45,810
It does not depend on what codeword was transmitted
And this is easy to see So Syndrome is r H
60
00:08:45,810 --> 00:08:56,820
transpose which we can write r is my transmitted
codeword plus error vector this whole multiplied
61
00:08:56,820 --> 00:09:01,720
by H transpose So this I can write as v H
transpose and e H transpose
62
00:09:01,720 --> 00:09:09,810
Now what is v H transpose v H transpose is
0 because v is a valid codeword Then Syndrome
63
00:09:09,810 --> 00:09:19,510
is nothing but e H transpose So Syndrome does
not depend on transmitted codeword It only
64
00:09:19,510 --> 00:09:29,820
depends on the error pattern So I have an
example here For 7 4 linear block code whose
65
00:09:29,820 --> 00:09:36,980
parity check matrix is given by this Now let's
say my received sequence is received coded
66
00:09:36,980 --> 00:09:43,550
sequence is this And I am interested to find
whether there is any error in this received
67
00:09:43,550 --> 00:09:51,360
sequence So how do I do that So first I will
compute the Syndrome So what is Syndrome of
68
00:09:51,360 --> 00:10:00,170
r This Syndrome is r H transpose so r is this
and H is given so H transpose is basically
69
00:10:00,170 --> 00:10:09,500
1 0 0 0 1 0 0 0 1 so this is my H transpose
and when I multiply this I multiply this by
70
00:10:09,500 --> 00:10:21,810
this multiply this by this multiply this by
this what I get is 1 1 1 which is not 0 that
71
00:10:21,810 --> 00:10:30,760
means there is an error in my received sequence
As I said the Syndrome only depends on the
72
00:10:30,760 --> 00:10:39,050
error pattern It does not depend on transmitted
codeword And if you have your parity check
73
00:10:39,050 --> 00:10:45,940
matrix in a systematic form then you can write
your Syndrome equations in terms of error
74
00:10:45,940 --> 00:10:57,310
patterns like this So these are your basically
n minus k Syndrome equations
75
00:10:57,310 --> 00:11:08,040
So to recap what we have said so far we have
said that for error detection we need to compute
76
00:11:08,040 --> 00:11:16,200
the Syndrome And if the Syndrome is non-zero
that means an error has occurred Now we will
77
00:11:16,200 --> 00:11:25,649
move into error correction and we will talk
about how we can use the Syndrome for error
78
00:11:25,649 --> 00:11:34,589
correction So as we saw basically from the
Syndrome equations we had basically one cross
79
00:11:34,589 --> 00:11:44,950
n and h transpose was n cross n minus k So
we essentially had n minus k Syndrome equations
80
00:11:44,950 --> 00:11:52,060
So we had total n minus k equations and how
many unknowns we have we have n unknowns from
81
00:11:52,060 --> 00:12:01,660
location error at location 1 to error at location
n So we have n unknowns e 0 e 1 e 2 e n minus
82
00:12:01,660 --> 00:12:12,490
1 where as we have only n minus k equations
So we have less equations more unknowns And
83
00:12:12,490 --> 00:12:23,209
what we need to do is we need to solve this
set of equations to find out what these unknown
84
00:12:23,209 --> 00:12:31,639
quantities are Because to find out the error
pattern we need to know what these e i's are
85
00:12:31,639 --> 00:12:41,420
Ok So we need to solve these set of n minus
k equations to get back our corrected sequence
86
00:12:41,420 --> 00:12:49,399
And what would be our corrected sequence Estimated
codeword would be nothing but r received sequence
87
00:12:49,399 --> 00:12:54,740
plus estimated error pattern
88
00:12:54,740 --> 00:13:00,000
So we need to when we do error correction
essentially what we are trying to do is we
89
00:13:00,000 --> 00:13:08,410
are trying to estimate what the error pattern
is So as we can see from here because we have
90
00:13:08,410 --> 00:13:17,610
n minus k equations and n unknowns we have
total 2 to the power k solutions of these
91
00:13:17,610 --> 00:13:27,290
n minus k equations So there are total 2 k
solutions to these equations and out of them
92
00:13:27,290 --> 00:13:33,959
there is only one which is the correct error
pattern Out of those 2 k solutions there is
93
00:13:33,959 --> 00:13:44,060
only 1 error pattern which is correct Now
how do we choose the most likely error pattern
94
00:13:44,060 --> 00:13:51,050
from these 2 k solutions How do we choose
the most likely pattern That's basically what
95
00:13:51,050 --> 00:13:59,150
our objective is So when we try to minimize
probability of error we want to choose the
96
00:13:59,150 --> 00:14:06,560
most probable error pattern We want to choose
the most probable error pattern from those
97
00:14:06,560 --> 00:14:18,490
2 k solutions of this set of equations And
as we said we did an exercise you know in
98
00:14:18,490 --> 00:14:27,519
previous lectures we have shown the decoding
rule for maximum likelihood decoding rule
99
00:14:27,519 --> 00:14:33,709
for a binary symmetric channel and we have
shown that for maximum likelihood decoder
100
00:14:33,709 --> 00:14:43,690
we will choose our estimated codeword as one
that will minimize the Hamming distance between
101
00:14:43,690 --> 00:14:48,360
the received codeword and the transmitted
codeword In other words it would minimize
102
00:14:48,360 --> 00:14:57,459
the Hamming weight of the error pattern e
So from the 2 k different solutions basically
103
00:14:57,459 --> 00:15:03,670
the one which has the least Hamming weight
that's the best solution for the maximum likelihood
104
00:15:03,670 --> 00:15:09,360
decoding rule for a binary symmetric channel
105
00:15:09,360 --> 00:15:16,560
So let's take an example now So we have a
7 4 code whose parity check matrix is given
106
00:15:16,560 --> 00:15:24,839
by this and our transmitted codeword is this
and received codeword is this We can see that
107
00:15:24,839 --> 00:15:31,510
there is an error in this location This was
transmitted as 1 and this was received as
108
00:15:31,510 --> 00:15:40,600
0 so there is an error in this location Now
how do we find out that there is an error
109
00:15:40,600 --> 00:15:46,529
and there is an error in this location So
first thing that we will do is we will compute
110
00:15:46,529 --> 00:15:56,410
the Syndrome When we will compute the Syndrome
which is r H transpose what we get is a non-zero
111
00:15:56,410 --> 00:16:05,320
and since this is non-zero this means there
is an error Now next step is to find out where
112
00:16:05,320 --> 00:16:19,220
the error has occurred So in this case we
can write the Syndrome in terms of error bits
113
00:16:19,220 --> 00:16:28,339
So we have 3 equations and we have total 7
unknowns right And these equations are basically
114
00:16:28,339 --> 00:16:37,320
given by this We have our H matrix given by
this This is our error pattern so when we
115
00:16:37,320 --> 00:16:48,390
compute e H transpose we get set of these
3 equations And there are 16 solutions to
116
00:16:48,390 --> 00:16:56,470
this set of equations This set of 3 equations
where there are 7 unknowns there are total
117
00:16:56,470 --> 00:16:59,320
2 to power 4 different solutions
118
00:16:59,320 --> 00:17:08,630
These are the 16 different solutions and as
we said the maximum likelihood decoding rule
119
00:17:08,630 --> 00:17:14,490
for a binary symmetric channel will chose
an error pattern that has minimum Hamming
120
00:17:14,490 --> 00:17:23,160
weight so which has minimum number of 1s So
you can see among these 16 solutions the one
121
00:17:23,160 --> 00:17:30,520
that has minimum Hamming weight minimum number
of 1s is this All others have this has four
122
00:17:30,520 --> 00:17:37,110
1's this has four 1's this has five 1's three
1's two 1's you can check basically this one
123
00:17:37,110 --> 00:17:44,100
has the least number of 1's So this error
pattern has the least of Hamming weight So
124
00:17:44,100 --> 00:17:52,130
this is out of those 2 to the power 4 solutions
this is the most likely solution And this
125
00:17:52,130 --> 00:18:00,049
we can verify also because we were given the
received sequence and we knew what was the
126
00:18:00,049 --> 00:18:06,210
transmitted sequence was error pattern indeed
was this which we found out from by solving
127
00:18:06,210 --> 00:18:17,360
these set of equations So to recap then now
when we want to do error correction what do
128
00:18:17,360 --> 00:18:30,940
we need to do we need to solve for this Syndrome
equations And there are n minus k equations
129
00:18:30,940 --> 00:18:40,539
but there will be n unknowns so this will
have 2 k solutions and we will have to pick
130
00:18:40,539 --> 00:18:48,730
the most probable solution from these set
of 2 k solutions The next lecture we are going
131
00:18:48,730 --> 00:18:54,420
to talk about a general decoding algorithm
for a linear block code thank you