1 00:00:14,219 --> 00:00:21,820 Welcome to the course on Coding Theory Today in this lecture we are going to describe what 2 00:00:21,820 --> 00:00:33,510 we mean by generator matrix and parity check matrix So we will continue our discussion 3 00:00:33,510 --> 00:00:39,960 with introduction to linear block codes We will first describe what is a generator matrix 4 00:00:39,960 --> 00:00:44,160 and what is a parity check matrix and how are they related So as we described in the 5 00:00:44,160 --> 00:00:53,760 last class an encoder for a linear block codes what it does it takes a block of k-bits and 6 00:00:53,760 --> 00:01:02,970 maps it to the to n-bit Now the matrix we can use a k cross n matrix to define this 7 00:01:02,970 --> 00:01:13,500 mapping from k information bits to n coded bits and this matrix is basically our generator 8 00:01:13,500 --> 00:01:22,830 matrix for a so for a n-k linear block code the mapping of k information bits to n-coded 9 00:01:22,830 --> 00:01:32,850 bits is defined by this generator matrix G which is of rank k so if we denote information 10 00:01:32,850 --> 00:01:45,020 bits by u and we denote our coded bits by v then we can write v as u times g where our 11 00:01:45,020 --> 00:01:56,460 u is 1 cross k vector and this is generator matrix is k cross n matrix and our output 12 00:01:56,460 --> 00:02:00,479 coded bit is 1 cross n vector 13 00:02:00,479 --> 00:02:09,399 So as a name suggests basically generator matrix is used to generate our codeword So 14 00:02:09,399 --> 00:02:16,819 we generate our codewords using this generator matrix and this generator matrix gives the 15 00:02:16,819 --> 00:02:25,980 mapping between the information bits u to coded bits v So how do we find codewords We 16 00:02:25,980 --> 00:02:34,170 find codewords by taking linear combinations of rows of this generator matrix In case of 17 00:02:34,170 --> 00:02:43,810 binary codes so then these entries in the generator matrix are either 0 or 1 depending 18 00:02:43,810 --> 00:02:53,560 upon which bits are used to generate a particular coded sequence So we form a set of 2 k codewords 19 00:02:53,560 --> 00:03:03,290 by taking linear combinations of rows of these generator matrix So we can as I said we can 20 00:03:03,290 --> 00:03:14,100 write our coded sequence as q times p which is basically linear combination of rows of 21 00:03:14,100 --> 00:03:23,790 the generator matrix So these are basically linearly independent k rows and the rank of 22 00:03:23,790 --> 00:03:30,860 this generator matrix is k Since we are without loss of generality since we are talking about 23 00:03:30,860 --> 00:03:39,300 binary linear block codes so we will be doing this addition modulus 2 So what are the properties 24 00:03:39,300 --> 00:03:46,940 of linear block code Sum of any two codewords in a linear code is also a valid codeword 25 00:03:46,940 --> 00:04:01,459 So if v 1 and v 2 are valid codewords then v 1 plus v 2 will also be a valid codeword 26 00:04:01,459 --> 00:04:11,159 Also an all zero codeword is a valid codeword in any linear block codes So we can define 27 00:04:11,159 --> 00:04:20,629 a linear block code n k linear block code as k dimensional space of vector space v n 28 00:04:20,629 --> 00:04:28,220 of all binary n-tuples So we can define a linear binary block codes as a k dimensional 29 00:04:28,220 --> 00:04:35,840 subspace of vector space v n of all binary n tuples 30 00:04:35,840 --> 00:04:44,500 Now let us take an example to illustrate what is a generator matrix So in this example we 31 00:04:44,500 --> 00:04:55,520 have considered 3 information bits and 6 coded bits And in this table I have given you the 32 00:04:55,520 --> 00:05:06,490 set of 8 information sequences and their corresponding codewords So how do we find the generator 33 00:05:06,490 --> 00:05:14,060 matrix in this case So we will have to look at each of this code bits and see how are 34 00:05:14,060 --> 00:05:24,150 we generating these code bits in terms of message bits u 0 u 1 and u 2 So first thing 35 00:05:24,150 --> 00:05:32,770 we are going to do is look at each of these code bits v 0 v 1 v 2 v 3 v 4 v 5 and write 36 00:05:32,770 --> 00:05:45,389 them in terms of u 0 u 1 u 2 Ok So let’s look at each of these So v 0 is u 1 plus u 37 00:05:45,389 --> 00:05:55,490 2 we can see easily v 0 is this column and we can see this is same as u 1 plus u 2 So 38 00:05:55,490 --> 00:06:03,229 u 1 plus u 2 in this case is 0 u 1 plus u 2 is 0 1 plus 0 is 1 1 plus 0 plus 1 is 1 39 00:06:03,229 --> 00:06:11,860 1 plus 1 is 0 modular 2 and 1 plus 1 is 0 modular 2 So this v 0 is basically given by 40 00:06:11,860 --> 00:06:23,370 u 1 plus u 2 Similarly we can see v 1 is given by u 0 plus u 2 and v 2 is given by u 0 plus 41 00:06:23,370 --> 00:06:34,370 u 1 So let's just check let's say v 2 v 2 you can see is given by v 2 is given by u 42 00:06:34,370 --> 00:06:44,170 0 plus u 1 You can check v 2 is given by so u 0 plus u 1 0 plus 0 is 0 1 plus 0 is 1 0 43 00:06:44,170 --> 00:06:55,120 plus 1 is 1 1 plus 1 is 0 0 plus 0 is 0 1 plus 0 is 1 0 plus 1 is 1 and 1 plus 1 is 44 00:06:55,120 --> 00:07:09,290 0 Similarly we notice that v 3 v 4 v 5 are nothing but information bits u 0 u 1 and u 45 00:07:09,290 --> 00:07:17,860 2 respectively So let's go back v 3 is this column and we can see this is same as u 0 46 00:07:17,860 --> 00:07:31,419 0 1 0 1 0 1 0 1 similarly v 4 is equal to u 1 and v 5 is same as u 2 47 00:07:31,419 --> 00:07:44,419 So now we have written our coded bits in terms of our information bits This set of 6 equations 48 00:07:44,419 --> 00:07:53,020 I can write it in a matrix form So I can write my coded bits in terms of information bits 49 00:07:53,020 --> 00:08:02,150 and this matrix G which is our generator matrix will tell us how are we generating each of 50 00:08:02,150 --> 00:08:10,819 these coded bits as a linear combination of these information bits So if we compare each 51 00:08:10,819 --> 00:08:24,050 equation let’s look at v 0 So what is v 0 v 0 is u 0 g 0 0 plus u 1 g 1 0 plus u 2 52 00:08:24,050 --> 00:08:33,320 g 2 0 and what do we see here v 0 is u 1 plus u 2 So that means g 0 0 is 0 because there 53 00:08:33,320 --> 00:08:42,919 is no u 0 term here g 1 0 is 1 because there is a u 1 term here and g 2 0 is 1 because 54 00:08:42,919 --> 00:08:57,279 there is a u 2 term here So this will be 0 1 1 Similarly look at v 1 v 1 is u 0 g 0 1 55 00:08:57,279 --> 00:09:11,589 plus u 1 g 1 1 plus u 2 g 2 1 And if we compare it with v 1 here we see v 1 is u 0 plus u 56 00:09:11,589 --> 00:09:22,319 2 that means this g 0 1 should be 1 g 1 1 should be 0 and this should be 1 Likewise 57 00:09:22,319 --> 00:09:30,179 we build up the other columns of the matrix So if we do that what we get is something 58 00:09:30,179 --> 00:09:40,730 like this We can verify basically Let's take second last column second last column so what 59 00:09:40,730 --> 00:09:53,350 is v 4 v 4 is u 0 times g 0 4 plus u 1 times g 1 4 plus u 2 times g 2 4 and what is v 4 60 00:09:53,350 --> 00:09:59,949 v 4 is u 1 so then this should be 1 and this should be 0 and this should be 0 and this 61 00:09:59,949 --> 00:10:10,610 is what we have 0 1 0 So now we can basically find out the generator matrix so linear block 62 00:10:10,610 --> 00:10:19,329 code is completely described by its generator matrix And we said we can use the generator 63 00:10:19,329 --> 00:10:23,019 matrix to generate our codewords 64 00:10:23,019 --> 00:10:33,040 For example if my information sequence is 1 0 1 what should be my corresponding coded 65 00:10:33,040 --> 00:10:40,420 bits for the information sequence 1 0 1 How do I find that So as I know my output codeword 66 00:10:40,420 --> 00:10:48,279 is basically u times G so I will take linear combinations of rows of my generator matrix 67 00:10:48,279 --> 00:10:55,209 What are the rows of my generator matrix These are the 3 rows of my generator matrix So my 68 00:10:55,209 --> 00:11:03,420 coded bit corresponding to this information sequence would be 1 times G 0 plus 0 times 69 00:11:03,420 --> 00:11:14,399 G 1 plus 1 times G 2 So that's what I have written here 1 times G 0 0 times G 1 plus 70 00:11:14,399 --> 00:11:24,639 1 times G 2 So this is basically 0 So what I have is then this plus this right So let's 71 00:11:24,639 --> 00:11:35,269 look at 0 plus 1 would be 1 1 plus 1 would be 0 1 plus 0 is 1 1 plus 0 is 1 0 plus 0 72 00:11:35,269 --> 00:11:45,399 is 0 and 0 plus 1 is 1 So my codeword corresponding to this information message bits this information 73 00:11:45,399 --> 00:11:52,720 bit is given by 1 0 1 1 0 1 Ok 74 00:11:52,720 --> 00:12:04,230 Now what do we mean by a linear code in systematic form Now if we are able to among the coded 75 00:12:04,230 --> 00:12:12,369 bits if we are able to separate them out into if the message bits appear directly in the 76 00:12:12,369 --> 00:12:18,509 coded bit sequence then we can separate out the message bits from the parity bits For 77 00:12:18,509 --> 00:12:30,009 example go back to this example What do we have here We have 3 of these coded bits exactly 78 00:12:30,009 --> 00:12:36,949 same as information bits and the other 3 bits parity bits are linear combination of this 79 00:12:36,949 --> 00:12:44,959 message bits So from the output codeword we we can clearly separate out the information 80 00:12:44,959 --> 00:12:57,069 sequence which is in this case given by v 3 v 4 and v 5 So in this case v 1 v 2 v 3 81 00:12:57,069 --> 00:13:05,949 are these n minus k parity bits and v 3 v 4 v 5 are my information bits So in this particular 82 00:13:05,949 --> 00:13:14,529 example we can see that we are able to separate out information bits directly from the coded 83 00:13:14,529 --> 00:13:23,040 bits So in a systematic a block code in a systematic form we are able to separate out 84 00:13:23,040 --> 00:13:33,350 the information bit part from the coded bits So a generator matrix for a linear block code 85 00:13:33,350 --> 00:13:45,730 in systematic form will be of the form like this or it would be basically i times i k 86 00:13:45,730 --> 00:13:58,179 times some some either of this form Now why do we say that So only when we have our part 87 00:13:58,179 --> 00:14:03,519 of our generator matrix of the form of identity then what is going to happen 88 00:14:03,519 --> 00:14:08,350 When we multiply our information sequence with this sort of generator matrix you will 89 00:14:08,350 --> 00:14:16,639 see part of my coded bits will just depend on one particular information bit sequence 90 00:14:16,639 --> 00:14:25,970 So if I have write down the corresponding equations for coded sequence what you will 91 00:14:25,970 --> 00:14:36,509 see that some coded bits directly depend on the message bits and then rest are which are 92 00:14:36,509 --> 00:14:43,350 parity bits are linear combination of these message bits So in a systematic form basically 93 00:14:43,350 --> 00:14:51,980 we can separate out the message part from the parity bit part So as I said for a systematic 94 00:14:51,980 --> 00:15:01,819 linear block codes the message part will consist of the k information bits and the remaining 95 00:15:01,819 --> 00:15:10,149 n minus k bits which are the parity bits basically they will be linear combination of these message 96 00:15:10,149 --> 00:15:17,709 bits So we can write down the encoding equations for these matrices for these for systematic 97 00:15:17,709 --> 00:15:29,170 code so if you look at what is our encoding equation Our v is u times G where G is of 98 00:15:29,170 --> 00:15:43,359 form like this Ok So if we write u which is basically u 0 u 1 to u k minus 1 times this 99 00:15:43,359 --> 00:15:53,779 G matrix what we will get is a form like this So you will have n minus k parity equations 100 00:15:53,779 --> 00:16:00,100 parity check equations which are given by this expression and then you will have remaining 101 00:16:00,100 --> 00:16:10,679 k unaltered message bits So for a linear block code in a systematic form the encoding equations 102 00:16:10,679 --> 00:16:17,649 will be of this form And as I said since we are restricting ourselves without any loss 103 00:16:17,649 --> 00:16:25,339 of generality to binary codewords this addition is basically done modulo 2 104 00:16:25,339 --> 00:16:33,410 So what we have seen so far is we can describe a linear block code by its generator matrix 105 00:16:33,410 --> 00:16:40,910 which is a k cross n matrix and we can use this generator matrix to generate our set 106 00:16:40,910 --> 00:16:49,060 of codewords Now there is another matrix which we call parity check matrix which is related 107 00:16:49,060 --> 00:16:57,279 to our generator matrix we will show which can also be used to completely describe a 108 00:16:57,279 --> 00:17:09,689 linear block code So for a n k linear block code can be specified by a n minus k cross 109 00:17:09,689 --> 00:17:17,660 n parity check matrix which we denote by G H the generator matrix we denote by G and 110 00:17:17,660 --> 00:17:25,110 the parity check matrix we denote by H Now this parity check matrix has this property 111 00:17:25,110 --> 00:17:37,250 that if v is your valid codeword if and only if v H transpose is going to be 0 So if v 112 00:17:37,250 --> 00:17:45,399 is a valid codeword v H transpose will be 0 So let us see how we can derive our parity 113 00:17:45,399 --> 00:17:52,350 check matrix from a generator matrix and what's a relation of the generation matrix with the 114 00:17:52,350 --> 00:18:02,670 parity check matrix So we will take an example of a 7 4 systematic linear block code whose 115 00:18:02,670 --> 00:18:10,230 generator matrix is given by this So since this is a systematic code we can write it 116 00:18:10,230 --> 00:18:21,429 of the form p times this i k This generator matrix can be written of this form Ok Now 117 00:18:21,429 --> 00:18:29,779 from this generator equation we can write our coded bits in terms of our message bits 118 00:18:29,779 --> 00:18:38,970 So let's do that So the encoding equation this was our generator matrix G this is our 119 00:18:38,970 --> 00:18:46,289 information bits message bits u and this is our coded bits v So we can write v as u times 120 00:18:46,289 --> 00:18:57,200 G So it’s a 7 4 code So there are 4 information bits I denote them by u 0 u 1 u 2 u 3 And 121 00:18:57,200 --> 00:19:04,159 there are 7 coded bits I denote them by v 0 v 1 v 2 to v 6 And this generator matrix 122 00:19:04,159 --> 00:19:12,080 I have already given you this So I can write down the encoding equations We do that You 123 00:19:12,080 --> 00:19:20,720 can see from this what is v 0 It is u 0 plus u 2 plus u 3 124 00:19:20,720 --> 00:19:31,659 There is a typing mistake here This should have been u 2 So v 0 is u 0 times 1 plus 0 125 00:19:31,659 --> 00:19:44,140 times u 1 plus 1 times u 2 plus 1 times u 3 So this v 0 is given by u 0 plus u 2 plus 126 00:19:44,140 --> 00:19:58,360 u 3 Similarly what is v 1 v 1 is given by u 0 plus u 1 plus u 2 v 2 is given by u 1 127 00:19:58,360 --> 00:20:10,269 u 0 into 0 u 1 into 1 u 2 plus u 2 into 1 plus u 3 into 1 So v 2 is given by u 1 plus 128 00:20:10,269 --> 00:20:20,289 u 2 plus u 3 So that's what I have here What is v 3 v 3 is given by u 0 into 1 and rest 129 00:20:20,289 --> 00:20:38,169 are all 0s So v 3 is nothing but u 0 Similarly v 4 is u 1 v 5 is u 2 and v 6 is u 3 So I 130 00:20:38,169 --> 00:20:49,039 now have set of 7 coded bits and this shows the relation between the coded bits and information 131 00:20:49,039 --> 00:20:55,389 bits Now we are since we are restricting ourselves to binary codes we can even write this equation 132 00:20:55,389 --> 00:21:12,139 like this v 0 plus u 0 plus u 2 plus u 3 is equal to 0 correct Because this v 0 v 1 is 133 00:21:12,139 --> 00:21:17,490 nothing but parity bit which is basically nothing but like 1 or 0 So we add this to 134 00:21:17,490 --> 00:21:30,290 this Modulo 2 sum will be 0 So this similarly we can write as v 1 plus u 0 plus u 1 plus 135 00:21:30,290 --> 00:21:43,860 u 2 equal to 0 and this can be written as v 2 plus u 1 plus u 2 plus u 3 is equal to 136 00:21:43,860 --> 00:21:54,159 0 The next what we would try to do is we would try to write these parity check equations 137 00:21:54,159 --> 00:22:03,879 in terms of other coded bits So we can see here u 0 is nothing but v 3 So wherever u 138 00:22:03,879 --> 00:22:14,580 0 appears we can replace it by v 3 Similarly u 1 is equal to v 4 So wherever u 1 appears 139 00:22:14,580 --> 00:22:21,960 we can replace it by v 4 u 2 is equal to v 5 so we can replace u 2 by v 5 And u 3 is 140 00:22:21,960 --> 00:22:31,120 equal to v 6 We can replace u 3 in terms of v 6 By doing this what we will get is set 141 00:22:31,120 --> 00:22:40,590 of equations which basically are dependent on these coded bits If we do that what we 142 00:22:40,590 --> 00:22:51,789 get is something like this The first expression basically which was v 0 plus u 0 the u 2 plus 143 00:22:51,789 --> 00:22:52,879 u 3 144 00:22:52,879 --> 00:23:04,379 Now this can be re-written as v 0 plus what is u 0 u 0 is v 3 v 3 plus what is u 2 u 2 145 00:23:04,379 --> 00:23:16,680 is v 5 v 5 plus what is u 3 it is v 6 So v 0 plus v 3 plus v 5 plus v 6 is 0 and that 146 00:23:16,680 --> 00:23:28,529 is what we have here v 0 plus v 3 plus v 5 plus v 6 is equal to 0 Similarly we can write 147 00:23:28,529 --> 00:23:37,940 the other equations as well Here also we will replace u 0 u 1 u 2 by v 3 v 4 v 5 and what 148 00:23:37,940 --> 00:23:47,049 we will get is v 1 plus v 3 plus v 4 plus v 5 is equal to 0 and similarly the last parity 149 00:23:47,049 --> 00:23:57,950 check equation can be written as v 2 plus v 2 plus u 1 is v 4 u 2 is v 5 plus u 3 is 150 00:23:57,950 --> 00:24:06,480 v 6 So that's what we have here u 2 v 2 plus v 4 plus v 5 plus v 6 is equal to 0 So now 151 00:24:06,480 --> 00:24:14,019 we have set of encode equations in terms of coded bits Next the same thing we can write 152 00:24:14,019 --> 00:24:25,730 it in a matrix form So I have my coded bits v 0 to v 6 I have 3 sets of parity check equations 153 00:24:25,730 --> 00:24:33,240 this this and this And the same thing I can write it in a matrix form like this Now you 154 00:24:33,240 --> 00:24:40,380 can see these are equivalent So look let's look at first equation This is v 0 plus v 155 00:24:40,380 --> 00:24:49,299 3 plus v 5 plus v 6 is 0 You can see which are the elements which are so v 0 times 1 156 00:24:49,299 --> 00:24:58,629 this is v 3 times 1 plus v 5 times 1 plus v 6 times 1 So that's what is defined in this 157 00:24:58,629 --> 00:25:08,490 equation Similarly we can see this equation This v 1 plus v 3 plus v 4 plus v 5 is equal 158 00:25:08,490 --> 00:25:17,850 to 0 and this last equation this is v 2 plus v 4 plus v 5 plus v 6 is 0 And what we did 159 00:25:17,850 --> 00:25:22,149 we say about parity check matrix 160 00:25:22,149 --> 00:25:31,600 We said that if H is a parity check matrix it is n minus k cross matrix and it has this 161 00:25:31,600 --> 00:25:42,549 property that v H transpose is 0 So we have we can write this as this is my v this is 162 00:25:42,549 --> 00:25:51,419 my H transpose v H transpose is 0 so then what is my H matrix H matrix is a transpose 163 00:25:51,419 --> 00:26:11,269 of this matrix so this will be 1 0 0 0 1 0 0 0 1 1 1 0 0 1 1 1 1 1 and 1 0 1 This is 164 00:26:11,269 --> 00:26:21,999 my so for the 7 4 code 7 4 code this is basically 3 cross 7 As I said n minus k cross n matrix 165 00:26:21,999 --> 00:26:34,399 this is my parity check matrix corresponding to this same code which is generated by this 166 00:26:34,399 --> 00:26:39,010 Another interesting property which you can generally see is so v H transpose is 0 I can 167 00:26:39,010 --> 00:26:49,070 write this u times v H transpose is equal to 0 In other words v H transpose is 0 so 168 00:26:49,070 --> 00:26:59,350 what does that mean The rows of G matrix and rows of H matrix are orthogonal to each other 169 00:26:59,350 --> 00:27:08,950 So the H lies in the null space of G so as we can see from this that generator matrix 170 00:27:08,950 --> 00:27:18,340 and parity check matrix are related to each other And they have this property that rows 171 00:27:18,340 --> 00:27:26,470 of G matrix and H matrix are basically orthogonal to each other So if you have a systematic 172 00:27:26,470 --> 00:27:34,399 code whose generator matrix can be written in this form because H lies in the null space 173 00:27:34,399 --> 00:27:41,750 of G we can write down its corresponding H matrix very easily And this is basically given 174 00:27:41,750 --> 00:27:48,169 by so if a generator matrix can be written of the form P and identity matrix we can write 175 00:27:48,169 --> 00:27:55,019 its parity check matrix as identity matrix and P transpose 176 00:27:55,019 --> 00:28:01,350 So let's take an example of a generator matrix of a systematic code This is systematic code 177 00:28:01,350 --> 00:28:07,330 we can see we can separate out this generator matrix as some matrix P and some identity 178 00:28:07,330 --> 00:28:14,440 matrix So this we can write as H matrix we can write as identity matrix and P transpose 179 00:28:14,440 --> 00:28:24,159 So then this can be written as 1 1 0 is 1 1 0 0 1 1 0 1 1 1 1 1 1 1 1 0 1 0 so this 180 00:28:24,159 --> 00:28:33,299 is my h matrix corresponding to this So whether you are given a generator matrix or a parity 181 00:28:33,299 --> 00:28:40,460 check matrix your linear block code is completely specified by either of them And as I said 182 00:28:40,460 --> 00:28:46,119 we use the generator matrix to generate our code set of codewords where as parity check 183 00:28:46,119 --> 00:28:53,570 matrix as the name suggests is used to check whether the parity check constraints are satisfied 184 00:28:53,570 --> 00:29:00,190 As we said basically parity check matrix has this property that if v is a valid codeword 185 00:29:00,190 --> 00:29:07,020 if and only if v H transpose is zero and we use this property in decoding so that's why 186 00:29:07,020 --> 00:29:12,549 you see the name parity check matrix because this matrix H is essentially used to in some 187 00:29:12,549 --> 00:29:20,299 sense check whether the parity check constraints of the code are satisfied or not Thank you