1
00:00:25,260 --> 00:00:28,230
Today we shall begin with the lecture on the
subject of wavelets and multirate digital
2
00:00:28,230 --> 00:00:33,010
signal processing in which our objective would
be to introduce the Haar multiresolution analysis
3
00:00:33,010 --> 00:00:42,610
about which we had very briefly talked in
the previous lecture Before I go on to the
4
00:00:42,610 --> 00:00:49,489
analytical and mathematical details of the
Haar multiresolution analysis or MRA as it
5
00:00:49,489 --> 00:00:51,840
is called for short let me once again review
the idea behind the Haar form of analysis
6
00:00:51,840 --> 00:00:52,840
of functions
7
00:00:52,840 --> 00:01:00,690
Recall that Haar was a mathematician or mathematician-scientist
if you would like to call him that And the
8
00:01:00,690 --> 00:01:07,780
very radical idea that he gave was that one
could think of continuous functions in terms
9
00:01:07,780 --> 00:01:17,090
of discontinuous ones and do so to the limit
of reaching any degree of continuity that
10
00:01:17,090 --> 00:01:23,540
you desire What I mean is start from a very
discontinuous function and then make it smoother
11
00:01:23,540 --> 00:01:29,170
and smoother all the while adding discontinuous
functions until you go arbitrarily close to
12
00:01:29,170 --> 00:01:33,710
the continuous function that you are trying
to approximate
13
00:01:33,710 --> 00:01:43,030
This is the central idea in the Haar way of
representing functions We also briefly discussed
14
00:01:43,030 --> 00:01:49,180
by this was something important it seemed
like something silly to do at 1st glance but
15
00:01:49,180 --> 00:01:58,580
actually is very important And the reason
why it is important as we mentioned was if
16
00:01:58,580 --> 00:02:08,759
you think about digitally communicating say
for example an audio piece you are doing exactly
17
00:02:08,759 --> 00:02:19,560
that The beautiful smooth audio pattern is
being converted into a highly discontinuous
18
00:02:19,560 --> 00:02:22,890
stream of bits
19
00:02:22,890 --> 00:02:33,639
What I mean by discontinuous is when you transmit
the stream of bits on a communication channel
20
00:02:33,639 --> 00:02:45,420
you are in fact introducing discontinuities
every time a bit changes So after every bit
21
00:02:45,420 --> 00:02:56,489
interval there is a change of waveform and
therefore this continuity at some level even
22
00:02:56,489 --> 00:03:01,299
if not in the function in its derivative or
in its 2nd derivative whatever be Whatever
23
00:03:01,299 --> 00:03:10,180
it is the idea of representing continuous
functions in terms of discontinuous ones has
24
00:03:10,180 --> 00:03:15,340
its place in practical communication and therefore
what Haar did was something very useful to
25
00:03:15,340 --> 00:03:16,340
us today
26
00:03:16,340 --> 00:03:20,379
What we are going to do today is to build
up the idea of wavelets in fact more specifically
27
00:03:20,379 --> 00:03:29,159
what are called dyadic wavelets starting from
the Haar wavelet And to do that let us 1st
28
00:03:29,159 --> 00:03:35,529
consider how we represent a picture on a screen
and I am going to show that schematically
29
00:03:35,529 --> 00:03:39,000
in the drawing here
30
00:03:39,000 --> 00:03:48,819
So you see let us assume that this is the
picture boundary and I am trying to represent
31
00:03:48,819 --> 00:03:59,389
this picture on the screen whatever that picture
might be So just for the sake of drawing let
32
00:03:59,389 --> 00:04:11,230
me draw some kind of a pattern there let us
say you have a tree and some person standing
33
00:04:11,230 --> 00:04:18,579
there and forgive my drawing but whatever
it is maybe some grass may be here Now this
34
00:04:18,579 --> 00:04:29,600
is inherently a continuous picture How do
I represent it in the computer I divide this
35
00:04:29,600 --> 00:04:36,960
entire area into very small subareas so I
visualize this being divided into tiny what
36
00:04:36,960 --> 00:04:39,860
are called picture elements or pixels
37
00:04:39,860 --> 00:04:49,060
So each small area here is a pixel a picture
element so to speak And there are for example
38
00:04:49,060 --> 00:04:57,010
suppose I make 512 divisions on the vertical
and 512 divisions on the horizontal I say
39
00:04:57,010 --> 00:05:09,700
that I have a 512 cross 512 image that many
pixels and in each pixel region I represent
40
00:05:09,700 --> 00:05:11,080
the image by a constant
41
00:05:11,080 --> 00:05:16,750
So the 1st thing to understand is there is
a piecewise constant representation let us
42
00:05:16,750 --> 00:05:21,650
write that down there is a piecewise constant
representation of the image one constant for
43
00:05:21,650 --> 00:05:28,300
each piece and that piece is the pixel or
the picture element Now suppose I increase
44
00:05:28,300 --> 00:05:32,840
the resolution so I go from a resolution of
512 so I take the same what I mean is I take
45
00:05:32,840 --> 00:05:43,080
the same picture same picture In this case
I make a division 512 cross 512 in this case
46
00:05:43,080 --> 00:05:45,490
I make a division 1024 cross 1024
47
00:05:45,490 --> 00:05:53,560
Now obviously the pixel area here let us say
the pixel area here is P2 and the pixel area
48
00:05:53,560 --> 00:06:03,430
here is P1 it is very easy to see that P2
is one 4th of P1 and therefore I have reduced
49
00:06:03,430 --> 00:06:11,520
the area by factor of 4 Naturally if I use
a constant to represent the value or the you
50
00:06:11,520 --> 00:06:22,340
know the intensity of the picture on each
pixel here and do the same here what you see
51
00:06:22,340 --> 00:06:27,620
in this picture is going to be closer to the
original picture in some sense than what you
52
00:06:27,620 --> 00:06:28,620
see here
53
00:06:28,620 --> 00:06:34,240
So in other words we can capture this by saying
the smaller the pixel area the larger the
54
00:06:34,240 --> 00:06:43,270
resolution Now this is the beginning of the
Haar multiresolution analysis The more we
55
00:06:43,270 --> 00:07:00,370
reduce the pixel area the closer we are going
to go to the original image Even though this
56
00:07:00,370 --> 00:07:08,940
captures the idea that we are trying to build
it is not quite the idea of the Haar MRA The
57
00:07:08,940 --> 00:07:16,670
Haar MRA does something deeper and that is
what I am now going to explain mathematically
58
00:07:16,670 --> 00:07:24,670
in some depth Now here I gave the example
of a two-dimensional situation which apparently
59
00:07:24,670 --> 00:07:31,460
is more difficult than one-dimensional but
it is easier for us to understand physical
60
00:07:31,460 --> 00:07:37,920
We can more easily relate to the idea of a
piecewise constant representation in the context
61
00:07:37,920 --> 00:07:44,320
of images or pictures but the same thing could
be true of audio for example So you could
62
00:07:44,320 --> 00:07:53,150
visualize a situation though seemingly more
unnatural where you record an audio piece
63
00:07:53,150 --> 00:08:05,960
by dividing the time over which the audio
is recorded into small segments Now let me
64
00:08:05,960 --> 00:08:11,240
show that pictorially it would be easier to
understand
65
00:08:11,240 --> 00:08:16,761
So suppose for example you had this waveform
here the one-dimensional version So suppose
66
00:08:16,761 --> 00:08:24,130
I have this is the time axis and I have this
waveform here assuming that this is the audio
67
00:08:24,130 --> 00:08:30,650
waveform audio voltage recording let us without
any loss of generality assume that this is
68
00:08:30,650 --> 00:08:42,320
the 0 point in time so let Time be represented
by t and let this be the 0 point in time Now
69
00:08:42,320 --> 00:08:47,769
let me assume that I divide this time axis
into smaller intervals of size T here this
70
00:08:47,769 --> 00:08:50,209
point is T this point is 2T and so on
71
00:08:50,209 --> 00:08:54,500
I make a piecewise constant approximation
that means I represent the audio voltage in
72
00:08:54,500 --> 00:09:03,020
each of these regions of size T by one number
Now what is the most obvious number or what
73
00:09:03,020 --> 00:09:10,140
are the set of most obvious numbers that one
can use to represent this waveform in each
74
00:09:10,140 --> 00:09:19,400
of these time intervals For example in this
time interval or for that matter in any of
75
00:09:19,400 --> 00:09:29,480
the time intervals it makes sense to take
the area under the curve and divide by the
76
00:09:29,480 --> 00:09:37,910
time interval to get the average of the waveform
in that time interval and use that as a number
77
00:09:37,910 --> 00:09:38,910
to represent the function
78
00:09:38,910 --> 00:09:43,580
Here for example you can visualize that the
average will lie somewhere here I am just
79
00:09:43,580 --> 00:09:51,960
showing it in dotted so average So intuitively
it makes sense to represent the voltage waveform
80
00:09:51,960 --> 00:10:01,470
in each of these intervals of size T by the
average of that waveform in that interval
81
00:10:01,470 --> 00:10:07,710
is that right Let us write that down mathematically
82
00:10:07,710 --> 00:10:17,180
So what we are saying if you have a function
x of time a good piecewise constant representation
83
00:10:17,180 --> 00:10:25,220
is the following Over the interval of T over
the interval from say 0 to T now you know
84
00:10:25,220 --> 00:10:38,080
strictly it is the open interval between 0
and T the representation would be integrate
85
00:10:38,080 --> 00:10:48,955
x of t dt from 0 to T and divide by T the
average Now of course on any particular integral
86
00:10:48,955 --> 00:10:54,264
of T the same holds So we say that on every
interval of T on any particular interval of
87
00:10:54,264 --> 00:10:57,970
T of size T the average would be obtained
by 1 by T integral over that interval of T
88
00:10:57,970 --> 00:11:04,780
when you write it like this you mean that
particular interval of T
89
00:11:04,780 --> 00:11:11,279
Integral of x t with respect to small t This
is a piecewise constant representation of
90
00:11:11,279 --> 00:11:18,589
the function on that interval of size T Now
the same thing could be done for an interval
91
00:11:18,589 --> 00:11:27,180
of size T by 2 So over an interval of size
T by 2 you would similarly have 1 by T by
92
00:11:27,180 --> 00:11:37,180
2 integral over that interval of length T
by 2 x t dt Now we are going closer to the
93
00:11:37,180 --> 00:11:47,529
idea of wavelets Let us pick a particular
interval of size T in fact again without any
94
00:11:47,529 --> 00:11:52,990
loss of generality let us choose the interval
from 0 to T and divide it into 2 sub intervals
95
00:11:52,990 --> 00:11:53,990
of size T by 2
96
00:11:53,990 --> 00:11:59,529
So what I mean is take this interval of size
T 0 to T I am expanding it so you have this
97
00:11:59,529 --> 00:12:05,310
function here over that interval divide this
into 2 sub intervals of size T by 2 1st take
98
00:12:05,310 --> 00:12:10,680
the piecewise constant approximation on the
entire interval of T and I show that by a
99
00:12:10,680 --> 00:12:15,760
dot and dash line You can visualize the average
will be somewhere here So this is the average
100
00:12:15,760 --> 00:12:26,060
on the entire interval 0 to T Now I take the
sub intervals of size T by 2 so I have this
101
00:12:26,060 --> 00:12:30,300
sub interval of size T by 2 I use a dash and
cross to write down the average there so I
102
00:12:30,300 --> 00:12:33,390
have dash and cross dash and cross here
103
00:12:33,390 --> 00:12:41,580
You can visualize that in this sub interval
the average will be somewhere here And similarly
104
00:12:41,580 --> 00:12:48,279
in this sub interval you could write down
the average something like this Now let us
105
00:12:48,279 --> 00:12:56,220
keep this a main let us call this average
A on T let us call this average A1 on T by
106
00:12:56,220 --> 00:13:03,050
2 and let us call this average A2 on the interval
of size T by 2 and let us write down the expressions
107
00:13:03,050 --> 00:13:04,630
for each of these averages What are the expressions
108
00:13:04,630 --> 00:13:08,790
AT is obviously 1 by T integral x of t dt
from 0 to T A1 T by 2 is 1 by T by 2 integral
109
00:13:08,790 --> 00:13:17,890
from 0 to T by 2 x t dt And similarly A2 T
by 2 is 1 by T by 2 integral from T by 2 to
110
00:13:17,890 --> 00:13:27,740
T x t dt For convenience let me flash all
the 3 expressions before you once again AT
111
00:13:27,740 --> 00:13:32,550
is the average over the entire interval of
T A1 T by 2 the average over the 1st interval
112
00:13:32,550 --> 00:13:38,220
of T Y2 with this expression and A2 T by 2
the average from T by 2 to T the 2nd sub interval
113
00:13:38,220 --> 00:13:39,220
of size T by 2 with this expression
114
00:13:39,220 --> 00:13:45,380
And just to get our ideas straight here again
is the picture Now the key idea in the Haar
115
00:13:45,380 --> 00:13:56,070
multiresolution analysis is to try and relate
these 3 terms So to relate AT A1 T by 2 A2
116
00:13:56,070 --> 00:14:00,680
T by 2 and it is in that relationship that
the Haar wavelet is hidden So what is the
117
00:14:00,680 --> 00:14:01,680
relationship Now the relationship is very
simple I mean all that we need to do is to
118
00:14:01,680 --> 00:14:09,120
notice that we have divided the integral from
0 to T into 2 integrals over 0 to T by 2 and
119
00:14:09,120 --> 00:14:17,220
T by 2 to T and then remember there is a slight
difference in the constant associated
120
00:14:17,220 --> 00:14:24,260
So we have a constant of 1 by T in AT and
a concept of 1 by T by 2 in A1 T by 2 and
121
00:14:24,260 --> 00:14:29,450
in A2 T by 2 whereupon we have this very simple
relationship between the 3 quantities AT is
122
00:14:29,450 --> 00:14:41,380
half I leave it to you to verify it is half
of A1 T by 2 plus A2 T by 2 And how do we
123
00:14:41,380 --> 00:14:47,390
interpret this let me try and you know kind
of focus just on this relationship in other
124
00:14:47,390 --> 00:14:54,950
words let us just focus on these 3 constants
and make a drawing there So what we are saying
125
00:14:54,950 --> 00:15:05,020
is something like this I have this AT there
I have this A1 T by 2 here and I have this
126
00:15:05,020 --> 00:15:06,690
A2 T by 2 there
127
00:15:06,690 --> 00:15:08,180
And we are saying this plus this by 2 gives
you this in other words this is as much higher
128
00:15:08,180 --> 00:15:12,370
above AT as this is lower what we are saying
is these 2 heights are the same that is what
129
00:15:12,370 --> 00:15:19,800
this relationship implies Now another way
of saying it is if I were to make a piecewise
130
00:15:19,800 --> 00:15:23,260
constant approximation on intervals of size
T how would they look So let me just sketch
131
00:15:23,260 --> 00:15:24,260
them
132
00:15:24,260 --> 00:15:25,260
So I take this function once again here I
have this function here I have divide it into
133
00:15:25,260 --> 00:15:27,761
sizes of intervals of size T let me show just
2 intervals for the moment So this is how
134
00:15:27,761 --> 00:15:32,760
the function would look when you would make
a piecewise constant approximation on intervals
135
00:15:32,760 --> 00:15:38,910
of size T and when you do it on intervals
of size T by 2 it would look like this something
136
00:15:38,910 --> 00:15:46,730
like this Now this is a function so let me
highlight it now let me darken it This is
137
00:15:46,730 --> 00:15:52,550
in its own right a function piecewise constant
function the one which I have darkened here
138
00:15:52,550 --> 00:15:54,029
And this is in its own right the darkened
part is in is in its own right an approximation
139
00:15:54,029 --> 00:16:00,670
to the original function here Similarly let
me now darken this and put some other mark
140
00:16:00,670 --> 00:16:05,960
on it let us keep the crosses so I will darken
this and I will put crosses on it this is
141
00:16:05,960 --> 00:16:06,960
also another function So dark and cross function
is another function that is in its own right
142
00:16:06,960 --> 00:16:09,050
an approximation too So let us give them names
143
00:16:09,050 --> 00:16:14,709
Let us call this function just the dark one
as f1 t and let us call this function the
144
00:16:14,709 --> 00:16:19,840
one which we have shown with the dark and
cross as f2 t f2 t minus f1 t is like additional
145
00:16:19,840 --> 00:16:24,830
information what we are saying is instead
of a piecewise constant approximation on interval
146
00:16:24,830 --> 00:16:30,149
of size T when we try and make a piecewise
constant approximation on intervals of size
147
00:16:30,149 --> 00:16:37,120
T by 2 we are bringing in something more Go
back to the original case of the picture we
148
00:16:37,120 --> 00:16:40,050
have inherently underlying a continuous two-dimensional
picture a continuous two-dimensional scene
149
00:16:40,050 --> 00:16:42,700
When we make an approximation with a 512 cross
512 resolution then we have actually brought
150
00:16:42,700 --> 00:16:47,120
in one level of detail when we go to 1024
cross 1024 representation the level of detail
151
00:16:47,120 --> 00:16:55,700
is 4 times more What is the additional detail
that we have got in going from 512 cross 512
152
00:16:55,700 --> 00:17:05,089
to 1024 cross 1024 In effect when we take
this difference f2 t minus f1 t we are answering
153
00:17:05,089 --> 00:17:06,079
that question