1
00:01:08,689 --> 00:01:15,590
Today we start our 6th lecture. This is the
2nd lecture on Informed Search. In the last
2
00:01:15,590 --> 00:01:22,229
class we looked at the different properties
of A star. Today we will continue with A star
3
00:01:22,229 --> 00:01:29,229
and talk about another algorithm called IDA
star.
4
00:01:29,409 --> 00:01:36,409
The instructional objectives of today's lecture
are the following:
5
00:01:36,889 --> 00:01:42,959
In this lecture the student will learn the
algorithm IDA star which is an algorithm for
6
00:01:42,959 --> 00:01:49,959
Informed Search. They will be able to analyze
the space and time complexity of IDA star
7
00:01:51,239 --> 00:01:57,409
and they should be able to understand the
difference between IDA star and the algorithm
8
00:01:57,409 --> 00:02:04,409
A star. The student will also get an introduction
to some of the restricted memory search algorithms
9
00:02:04,619 --> 00:02:11,619
like RBFS that is recursive best first search
as well as MA star. We will also discuss briefly,
10
00:02:18,069 --> 00:02:24,840
some local search algorithms like hill climbing
as also simulated annealing. The students
11
00:02:24,840 --> 00:02:29,930
will get a brief idea about these algorithms.
They will also have an understanding about
12
00:02:29,930 --> 00:02:32,900
the properties of search algorithms.
13
00:02:32,900 --> 00:02:39,900
Let us just quickly go over A star search.
So A star is a best first search where we
14
00:02:45,160 --> 00:02:52,160
use the function f(n) is equal to g(n) plus
h(n) to order the nodes for expansion. The
15
00:02:56,370 --> 00:03:03,370
nodes are expanded according to their increasing
f values. We have also seen that in case h(n)
16
00:03:08,389 --> 00:03:15,389
is admissible then search will find the optimal
solution. A heuristic function h(n) is said
17
00:03:16,780 --> 00:03:23,590
to be admissible if it underestimates the
cost of any solution which can be reached
18
00:03:23,590 --> 00:03:24,190
from the node.
19
00:03:24,190 --> 00:03:31,190
So, to briefly recapitulate algorithm A star
we have two lists OPEN and CLOSED.
20
00:03:36,949 --> 00:03:43,859
If we want to deal with a state space containing
repeated states and we want to ensure that
21
00:03:43,859 --> 00:03:50,859
a particular state does not get expanded multiple
times we need to keep along with the OPEN
22
00:03:52,000 --> 00:03:57,650
list all other lists as CLOSED which consists
of nodes which have already been expanded.
23
00:03:57,650 --> 00:04:02,250
OPEN is the list of nodes on the frontier
of the search tree.
24
00:04:02,250 --> 00:04:08,780
Initially CLOSED is empty and OPEN has a single
node the start node and the path associated
25
00:04:08,780 --> 00:04:15,780
with the node is an empty path. The algorithm
A star executes this loop. While OPEN is not
26
00:04:17,099 --> 00:04:24,099
empty certain things are done. So in this
loop that we will see in the next slide we
27
00:04:25,750 --> 00:04:31,770
try to see if we can find the path from the
start node to a goal node. If we do not find
28
00:04:31,770 --> 00:04:38,490
such a path in this loop and we have no more
lists no more nodes to expand. That is, OPEN
29
00:04:38,490 --> 00:04:45,490
becomes empty, the algorithm should return
failure. Now this is what we do in the body
30
00:04:47,819 --> 00:04:49,949
of this loop.
31
00:04:49,949 --> 00:04:56,949
We remove from OPEN the node n, p whose f(n)
value is minimum. So n is the node, p is the
32
00:05:02,410 --> 00:05:08,930
path associated with the node, n is that node
whose f(n) value is minimum, we put n, p on
33
00:05:08,930 --> 00:05:15,009
CLOSED because it has been expanded. If (n)
is a goal node we return success and we return
34
00:05:15,009 --> 00:05:22,009
the path b which is the path associated with
this loop. Otherwise we find all edges coming
35
00:05:24,470 --> 00:05:29,830
out of the node n that is edges connecting
node and with some other node m.
36
00:05:29,830 --> 00:05:36,830
Suppose there is edge e connecting n with
m whose cost is c. Now we consider this state
37
00:05:39,130 --> 00:05:46,130
m. If m is already on CLOSED with the path
q and our current path p concat e happens
38
00:05:51,030 --> 00:05:58,030
to be cheaper than the cost of the path q
then we will like to consider this current
39
00:06:00,240 --> 00:06:03,120
path as the best path here.
40
00:06:03,120 --> 00:06:10,120
If q is cheaper we throw away our recorder
path. But if m, q is on CLOSED and the current
41
00:06:10,470 --> 00:06:17,470
path is cheaper than q we remove the node
n from CLOSED and put the new node the current
42
00:06:18,840 --> 00:06:25,840
node whose state is m whose path is p concat
e we put this node at OPEN. Otherwise if it
43
00:06:28,960 --> 00:06:35,960
is the case that a node m is on OPEN already
with a different path q and the current path
44
00:06:39,210 --> 00:06:46,210
cost is cheaper than the cost of q then we
replace q with the current path p concat e.
45
00:06:47,570 --> 00:06:54,570
Otherwise if m is not on OPEN and not on CLOSED
we put m, p, e on OPEN. So this algorithm
46
00:07:03,850 --> 00:07:09,979
is expanded in this section.
47
00:07:09,979 --> 00:07:16,220
In the last class we have done an analysis
of the algorithm A star and we have seen that
48
00:07:16,220 --> 00:07:23,220
A staris an optimum algorithm. That is, if
there is a path from the start state to the
49
00:07:24,139 --> 00:07:31,139
goal state then A star is guaranteed to find
the optimum path and optimum path to the goal
50
00:07:35,560 --> 00:07:42,560
node. We have also seen that all algorithms
that use the same heuristic function h(n)
51
00:07:45,040 --> 00:07:52,040
A star is an optimally efficient algorithm.
A star is also complete. That is, it finds
52
00:07:54,009 --> 00:08:01,009
a path if one exists. And the number of nodes
searched by A star is exponential in the worst
53
00:08:06,240 --> 00:08:08,259
case.
54
00:08:08,259 --> 00:08:15,259
So, if we model the search space and assume
that the search space is a uniform b r e tree
55
00:08:24,449 --> 00:08:31,449
and there is a unique start state s and one
goal state g whose distance from the start
56
00:08:31,780 --> 00:08:38,780
state s is N and every arc has a cost of 1
then we can show that the number of nodes
57
00:08:42,079 --> 00:08:49,079
expanded A star is exponential in n unless
the heuristic estimate is logarithmically
58
00:08:50,110 --> 00:08:57,110
accurate. That is, suppose A star n is the
actual cost from n to the goal, the optimum
59
00:09:01,860 --> 00:09:08,860
cost of n to the goal and suppose h(n) is
our estimate. If h(n) minus h star n is less
60
00:09:11,819 --> 00:09:18,819
than or of the order of log of h star n then
the heuristic is said to be logarithmically
61
00:09:22,100 --> 00:09:29,100
accurate. And in this case A star has a polynomial
complexity on such a search tree.
62
00:09:33,339 --> 00:09:40,339
However, in most cases we can say utmost that
practically the heuristic functions we can
63
00:09:41,190 --> 00:09:48,190
get are utmost proportional to A star m. The
error is proportional. Error is not logarithmic
64
00:09:49,300 --> 00:09:56,300
and in those cases A star has exponential
complexity. So A star has exponential complexity.
65
00:10:08,870 --> 00:10:15,870
As we have seen A star tries to improve upon
general breadth first search by using this
66
00:10:16,910 --> 00:10:20,779
heuristic function to restrict the number
of nodes being expanded.
67
00:10:20,779 --> 00:10:27,310
However, this restriction is not good enough
because the complexity of A star is still
68
00:10:27,310 --> 00:10:33,070
exponential. If we get a better heuristic
function then the number of nodes in expansion
69
00:10:33,070 --> 00:10:40,070
can become less. So A star still can take
a lot of time on a reasonably large problem.
70
00:10:43,670 --> 00:10:50,670
But even worse than the time complexity of
A star is the fact that it becomes difficult
71
00:10:53,390 --> 00:11:00,390
to use A star even before the time runs out
because the size of the OPEN queue becomes
72
00:11:00,709 --> 00:11:07,700
too large because the space complexity of
A star is the space required for storing the
73
00:11:07,700 --> 00:11:11,850
OPEN queue. And in those cases where we use
the CLOSED queue we also have to store the
74
00:11:11,850 --> 00:11:14,140
CLOSED.
75
00:11:14,140 --> 00:11:21,140
Now the solution so that A star does not have
this bottleneck or such algorithm does not
76
00:11:21,170 --> 00:11:28,170
have this bottleneck is to have a search algorithm
which uses less space. Now we will discuss
77
00:11:28,820 --> 00:11:35,820
an algorithm IDA star which is Iterative Deepening
algorithm. And IDA star requires less space.
78
00:11:39,570 --> 00:11:46,570
We have already talked about Iterative-Deepening
search which is basically search which does
79
00:11:47,200 --> 00:11:54,200
a level by level expansion of the search tree.
And we have seen that Iterative-Deepening
80
00:11:54,310 --> 00:12:01,310
depth first search has a space complexity
of order b, d where b is the branching factor
81
00:12:01,550 --> 00:12:04,110
and d is the depth of the tree.
82
00:12:04,110 --> 00:12:11,110
Here d is utmost the N which is the cost of
the solution. So, if we use Iterative Deepening
83
00:12:11,610 --> 00:12:18,610
search we can have an algorithm which works
in linear space. However, Iterative Deepening
84
00:12:20,180 --> 00:12:24,800
A star is actually slightly different from
Iterative-Deepening search. It is actually
85
00:12:24,800 --> 00:12:31,800
very similar where the nodes to be expanded
in a particular iteration depend of the f
86
00:12:31,950 --> 00:12:38,950
value of the nodes and not on the level of
the nodes. So, in Iterative-Deepening A star
87
00:12:40,190 --> 00:12:47,190
the cutoff for nodes expanded in iteration
is decided by the f value of the nodes. Therefore
88
00:12:47,769 --> 00:12:54,769
we fix a limit of the f value and in an iteration
we only expand those nodes in a depth first
89
00:12:55,529 --> 00:13:02,529
manner whose f value is less than or equal
to this f limit. In the next iteration we
90
00:13:03,779 --> 00:13:10,779
increase the value of this limit. Now let
us look at Iterative-Deepening A star.
91
00:13:23,579 --> 00:13:30,579
In Iterative-Deepening A star is a depth first
Iterative-Deepening search but the depth bound
92
00:13:31,970 --> 00:13:38,970
used is a f limit. So we start with the initial
limit as the f value of start. We use a f
93
00:13:47,560 --> 00:13:54,560
value of start as the initial cutoff or initial
limit. Then with that limit we do a depth
94
00:13:57,630 --> 00:14:04,630
first search. And in doing this depth first
search we prune any node if f of that node
95
00:14:05,279 --> 00:14:12,279
is greater than the value of f limit. So,
after the current iteration the next f limit
96
00:14:15,560 --> 00:14:20,380
is chosen to be the minimum cost of any node
that was proved.
97
00:14:20,380 --> 00:14:27,380
Now let us try to illustrate Iterative-Deepening
A star on the following graph. We have six
98
00:14:30,790 --> 00:14:37,790
nodes a b c d e f. Initially we fix a f value
equal to the f value of the start node. Suppose
99
00:14:43,269 --> 00:14:50,269
the start node is a and suppose the f value
of the start node is 15 so using the f limit
100
00:14:51,930 --> 00:14:58,930
as 15 we do a depth first search in which
a, b and e are expanded. Now c and f were
101
00:15:02,649 --> 00:15:09,649
the nodes which were pruned in this first
iteration and suppose out of f value of f
102
00:15:14,459 --> 00:15:21,430
and f value of c the f value of c was minimum
and it was 21 And at the next iteration we
103
00:15:21,430 --> 00:15:28,430
will use the f value of 21 and we will do
a depth first search with this f limit equal
104
00:15:32,410 --> 00:15:39,410
to 21. And in the next iteration c was the
node pruned with a particular f value.
105
00:15:40,850 --> 00:15:47,850
We will set that equal to the f limit and
do the next iteration and so on. So this is
106
00:15:48,490 --> 00:15:54,130
the schemata of Iterative-Deepening A star.
Basically it is Iterative Deepening search
107
00:15:54,130 --> 00:16:01,130
where search is pruned when the f value of
a node exceeds the current f limit. And the
108
00:16:01,459 --> 00:16:08,459
minimum f value of a node pruned is used as
the next value of f limit. Now let us look
109
00:16:09,000 --> 00:16:12,550
at the properties of the algorithm IDA star.
110
00:16:12,550 --> 00:16:19,550
IDA star is complete and it is optimal. It
is complete because it will find a solution
111
00:16:20,540 --> 00:16:26,880
if one exists. It is optimal because the first
time it expands a goal node it would have
112
00:16:26,880 --> 00:16:33,880
found an optimum solution. The space usage
is proportional to the depth of the solution.
113
00:16:35,620 --> 00:16:42,130
Each iteration of IDA star is a depth first
search so it requires only space equal to
114
00:16:42,130 --> 00:16:48,449
b times d, the branching factor times the
depth of the tree so space requirement is
115
00:16:48,449 --> 00:16:50,570
linear.
116
00:16:50,570 --> 00:16:57,570
Now you note what the extra nodes are which
are expanded in every iteration. They are
117
00:16:59,850 --> 00:17:06,850
those nodes whose f value is immediately just
greater than the f value of the nodes in the
118
00:17:08,100 --> 00:17:15,100
previous iteration. So the number of nodes
expanded in subsequent iterations is the nodes
119
00:17:16,010 --> 00:17:23,010
with increasing f values. Hence IDA star is
an optimal algorithm. It is obvious that IDA
120
00:17:29,490 --> 00:17:36,269
star expands more nodes than A star because
certain nodes are expanded more than once.
121
00:17:36,269 --> 00:17:42,909
How many more nodes IDA star expand compared
to A star?
122
00:17:42,909 --> 00:17:49,909
That depends on the number of unique values
of f(n). So, if it is the case that between
123
00:17:50,389 --> 00:17:57,389
two iterations the extra number of nodes expanded
is large then IDA star is fairly efficient
124
00:17:59,010 --> 00:18:06,010
in the number of nodes expanded. But consider
the case when every node has a unique f value.
125
00:18:08,090 --> 00:18:14,820
In that case at every iteration only one new
node will be expanded and in this case IDA
126
00:18:14,820 --> 00:18:21,820
star will expand the square of the number
of nodes in A star. So space required by IDA
127
00:18:24,299 --> 00:18:29,429
star is O(bd), number of nodes expanded relative
to A star.
128
00:18:29,429 --> 00:18:36,429
In 8 puzzle there are few values of f and
the number of nodes expanded by IDA star is
129
00:18:38,679 --> 00:18:45,679
very close to the number of nodes A star expands.
Suppose in 8 puzzle we use a Manhattan distance
130
00:18:48,649 --> 00:18:53,970
or the number of misplaced tiles as the heuristic
function the number of such values such a
131
00:18:53,970 --> 00:19:00,820
heuristic can take is very small, the number
of misplaced tiles can be utmost 8. So the
132
00:19:00,820 --> 00:19:07,690
total f value is actually also not very large.
As a result the number of unique m values
133
00:19:07,690 --> 00:19:14,690
is small. So IDA star tends to expand a large
number of extra nodes in every iteration.
134
00:19:19,139 --> 00:19:25,499
And the number of iterations is equal to the
number of distinct f values.
135
00:19:25,499 --> 00:19:32,499
However, consider the traveling salesperson
problem where we have a graph of n cities
136
00:19:36,299 --> 00:19:40,739
and the objective of the graph in salesperson
is to start from a city traverse all the cities
137
00:19:40,739 --> 00:19:47,200
exactly once and come back to the starting
city. Imagine that every distance between
138
00:19:47,200 --> 00:19:54,200
a pair of cities is a different real value.
In this case it is possible that each f value
139
00:19:54,869 --> 00:20:01,529
is unique. So IDA star will expand one new
node in every iteration and the number of
140
00:20:01,529 --> 00:20:07,330
nodes expanded would be the first iteration
1 second iteration 2 third iteration 3 and
141
00:20:07,330 --> 00:20:13,639
in the last iteration the number of nodes
expanded will be equal to n. So the total
142
00:20:13,639 --> 00:20:20,639
number of nodes expanded equal to O(n2) where
n is the number of nodes A star expands. So
143
00:20:23,710 --> 00:20:30,710
we see that IDA star can expand utmost a quadratic
number of nodes expanded corresponding to
144
00:20:32,659 --> 00:20:34,529
A star.
145
00:20:34,529 --> 00:20:41,529
However, in other situations like 8 puzzle
IDA star would expand only a constant factor
146
00:20:46,919 --> 00:20:53,919
more nodes than A star would expand. Also,
we see that because in IDA star basically
147
00:20:55,399 --> 00:21:02,399
we are using a depth first search we cannot
avoid repeated node expansion so IDA star
148
00:21:03,559 --> 00:21:06,690
does expand duplicate nodes in cyclic graphs.
149
00:21:06,690 --> 00:21:13,690
If a graph contains cycles IDA star will not
be able to detect the cycles and it will expand
150
00:21:16,690 --> 00:21:23,690
those nodes repeatedly. But the main advantage
of ID star is its linear memory. For example,
151
00:21:24,619 --> 00:21:31,619
15 puzzle is a problem similar to eight puzzle
except that the problem is on a 4 into 4 grid,
152
00:21:33,720 --> 00:21:40,720
there is one empty square. So if you run A
star on 15 puzzle it will take a huge amount
153
00:21:41,889 --> 00:21:48,889
of time to solve and a huge amount of memory.
And 24 puzzle possibly you cannot solve in
154
00:21:49,619 --> 00:21:56,619
A star using normal resources because of the
huge space requirement and time. However,
155
00:21:59,499 --> 00:22:06,259
you can use IDA star which does not need to
keep this OPEN queue which will be able to
156
00:22:06,259 --> 00:22:13,259
solve 8 puzzle and 15 puzzle. IDA star uses
very little memory, it only uses linear memory
157
00:22:18,849 --> 00:22:25,849
and it may expand up to n square nodes if
A star expands n nodes.
158
00:22:28,690 --> 00:22:35,690
The idea of memory limited heuristic search
algorithms is to use the available memory
159
00:22:35,729 --> 00:22:42,729
optimally so as to have a more efficient algorithm
which reduces node expansion. Several such
160
00:22:45,460 --> 00:22:52,460
algorithms have been proposed and we will
briefly mention these algorithms.
161
00:22:54,369 --> 00:23:01,369
Recursive Breadth First Search or RBFS is
an algorithm actually uses only linear space
162
00:23:04,190 --> 00:23:11,190
but it is often better than IDA star because
it mimics best first search. RBFS keeps track
163
00:23:17,379 --> 00:23:24,379
of the f value of the best alternative path
available from any ancestor of the current
164
00:23:25,450 --> 00:23:32,450
node. So whenever RBFS is a linear memory
algorithm it does not keep track of all the
165
00:23:36,849 --> 00:23:43,849
nodes but those nodes that it has expanded
and the algorithm chooses to forget. The algorithm
166
00:23:45,499 --> 00:23:52,499
backs up at a node the f value of the best
successor of that node. So, at a particular
167
00:24:03,419 --> 00:24:10,419
node we have current f value and we also know
what is the second best f value based on what
168
00:24:11,590 --> 00:24:18,590
we had expanded earlier. When the current
f value this alternate f value then RBFS will
169
00:24:26,359 --> 00:24:33,359
explore this alternate path. Let us now see
another algorithm MA star and its related
170
00:24:35,970 --> 00:24:38,080
algorithm SMA star.
171
00:24:38,080 --> 00:24:43,609
These are restricted memory best first search
algorithms that utilize all the available
172
00:24:43,609 --> 00:24:50,609
memory. The algorithms execute best first
search when memory is available. It does normal
173
00:24:53,570 --> 00:25:00,570
best first search. When the memory becomes
full the worst node is dropped and the value
174
00:25:01,999 --> 00:25:08,369
of the forgotten node is backed up to the
parent. When the node is dropped the value
175
00:25:08,369 --> 00:25:15,369
of this backed up node is backed up to the
parent.
176
00:25:24,859 --> 00:25:31,109
We have seen systematic search algorithms:
depth first search, breadth first search,
177
00:25:31,109 --> 00:25:38,109
A star, IDA star, bidirectional search and
so on. And we have seen that A star and IDA
178
00:25:39,349 --> 00:25:46,349
star and such algorithms or even RBFS, MS
star use some heuristic function to restrict
179
00:25:47,450 --> 00:25:54,450
the number of nodes that are expanded. And
the most natural types of heuristic functions
180
00:25:57,619 --> 00:26:04,619
we can obtain is that these search algorithms
have exponential time complexity. There are
181
00:26:05,899 --> 00:26:12,899
situations where it is not possible to use
such algorithms to get the best solution to
182
00:26:15,119 --> 00:26:17,489
a problem.
183
00:26:17,489 --> 00:26:24,489
Local search methods are used in a class of
search algorithms where we need not have an
184
00:26:31,389 --> 00:26:37,309
optimum solution. There are problems where
we need a solution and we can get a solution
185
00:26:37,309 --> 00:26:44,309
which may not be the optimum solution but
we can get the solution in limited time. Local
186
00:26:45,279 --> 00:26:52,279
search methods work on formulations where
in every configuration we have a complete
187
00:26:54,599 --> 00:27:01,599
state. Therefore local search does a search
on complete state formulations.
188
00:27:03,919 --> 00:27:10,919
In local search typically constant number
of nodes is kept in memory and these nodes
189
00:27:11,450 --> 00:27:17,349
are all configurations of the entire state.
These states are perturbed to get the next
190
00:27:17,349 --> 00:27:24,349
state. Local search is useful for two types
of problems. One it is useful for solving
191
00:27:25,119 --> 00:27:32,119
optimization problems in those optimization
problems where it is often easy to find a
192
00:27:32,219 --> 00:27:36,409
solution but hard to find the best solution.
193
00:27:36,409 --> 00:27:43,409
Consider the traveling salesperson problem
in a fully connected graph. It is easy to
194
00:27:45,450 --> 00:27:52,450
get one solution to the problem because any
permutation of the city is a solution but
195
00:27:55,059 --> 00:28:00,820
it is difficult to get the best solution because
there are n factorial such permutations and
196
00:28:00,820 --> 00:28:07,820
we do not know one efficient way of finding
the best permutation in polynomial time. So,
197
00:28:07,929 --> 00:28:14,139
in TSP the objective is to find the optimum
configuration which is difficult to find.
198
00:28:14,139 --> 00:28:20,539
We do not know to find a good algorithm but
if we just want to find a solution then it
199
00:28:20,539 --> 00:28:26,049
is easy.
200
00:28:26,049 --> 00:28:33,049
Many local search algorithms use iterative
improvement and we can try to get a solution
201
00:28:34,940 --> 00:28:41,940
and get improvements to that solution. Secondly,
for these optimization problems like TSP we
202
00:28:46,169 --> 00:28:52,469
do not have to keep track of the path to the
solution. When we have the solution we immediately
203
00:28:52,469 --> 00:28:59,460
know the path. In TSP we do not have to keep
track of the path. Local search is ideal for
204
00:28:59,460 --> 00:29:06,460
such cases. In problems like 8 puzzle the
final state does not give us any information
205
00:29:09,669 --> 00:29:15,710
about the path. The solution path has to be
obtained. Local search is not very good for
206
00:29:15,710 --> 00:29:20,769
such problems. But for problems like TSP where
the path is not important local search is
207
00:29:20,769 --> 00:29:22,619
very useful.
208
00:29:22,619 --> 00:29:29,619
Another example of a problem is the n queens
problem where we have a n by n chess board
209
00:29:30,399 --> 00:29:34,979
and we have to put n queens on the board so
that none of the queens are attacking each
210
00:29:34,979 --> 00:29:41,979
other. For this problem once we have the final
configuration we know the solution. So n queens
211
00:29:45,330 --> 00:29:52,330
problem is not obvious how we can get a solution.
So we can define n queens as an optimization
212
00:29:55,589 --> 00:30:02,589
problem. So the basic idea of the iterative
methods is to start with the solution and
213
00:30:07,789 --> 00:30:11,909
improve it so that we can get a better solution.
214
00:30:11,909 --> 00:30:18,399
Example is the n queens problem. We have to
put n queens on the chess board. So in this
215
00:30:18,399 --> 00:30:24,629
case we have to put 4 queens. What we can
do is that we can start with any placement
216
00:30:24,629 --> 00:30:31,169
of the 4 queens on the board. But if you place
the queens like this obviously there are many
217
00:30:31,169 --> 00:30:33,239
conflicts.
218
00:30:33,239 --> 00:30:40,239
We can reformulate our objectives as saying
we want to have a board with 0 conflicts.
219
00:30:40,389 --> 00:30:46,539
Or we can say we want to minimize the number
of conflicts. That is we want to minimize
220
00:30:46,539 --> 00:30:53,539
the number of pairs of queens which can attack
each other. So we start with this state where
221
00:30:53,940 --> 00:30:56,859
this pair is attacking each other, this pair
is attacking, this pair attacking, this pair
222
00:30:56,859 --> 00:31:03,859
is attacking, this pair is attacking. And
then we move to this configuration where this
223
00:31:04,289 --> 00:31:11,289
is attacking, this is attacking, this is attacking.
And then we move to this configuration where
224
00:31:11,379 --> 00:31:16,289
this pair is attacking each other and none
of the other pairs are attacking each other.
225
00:31:16,289 --> 00:31:23,289
So we have reduced the number of conflicts
between queens. So, hill climbing is one local
226
00:31:30,820 --> 00:31:37,669
search method. It is also called gradient
ascend or gradient descent. In hill climbing
227
00:31:37,669 --> 00:31:44,669
the idea is that, we start with a current
configuration and we move to a new configuration
228
00:31:45,919 --> 00:31:49,299
to maximize the value.
229
00:31:49,299 --> 00:31:56,299
For example, in n queens our objective is
to minimize the number of conflicts. So we
230
00:31:57,109 --> 00:32:03,219
start with an arbitrary configuration and
we move to a new configuration where the number
231
00:32:03,219 --> 00:32:10,219
of conflicts is reduced. So we find from the
current configuration different configurations
232
00:32:10,469 --> 00:32:17,469
of different neighbors and we move to that
neighbor which has the minimum number of conflicts.
233
00:32:17,779 --> 00:32:22,739
So we move to the neighbor which is best.
That is why it is called hill climbing. We
234
00:32:22,739 --> 00:32:29,739
go to the best neighbor and we continue until
we get to a state where all the neighbors
235
00:32:30,659 --> 00:32:35,330
are worse than this state.
236
00:32:35,330 --> 00:32:40,019
We can talk about hill climbing when we are
either minimizing a value function like minimizing
237
00:32:40,019 --> 00:32:47,019
the number of conflicts or maximizing the
value function, they are symmetric. For example,
238
00:32:48,459 --> 00:32:55,359
in 8 queens we can start with an initial state
and we can define our successor function as
239
00:32:55,359 --> 00:33:02,359
moving a queen to another square in the same
column. And the cost associated with the configuration
240
00:33:04,919 --> 00:33:08,070
is the number of pairs of queens that are
attacking each other.
241
00:33:08,070 --> 00:33:15,070
We want to minimize these pairs. This is a
schematic diagram of the state space of hill
242
00:33:19,320 --> 00:33:26,289
climbing. We are starting at a particular
position and we want to move to a neighbor
243
00:33:26,289 --> 00:33:29,409
whose value is large.
244
00:33:29,409 --> 00:33:36,409
Now depending on which initial state we start
from we can get stuck in local extremum. Suppose
245
00:33:41,269 --> 00:33:47,080
we start from this position and then we find
that this is the best neighbor and then we
246
00:33:47,080 --> 00:33:53,859
find this is the best neighbor so we can move
until we get here whose neighbors are all
247
00:33:53,859 --> 00:33:59,769
worse than this. This is the solution that
we have obtained and this also happens to
248
00:33:59,769 --> 00:34:06,769
be an optimum solution. But suppose you start
from here and then if you move to the best
249
00:34:08,510 --> 00:34:15,230
neighbor and then its best neighbor and then
here you will get to this position where all
250
00:34:15,230 --> 00:34:20,540
its neighbors are worse but nevertheless it
is not the optimum solution to this problem.
251
00:34:20,540 --> 00:34:26,290
So, if we use hill climbing because it is
a local search method we can get stuck at
252
00:34:26,290 --> 00:34:33,290
a local extremum instead of all this landing
up at the global optimum.
253
00:34:37,690 --> 00:34:44,690
Hence, we can compare our state space to that
of a physical system that is subject to natural
254
00:34:45,810 --> 00:34:52,339
interaction and we can look at the analogy
of our value function to the overall potential
255
00:34:52,339 --> 00:34:59,339
energy of the system. On every updating we
have delta e less than equal to 0. So this
256
00:34:59,830 --> 00:35:06,490
is the change in potential energy.
257
00:35:06,490 --> 00:35:13,490
Now in this schema of the value of the state
space in this state this is our global optimum.
258
00:35:30,589 --> 00:35:37,589
This diagram is for a minimization problem
where c is the global minimum and what we
259
00:35:37,730 --> 00:35:44,730
see is that, if we start at any of these positions
we can get to the global minimum. So this
260
00:35:49,519 --> 00:35:56,519
region is the region of attraction the basin
of attraction for c. If our initial starting
261
00:36:00,950 --> 00:36:07,950
is between these two lines then we can reach
this global minimum. However, if our initial
262
00:36:08,859 --> 00:36:15,510
starting position is somewhere here we cannot
reach this global minimum but we will instead
263
00:36:15,510 --> 00:36:22,510
settle to a local minimum. Therefore in local
search we cannot guarantee global minimization
264
00:36:36,339 --> 00:36:43,339
because of existence of many local minima
in general.
265
00:36:43,970 --> 00:36:49,940
Now the question is, so if you use hill climbing
you cannot avoid this problem how you can
266
00:36:49,940 --> 00:36:56,010
at all avoid this problem. Let us look at
an example: Suppose this is our value function
267
00:36:56,010 --> 00:37:03,010
at different states and this is the starting
point. Now if you start at this starting point
268
00:37:04,579 --> 00:37:11,579
and we use hill climbing or hill descending
this is the direction of steepest descent
269
00:37:12,410 --> 00:37:19,410
and this is where our ball will roll until
it settles at the local minimum. So this hump
270
00:37:20,619 --> 00:37:26,680
is acting as a barrier to the local search
and our ball is settling at the local minima
271
00:37:26,680 --> 00:37:30,279
instead of going into the global minima.
272
00:37:30,279 --> 00:37:37,279
Now, if you want to avoid this problem we
must let our ball also go in the sub optimum
273
00:37:38,279 --> 00:37:45,279
direction in order to get to the local minimum.
To the global minimum it may be necessary
274
00:37:46,869 --> 00:37:52,960
to climb the hill at certain points even though
you want to reach the deepest valley. So,
275
00:37:52,960 --> 00:37:59,960
occasional ascents are required so that the
ball can get over this hump and settle at
276
00:38:02,059 --> 00:38:09,059
a minimum which may be global or local. Therefore,
ascent will help escape the local optima.
277
00:38:13,150 --> 00:38:20,150
But if we allow ascent may also help the ball
get past the global optima after reaching
278
00:38:24,349 --> 00:38:31,319
it. We can avoid this by keeping track of
the best state. If you start from here and
279
00:38:31,319 --> 00:38:38,240
you also allow ascents and then we keep track
of the best state reached so far we may be
280
00:38:38,240 --> 00:38:45,240
able to identify the best state which this
ball has reached.
281
00:38:46,769 --> 00:38:53,769
Simulated annealing is an algorithm which
avoids some pitfalls of hill climbing and
282
00:38:55,130 --> 00:39:01,809
the basic idea is this. From the current state
we pick our random successor state. If that
283
00:39:01,809 --> 00:39:08,059
successor state has a better value than the
current state then we accept the transition.
284
00:39:08,059 --> 00:39:15,059
If it has a worse value we do not give up
but with some probability we accept a worse
285
00:39:20,029 --> 00:39:26,960
transition.
If we get a successor whose value is worse
286
00:39:26,960 --> 00:39:33,960
we flip a coin and accept the transition with
some probability. Sometimes we accept non
287
00:39:35,700 --> 00:39:42,369
optimal solutions. Initially with a higher
probability we accept non optimal solutions.
288
00:39:42,369 --> 00:39:48,170
As time passes with smaller probability we
accept non optimum solutions so that we allow
289
00:39:48,170 --> 00:39:55,170
our ball to settle at a minimum. There are
other local search algorithms like genetic
290
00:40:00,549 --> 00:40:06,309
algorithms.
291
00:40:06,309 --> 00:40:13,309
What we will do now is discuss the solutions
to the questions of lecture 4. This was the
292
00:40:15,990 --> 00:40:22,990
question. You are given a full 5 gallon jug
and an empty 2 gallon jug. Your objective
293
00:40:25,380 --> 00:40:32,380
is to fill the 2 gallon jug with exactly 1
gallon of water.
294
00:40:32,510 --> 00:40:39,380
You use the state space formulation like this.
A state is represented by a paid x, y where
295
00:40:39,380 --> 00:40:45,250
x is the number of gallons of water in the
5 gallon jug, y is number of gallons of water
296
00:40:45,250 --> 00:40:52,250
in the 2 gallon jug. Initial state is first
the jug contains 5 gallons and secondly the
297
00:40:53,460 --> 00:41:00,460
jug is empty so 5, 0. Goal state is (star,
1) that is the second jug must have 1 gallon.
298
00:41:01,230 --> 00:41:06,890
The first jug can have any value we do not
care. For this problem you have to create
299
00:41:06,890 --> 00:41:12,920
the search tree and discuss which search strategy
is appropriate for this problem. Let us look
300
00:41:12,920 --> 00:41:16,609
at the solution to this problem.
301
00:41:16,609 --> 00:41:23,609
Now this table illustrates the different operators
and the effect they have. These are the operators.
302
00:41:26,019 --> 00:41:33,019
Empty 5 gallon jug, empty 2 gallon jug, transfer
from 2 to 5, transfer from 5 to 2, transfer
303
00:41:36,650 --> 00:41:43,650
from 5 to 2 partially, now empty 5 so these
are the pre conditions. Empty 5 can be always
304
00:41:47,180 --> 00:41:54,180
used and as a result you move from the states
(x, y) to the state 0 y because this jug becomes
305
00:41:56,069 --> 00:42:03,069
empty. Empty 2 that is the empty 2 gallon
jug if you apply it on state (x, y) you get
306
00:42:03,740 --> 00:42:10,650
this state (x, 0). And 2 to 5 is the operator
to pour 2 gallon into 5 gallon.
307
00:42:10,650 --> 00:42:17,650
If you start with (x, 2) you get to (x plus
2, 0) so pour 2 gallon into 5 gallon so this
308
00:42:24,000 --> 00:42:31,000
2 gallon comes here. If you pour from 5 to
2 you can apply this only if x is greater
309
00:42:33,230 --> 00:42:40,230
than 2. That is, the 5 gallon jug initially
has more than 2 gallons of water, in that
310
00:42:40,869 --> 00:42:47,869
case (x, 0) will give you (x minus 2, 2).
And 5 to 2 partially you can apply when y
311
00:42:49,839 --> 00:42:56,839
has less than 2 gallons of water. In that
case 1y will become (0, y plus 1) so you pour
312
00:43:02,039 --> 00:43:04,980
the partial 5 gallon into 2 gallon.
313
00:43:04,980 --> 00:43:11,980
This table shows the effect of all the transitions.
Now these are the different pairs of states
314
00:43:13,660 --> 00:43:20,660
0, 0 0, 1 0, 2 1,0 1,1 1, 2 2, 0 2, 1 2, 2
all these are the states of our jug problem.
315
00:43:23,539 --> 00:43:30,539
Now the empty 5 takes you from 5, 2 to 0,
2 from 4, 2 to 0, 2 2, 2 to 0, 2 and so on.
316
00:43:32,930 --> 00:43:39,930
It also takes you from 5, 1 to 0, 1 3, 1 to
0, 1 2, 1 to 0, 1 from 0 to 0, 0 and so on.
317
00:43:41,329 --> 00:43:48,329
The next operator is empty 2 which takes you
from 5, 2 to 5, 0 4, 2 to 4, 0 5, 1 to 5,
318
00:43:49,000 --> 00:43:56,000
0 3, 1 to 3, 0 and so on. And 2 to 5 takes
you from 3, 2 to 5, 0 4, 1 to 5, 0 2, 2 to
319
00:43:57,680 --> 00:44:04,680
4, 0 3, 1 to 4, 0 and so on. And 5 to 2 takes
you from 5, 1 to 4, 2 5, 0 to 3, 2 4, 1 to
320
00:44:07,240 --> 00:44:14,240
3, 2 and so on. So 5 to 2 part takes you from
2, 2 to 2, 0 2, 1 to 2, 0 and so on.
321
00:44:21,849 --> 00:44:28,849
Now let us see how from this state space we
have to find the solution from the starting
322
00:44:32,710 --> 00:44:39,710
state to a goal state?
This is the state space we start with and
323
00:44:46,329 --> 00:44:53,329
this yellow box is our starting state. And
now if we do a search we find that the optimum
324
00:45:04,589 --> 00:45:11,589
solution will go from 5, 0 to 3, 2 then from
3, 2 to 3, 0 from 3, 0 to 1, 2 and from 1,
325
00:45:13,769 --> 00:45:20,769
2 to 1, 0 from 1, 0 to 0, 1. So, we have a
solution which requires five steps.
326
00:45:23,650 --> 00:45:30,619
Now let us see which search algorithm is appropriate
for this problem. Depth first search is not
327
00:45:30,619 --> 00:45:36,200
appropriate because as you can see the state
space is a graph and there are many repeated
328
00:45:36,200 --> 00:45:43,200
states so depth first search is not appropriate.
Breadth first search is appropriate.
329
00:45:43,890 --> 00:45:49,880
Now let us look at the second question from
lecture 4. You are given the following graph:
330
00:45:49,880 --> 00:45:55,930
Starting from state A you have to execute
depth first search and reach the goal node
331
00:45:55,930 --> 00:46:02,480
G and you have to show the order in which
the nodes are expanded. And the third question
332
00:46:02,480 --> 00:46:05,690
was you run Iterative-Deepening search on
the same graph.
333
00:46:05,690 --> 00:46:12,690
We will see the solution to this problem.
This is our graph and this is the search tree
334
00:46:15,519 --> 00:46:20,180
we obtained when we looked at these states.
335
00:46:20,180 --> 00:46:27,180
After we unfold this graph this is the tree
we get. So, if you depth first search, initially
336
00:46:28,019 --> 00:46:35,019
our list OPEN, that is fringe contains the
node A. Then A is removed from the fringe,
337
00:46:37,170 --> 00:46:44,170
B and C are expanded and put in OPEN. OPEN
is a stack when we do breadth first search.
338
00:46:45,630 --> 00:46:52,569
Then B is removed from the front of the stack
and D and E are added to the front of fringe.
339
00:46:52,569 --> 00:46:59,569
When the first node from fringe D is removed
and a successor F is added to the front. Then
340
00:47:00,720 --> 00:47:07,720
F is removed from fringe its successor H is
added, then H is removed and then E is removed.
341
00:47:10,710 --> 00:47:16,579
Then C is removed its successors D and G are
added in the beginning then D is removed from
342
00:47:16,579 --> 00:47:23,579
the front its successor F is added, F is removed
H is added H is removed G is removed and we
343
00:47:24,819 --> 00:47:31,819
have found a goal state. By depth first search
this is the order in which the nodes are expanded
344
00:47:32,369 --> 00:47:39,369
A B D F H E C D F H and G.
345
00:47:41,950 --> 00:47:46,759
Now let us run Iterative-Deepening search
on the same state space. Initially the limit
346
00:47:46,759 --> 00:47:53,759
is 0 so we can expand A only. Then we set
limit equal to 1 and we get this tree which
347
00:47:55,009 --> 00:48:02,009
is expanded in a depth first manner that is
A then B then C. This illustrates the second
348
00:48:05,450 --> 00:48:12,450
iteration of IDS. In the third iteration limit
is set to 2 and in this iteration this tree
349
00:48:15,230 --> 00:48:22,230
in this iteration this tree will be searched
in depth first manner. And when we do search
350
00:48:24,140 --> 00:48:31,140
this tree we get, first A is removed from
fringe and then B and C are added. B is removed
351
00:48:34,059 --> 00:48:40,410
D and E then D is expanded E is expanded C
is expanded, D and G are added to fringe D
352
00:48:40,410 --> 00:48:47,410
is expanded and finally G is expanded and
then we have found a goal node in this iteration
353
00:48:47,730 --> 00:48:54,730
where limit is equal
354
00:49:00,749 --> 00:49:06,819
to 2.
355
00:49:06,819 --> 00:49:13,819
For lecture 5 these are the questions:
You will be given a following search space.
356
00:49:19,880 --> 00:49:26,069
This search space is specified by this state.
357
00:49:26,069 --> 00:49:33,069
In this state space there are seven states
A B C D E F G. For state A there is a edge
358
00:49:37,130 --> 00:49:44,130
to state B with cost 4, there is an edge from
A to C with cost 1, there is an edge from
359
00:49:47,960 --> 00:49:54,960
B to D with cost 3, B to E with cost 8, C
to C cost 0, C to D cost 2, C to F cost 6,
360
00:50:02,910 --> 00:50:09,910
D to C cost 2, D to E cost 4, E to G cost
2 and F to G cost 8. So, given this state
361
00:50:16,670 --> 00:50:23,670
space you are required to draw the state space
of this problem. We assume that the initial
362
00:50:24,109 --> 00:50:31,109
state is A and the goal state is G. For this
state space you have to trace each of the
363
00:50:34,609 --> 00:50:41,609
following search strategies to create a search
tree and find a path from initial state to
364
00:50:41,609 --> 00:50:47,910
the goal state.
So you are required to use this state space
365
00:50:47,910 --> 00:50:54,910
and run uniform cost search greedy search
as well as A star search. For each of these
366
00:50:56,319 --> 00:51:03,019
three algorithms we will have their execution
and find the order in which the nodes are
367
00:51:03,019 --> 00:51:10,019
expanded. At each step of the search algorithm
you have to show which node is being expanded
368
00:51:10,099 --> 00:51:16,940
and the content of the OPEN list of fringe.
You also have to report the eventual solution
369
00:51:16,940 --> 00:51:23,119
found by each of the algorithms and the solution
cost that is obtained.
370
00:51:23,119 --> 00:51:30,119
Questions from lecture 5:
You are required to write the algorithm for
371
00:51:31,450 --> 00:51:36,690
bidirectional search using pseudo code. We
discussed bidirectional search in the last
372
00:51:36,690 --> 00:51:43,690
lecture but you will have to provide the pseudo
code for this algorithm. You assume that each
373
00:51:44,890 --> 00:51:51,890
search is a breadth first search and the forward
and backward searches alternate and they expand
374
00:51:53,259 --> 00:51:58,839
one node at a time.
375
00:51:58,839 --> 00:52:05,589
Third question:
You have to find the time complexity of bidirectional
376
00:52:05,589 --> 00:52:12,589
search assuming that the test for connecting
the two searches is done by comparing a newly
377
00:52:15,740 --> 00:52:22,740
generated state in the forward direction against
all the states generated in the backward direction
378
00:52:23,849 --> 00:52:30,789
one at a time. So you remember in bidirectional
search you have to check whether the two frontiers
379
00:52:30,789 --> 00:52:36,740
of the forward tree and backward tree are
met. Suppose you do this check by checking
380
00:52:36,740 --> 00:52:41,380
the node expanded in the forward tree with
all the nodes in the backward tree what would
381
00:52:41,380 --> 00:52:47,910
be time complexity of bidirectional search?
That is question number 3.
382
00:52:47,910 --> 00:52:54,910
Now we come to the questions from the current
lecture that is lecture 6. The problems for
383
00:52:57,430 --> 00:53:03,749
this lecture are of this type. You will be
given a search space. You can use actually
384
00:53:03,749 --> 00:53:10,749
the same state space that we gave in question
number 1 of lecture 5. In question number
385
00:53:14,029 --> 00:53:21,029
1 of lecture 5 we specified the search space
by this table. For the same search space you
386
00:53:26,509 --> 00:53:33,509
have to apply IDA star and for IDA star you
will also require the value of the heuristic
387
00:53:33,710 --> 00:53:40,710
function. Assume some values of h(n) and apply
IDA star on this state space.
388
00:53:42,049 --> 00:53:49,049
You have to compare IDA star with A star in
terms of time and space complexity in general.
389
00:53:49,630 --> 00:53:56,630
So you compare the correct heuristics of A
star with IDA star in terms of time complexity
390
00:53:57,640 --> 00:54:03,359
and in terms of space complexity, that is
the second question.
391
00:54:03,359 --> 00:54:10,359
Third question, suppose you are using hill
climbing to find a solution to the n queens
392
00:54:10,869 --> 00:54:17,869
problem the question is, is hill climbing
guaranteed to give you a solution?
393
00:54:21,220 --> 00:54:28,119
Question number 4:
If you use simulated annealing are you guaranteed
394
00:54:28,119 --> 00:54:35,119
to find the optimum solution of the traveling
salesperson problem? These are the questions
395
00:54:37,769 --> 00:54:44,769
of lecture 6. In next class we will discuss
some of the solution of the question both
396
00:54:47,460 --> 00:54:51,400
from lecture 5 and 6 that the end of todays
lecture