Heuristic Search

Abstract

This chapter provides in depth study of heuristic search methods—the methods for searching the goal (solution) to problems, that are more like human, and do not follow the exhaustive search approach, making them far more efficient than the uninformed search methods. The introduction starts with formal definition of heuristic search, then follows Hill-climbing searches, their algorithm and analysis, best-first search, its algorithm and analysis, optimization, A-star search, and approaches to better heuristics. Finally, the search methods—simulated annealing (based on treatment of metals), Genetic Algorithm (GA)-based search method, along with their analyses are presented, followed with chapter summary, and at end an exhaustive list of practice exercises, along with multiple-choice questions are provided.

Exercises

• 1. Answer the following short review questions.

  • a. In what condition the best-first search becomes the breadth-first?

  • b. What can you infer from the condition: $f(n) = g(n)$?

  • c. What can you infer from the condition: $f(n) = h(n)$?

  • d. In what situation the $A^*$ search become best-first search?

  • e. What is the primary drawback of best-first search?

  • f. Which search method(s) use the priority-queue data structure?

• 2. Consider the 3-puzzle problem, where the board is $2 \times 2$ and there are three tiles, numbered 1, 2, and 3, and blank tile. There are four operators, which move the blank tile up, down, left, and right. The start and goal states are given in Fig. 9.9. Show, how the path to the goal can be found using

  • a. Breath-first search.

  • b. Depth-first search.

  • c. $A^*$ search having g(n) equal to number of moves from start state, and h(n) is number of misplaced tiles.

  Assume that there is no possibility to remember states that have been visited earlier. Also, use the given operators in the given order unless the search method defines otherwise. Label each visited node with a number indicating the order in which they are visited. If a search method does not find a solution, explain why this happened.

Fig. 9.9

State-space search

• 3. Explain what algorithms or heuristics are suitable for solving constraint satisfaction problems under the following situations. Justify your answers.

  • a. The problem is so tightly constrained that it is highly unlikely that solutions exist.

  • b. The domain sizes vary significantly: some variables have very large domains (over 1,000 values) and some have very small domains (with fewer than 10 values).

  • c. Eight-Queens Problem: Arrange eight queens on a chess board in such a manner that none of them can attack any of the others. (Note: A queen will attack another queen if it is crossing other queen while moving horizontally, vertically, or diagonally).

  • d. The set of variables and set of domains are handled by a computer say, M. Each constraint is handled by a networked computer, say N. Traffic in the networks is slow. To check a particular constraint, computer M sends a message to computer N through the network, which in turn will send a message back to indicate whether the constraint is satisfied or violated.

• 4. Suggest a heuristic function for the 8-puzzle that sometimes overestimates, and show how it can lead to a suboptimal solution on a particular case.

• 5. Prove that, if the heuristic function h never overestimates by more than a constant cost c, then algorithm $A^*$ making use of h returns a solution whose cost exceeds that of the optimal solution by no more than c.

• 6. Give the name of the algorithms that results from each of the following special cases:

  • a. Local beam search with $k = 1$.

  • b. Local beam search with $k = \infty$.

  • c. Simulated annealing with $T = 0$ at all times.

  • d. Genetic algorithm with population size $N = 1$.

• 7. Explain, how will you use best-first search in each of the following cases? Give the data structure and explain logic.

  • a. Speech recognition

  • b. PCB design

  • c. Routing telephone traffic

  • d. Routing Internet traffic

  • e. Scene analysis

  • f. Mechanical theorem proving

• 8. What type of data structure is suitable for implementing best-first search, such that each node in the frontier is directly accessible, and all the vertices behind it remain in the order they have been visited.

• 9. Answer the following in one sentence/one word.

  • a. How will you detect during the search of a graph that a particular node has been already visited?

  • b. Is the best-first search optimal?

  • c. Is the best-first search order-preserving?

  • d. Is the best-first search admissible?

• 10. In the Traveling Salesperson Problem (TSP) one is given a fully connected, weighted, undirected graph and is required to find the Hamiltonian cycle (a cycle visiting all of the nodes in the graph exactly once) that has the least total weight.

  • a. Outline how hill-climbing search could be used to solve TSP.

  • b. How good results would you expect hill-climbing to attain?

  • c. Can other local search algorithms be used to solve TSP?

• 11. Show that if a heuristic is consistent, then it can never overestimate the cost to reach the goal state. In other words, if a heuristic is monotonic, then it is admissible.

• 12. Suggest an admissible heuristic that is not consistent.

• 13. Can GAs have Local maximas? If it is not, how does the GAs tries to avoid it? If yes, justify it.

• 14. Explain different data structures that can be used to implement the open list in BFS, DFS, Best-first search?

• 15. Find out the worst-case memory requirements for best-first search.

• 16. If there is no solution, will $A^*$ explore the whole graph? Justify.

• 17. Define and describe the following terms related to heuristics:

  Admissibility, monotonicity, informedness.

• 18. Show that:

  • a. The $A^*$ will terminate ultimately.

  • b. During the execution of the $A^*$ algorithm, there is always a node in the open-list, that lies on the path to the goal.

  • c. If there exits a path to goal, the algorithm $A^*$ will terminate by finding the path of to goal.

  • d. If there is no solution in $A^*$, the algorithm will explore the whole graph.

• 19. Discuss the ways using which h function in $f(n) = g(n) + h(n)$ can be improved during the search.

• 20. Why must the $A^*$ algorithm work properly on a graph search, with graph having cycles?

• 21. For the graph shown in Fig. 9.5, find out whether the $A^*$ search for this graph is

  • a. Optimal?

  • b. Oder-preserving?

  • c. Complete?

  • d. Sound?

• 22. Apply the BFS algorithm for robot path planing in the presence of obstacles for a square matrix of $8 \times 8$ given in Fig. 9.10. Write an algorithm to generate the frontier paths. Assume that each move of robot in horizontal (H) and vertical (V) covers a unit distance, and the robot can take only the H and V moves. The start and goal nodes are marked as S and G. Shaded tiles indicate obstacles, i.e., robot cannot pass through these.

Fig. 9.10

$8 \times 8$ tiles, with obstacles in shades

Fig. 9.11

Magic-puzzle

• 23. Redesign the problem of robot shown in Fig. 9.10 for $A^*$ search. Assume that value of h is number of squares equal to $V-i + H-j$, where V and H are both 8.

• 24. Solve the 8-puzzle manually for 20-steps, where heuristic is number of tiles out of place with respect to goal state. Assume that $f^*(n) = g^*(n) + h^*(n)$ $= 0 + h^*(n) = h^*(n)$, so that only the heuristics is deciding factor for next node. Note that algorithm shall be DFS. In case of ties, give preference to those nodes which are to the left of the search tree.

• 25. Find an appropriate state-space representation for the following problems. Also, suggest suitable heuristics for each.

  • a. Cryptarithmetic problems (e.g., TWO $+$ TWO $=$ FOUR)

  • b. Towers of Hanoi

  • c. Farmer, Fox, Goose, and Grain.

• 26. Suggest appropriate heuristics for each of the following problems:

  • a. Theorem proving using resolution method

  • b. Blocks world

• 27. If $P=$ “heuristic is consistent”, and $Q=$ “heuristic is admissible”. Then show that $P\Rightarrow Q$. Demonstrate by counter example that $Q \nRightarrow Q$.

• 28. Consider the magic-puzzle shown in Fig. 9.11. Suggest the formalism for searching the goal state when started from the start state. (Note that in the goal state all the rows, columns, and diagonals have equal sums equal to 15).

• 29. Make use of GA to solve 4-puzzle (Fig. 9.12). A move consists, sliding of either of tiles 1 or 2, or 3 into the blank tile. Such a movement creates blank tile at a different position, and the process is repeated until goal state is reached. The solution requires not only reaching to the goal state, but also finds the trace path to reach the goal. Construct a suitable fitness function to implement search by GA, the search should consider only those members of the population which correspond to valid moves.

• 30. For the graph shown in Fig. 9.13, make use of DFS and certain depth cut-off to backtrack the search from that cut-off.

Fig. 9.12

4-puzzle

Fig. 9.13

A graph with start node S goal G

Fig. 9.14

Graph with start node A and goal node G

• 31. Use best-first search for Fig. 9.14 to find out if the search from start node A to goal node G is

  • a. Oder-preserving

  • b. Admissible

  • c. Optimal

• 32. How you will apply the simulated annealing in the following scenarios? For each case, give the problem formation so as to compute the $\varDelta E$, temperature T, the states $s, s^\prime$; the state space $\mathbf {S}$, and object function $f: \mathbf {S} \rightarrow \mathbb {R}$, and perform 5–10 iterations steps manually.

  • a. 8-puzzle

  • b. 8-Queen problem

  • c. Tic-tac-toe problem

• 33. One fine morning you find that your laptop is not booting. There can be enumerable reasons for this. Assume that you are expert in installation and maintenance of laptops. Represent the search process for trouble-shooting the laptop by constructing a search tree.

  • a. Suggest, what search method you consider as most appropriate for this? Also, explain the justification of the particular method you have chosen?

  • b. What heuristics you would like to suggest for making the search efficient?

  • c. What are the characteristics of this search? Comment for admissibility, monotonocity, and completeness of this solution.

• 34. Assume a population of 10 members, and apply GA to find out the solution by performing five cycles of iterations, each having selection, mutation, and crossover. Verify that we are far closer to the solution than we were in the begin after performing these iterations. Represent the members as bit strings $\{0, 1\}^n$ for some integer n. Also, fix up some criteria for fitness function, as well as the probability of mutation, and point of crossover.

  • a. 8-puzzle

  • b. square-root of a number

  • c. Factors of an integer

• 35. What are the consequences of the following special cases of GA-based search?

  • a. Only the selection operation is performed in each iteration, based on the fitness value.

  • b. Only the crossover operation is performed in each iteration at a random position.

  • c. Only the mutation operation is performed in each iteration at a random bit position.

• 36. Simulated annealing is guided by a changing “temperature” value that determines the likelihood of visiting nodes that appear to be worse than the current node. How does the search behave for very low and very high temperature values, and why it behaves so?

• 37. Select the best alternatives in each of the following questions.

  • i. The mutation operation is good for the following:

    $\begin{array}{ll} \text {(a) noise tolerance} &{} \text {(b) hill-climbing}\\ \text {(c) random walk} &{} \text {(d) all above} \\ \end{array}$

  • ii. The following operation of GA has maximum contribution to search:

    $\begin{array}{ll} \text {(a) mutation} &{} \text {(b) selection}\\ \text {(c) crossover} &{} \text {(d) fitness function}\\ \end{array}$

  • iii. What operation of GA is responsible for random walk?

    $\begin{array}{ll} \text {(a) mutation}&{} \text {(b) crossover} \\ \text {(c) none above}&{} \text {(d) both a and b}\\ \end{array}$

  • iv. GAs are not good for the following purpose:

    $\begin{array}{ll} \text {(a) finding exact global optimum} &{} \text {(b) local search} \\ \text {(c) approximate solution}&{} \text {(d) global search} \\ \end{array}$

  • v. GAs are good in environments which are :

    $\begin{array}{ll} \text {(a) complex} &{} \text {(b) noisy} \\ \text {(c) dynamic} &{} \text {(d) all above} \\ \end{array}$

  • vi. GA is always:

    $\begin{array}{ll} \text {(a)}\,\mathbf {P} &{} \text {(b) nondeterministic} \\ \text {(c)}\,\mathbf {NP} &{} \text {(d) deterministic} \\ \end{array}$