Advertisement

Out-of-Core Parallel Frontier Search with MapReduce

  • Alexander Reinefeld
  • Thorsten Schütt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5976)

Abstract

Applying the MapReduce programming paradigm to frontier search yields simple yet efficient parallel implementations of heuristic search algorithms. We present parallel implementations of Breadth-First Frontier Search (BFFS) and Breadth-First Iterative-Deepening A* (BF-IDA*). Both scale well on high-performance systems and clusters. Using the N-puzzle as an application domain, we found that the scalability of BFFS and BF-IDA* is limited only by the performance of the I/O system. We generated the complete search space of the 15-puzzle (≈ 10 trillion states) with BFFS on 128 processors in 66 hours. Our results do not only confirm that the longest solution requires 80 moves [10], but also show how the utility of the Manhattan Distance and Linear Conflicts heuristics deteriorates in hard problems. Single random instances of the 15-puzzle can be solved in just a few seconds with our parallel BF-IDA*. Using 128 processors, the hardest 15-puzzle problem took seven seconds to solve, while hard random instances of the 24-puzzle still take more than a day of computing time.

Keywords

Goal Node Main Memory Solution Path Manhattan Distance MapReduce Framework 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Culberson, J., Schaeffer, J.: Searching with pattern databases. In: McCalla, G.I. (ed.) Canadian AI 1996. LNCS, vol. 1081, pp. 402–416. Springer, Heidelberg (1996)Google Scholar
  2. 2.
    Dean, J., Ghemawat, S.: MapReduce: Simplifed data processing on large clusters. In: OSDI (2004)Google Scholar
  3. 3.
    Dementiev, R., Kettner, L., Sanders, P.: STXXL: Standard template library for XXL data sets. In: 13th Annual European Symp. on Algorithms, pp. 640–651 (2005)Google Scholar
  4. 4.
    Edelkamp, S., Jabbar, S., Schrödl, S.: External A*. In: Biundo, S., Frühwirth, T., Palm, G. (eds.) KI 2004. LNCS (LNAI), vol. 3238, pp. 226–240. Springer, Heidelberg (2004)Google Scholar
  5. 5.
    Hansson, O., Mayer, A., Yung, M.: Critizising solutions to relaxed models yields powerful admissible heuristics. Information Sciences 63, 207–227 (1992)CrossRefGoogle Scholar
  6. 6.
    Hart, P., Nilsson, N., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Sys. Sci. Cyber. 4(2) (1968)Google Scholar
  7. 7.
    Korf, R.E.: Depth-first iterative-deepening: An optimal admissible tree search. Artificial Intelligence, 97–109 (1985)Google Scholar
  8. 8.
    Korf, R.E.: Divide-and-conquer bidirectional search: First results. In: IJCAI 1999, pp. 1184–1189 (1999)Google Scholar
  9. 9.
    Korf, R.E., Zhang, W.: Divide-and-conquer frontier search applied to optimal sequence alignment. In: AAAI 2000, pp. 910–916 (2000)Google Scholar
  10. 10.
    Korf, R.E., Schultze, P.: Large-scale parallel breadth-first search. In: AAAI 2005, pp. 1380–1385 (2005)Google Scholar
  11. 11.
    Korf, R.E., Zhang, W., Thayer, I., Hohwald, H.: Frontier search. J. ACM 52(5) (2005)Google Scholar
  12. 12.
    Korf, R.E.: Linear-time disk-based implicit graph search. J. ACM 55(6) (2009)Google Scholar
  13. 13.
    Reinefeld, A., Schnecke, V.: AIDA* – Asynchronous Parallel IDA*. In: Canadian Conf. Artificial Intelligence, pp. 295–302 (1994)Google Scholar
  14. 14.
    Romein, J.W., Bal, H.E., Schaeffer, J., Plaat, A.: A Performance Analysis of Transposition-Table-Driven Work Scheduling in Distributed Search. IEEE Trans. Parallel and Distributed Systems 13(5) (2002)Google Scholar
  15. 15.
    Zhang, Y., Hansen, E.A.: Parallel breadth-first heuristic search on a shared-memory architecture. In: Workshop on heuristic search, memory-based heuristics and their appl. (2006)Google Scholar
  16. 16.
    Zhou, R., Hansen, E.A.: Breadth-first heuristic search. In: 14th Intl. Conf. on Automated Planning and Scheduling, ICAPS 2004 (2004)Google Scholar
  17. 17.
    Zhou, R., Hansen, E.A.: Structured duplicate detection in external memory graph search. In: AAAI 2004, pp. 683–688 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Alexander Reinefeld
    • 1
  • Thorsten Schütt
    • 1
  1. 1.Zuse Institute BerlinGermany

Personalised recommendations