Skip to main content
SpringerLink
Log in

External-Memory State Space Search

  • Chapter
  • First Online:
Algorithm Engineering

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9220))

Abstract

Many state spaces are so big that even in compressed form they fail to fit into main memory. As a result, during the execution of a search algorithm, only a part of the state space can be processed in main memory at a time; the remainder is stored on a disk.

In this paper we survey research efforts in external-memory search for solving state space problems, where the state space is generated by applying rules. We study different form of expressiveness and the effect of guiding the search into the direction of the goal. We consider outsourcing the search to disk as well as its additional parallelization to many-core processing units. We take the sliding-tile puzzle as a running example.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aggarwal, A., Vitter, J.S.: The input/output complexity of sorting and related problems. J. ACM 31(9), 1116–1127 (1988)

    Article  MathSciNet  Google Scholar 

  2. Ajwani, D., Dementiev, R., Meyer, U.: A computational study of external-memory BFS algorithms. In: SODA, pp. 601–610 (2006)

    Google Scholar 

  3. Ajwani, D., Malinger, I., Meyer, U., Toledo, S.: Graph search on flash memory. MPI-TR (2008)

    Google Scholar 

  4. Allen, J.D.: The Complete Book of CONNECT 4: History, Strategy, Puzzles. Sterling Publishing, New York (2011)

    Google Scholar 

  5. Allis, L.V.: A knowledge-based approach to connect-four. The game is solved: white wins. Master’s thesis, Vrije Univeriteit, The Netherlands (1998)

    Google Scholar 

  6. Barnat, J., Brim, L., Edelkamp, S., Sulewski, D., Šimeček, P.: Can flash memory help in model checking? In: Cofer, D., Fantechi, A. (eds.) FMICS 2008. LNCS, vol. 5596, pp. 150–165. Springer, Heidelberg (2009). doi:10.1007/978-3-642-03240-0_14

    Chapter  Google Scholar 

  7. Barto, A., Bradtke, S., Singh, S.: Learning to act using real-time dynamic programming. Artif. Intell. 72(1), 81–138 (1995)

    Article  Google Scholar 

  8. Bloem, R., Ravi, K., Somenzi, F.: Symbolic guided search for CTL model checking. In: DAC, pp. 29–34 (2000)

    Google Scholar 

  9. Bonet, B.: Efficient algorithms to rank and unrank permutations in lexicographic order. In: AAAI-Workshop on Search in AI and Robotics (2008)

    Google Scholar 

  10. Bonet, B., Geffner, H.: Learning depth-first: a unified approach to heuristic search in deterministic and non-deterministic settings, and its application to MDPs. In: ICAPS, pp. 142–151 (2006)

    Google Scholar 

  11. Bošnački, D., Edelkamp, S., Sulewski, D.: Efficient probabilistic model checking on general purpose graphics processors. In: Păsăreanu, C.S. (ed.) SPIN 2009. LNCS, vol. 5578, pp. 32–49. Springer, Heidelberg (2009). doi:10.1007/978-3-642-02652-2_7

    Chapter  Google Scholar 

  12. Botelho, F.C., Pagh, R., Ziviani, N.: Simple and space-efficient minimal perfect hash functions. In: WADS, pp. 139–150 (2007)

    Google Scholar 

  13. Botelho, F.C., Ziviani, N.: External perfect hashing for very large key sets. In: CIKM, pp. 653–662 (2007)

    Google Scholar 

  14. Breyer, T.M., Korf, R.E.: 1.6-bit pattern databases. In: AAAI (2010)

    Google Scholar 

  15. Burns, E., Lemons, S., Ruml, W., Zhou, R.: Suboptimal and anytime heuristic search on multi-core machines. In: ICAPS (2009)

    Google Scholar 

  16. Burns, E., Lemons, S., Zhou, R., Ruml, W.: Best-first heuristic search for multi-core machines. In: IJCAI, pp. 449–455 (2009)

    Google Scholar 

  17. Cazenave, T.: Nested monte-carlo search. In: IJCAI, pp. 456–461 (2009)

    Google Scholar 

  18. Cederman, D., Tsigas, P.: A practical quicksort algorithm for graphics processors. Technical report 2008–01, Chalmers University of Technology (2008)

    Google Scholar 

  19. Cormen, T., Leiserson, C., Rivest, R.: Introduction to Algorithms. MIT Press, Cambridge (1990)

    MATH  Google Scholar 

  20. Culberson, J.C., Schaeffer, J.: Pattern databases. Comput. Intell. 14(4), 318–334 (1998)

    Article  MathSciNet  Google Scholar 

  21. Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. In: OSDI (USENIX Association, Berkeley, CA, USA) (2004)

    Google Scholar 

  22. Dietzfelbinger, M., Karlin, A., Mehlhorn, K., auf der Heide, F.M., Rohnert, H., Tarjan, R.E.: Dynamic perfect hashing upper and lower bounds. SIAM J. Comput. 23, 738–761 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  23. Döbbelin, R., Schütt, T., Reinefeld, A.: Building large compressed PDBs for the sliding tile puzzle. In: Computer Games, pp. 16–27 (2013)

    Google Scholar 

  24. Edelkamp, S.: Planning with pattern databases. In: ECP, pp. 13–24 (2001). Reprint 2013 by AAAI Press. http://www.aaai.org/ocs/index.php/ECP/ECP01

  25. Edelkamp, S.: External symbolic heuristic search with pattern databases. In: ICAPS, pp. 51–60 (2005)

    Google Scholar 

  26. Edelkamp, S., Jabbar, S.: Externalizing real-time model checking. In: MOCHART, pp. 67–83 (2006)

    Google Scholar 

  27. Edelkamp, S., Jabbar, S.: Large-scale directed model checking LTL. In: Valmari, A. (ed.) SPIN 2006. LNCS, vol. 3925, pp. 1–18. Springer, Heidelberg (2006). doi:10.1007/11691617_1

    Chapter  Google Scholar 

  28. Edelkamp, S., Jabbar, S., Bonet, B.: External memory value iteration. In: ICAPS, pp. 414–429 (2007)

    Google Scholar 

  29. Edelkamp, S., Jabbar, S., Schrödl, S.: External A*. In: Biundo, S., Frühwirth, T., Palm, G. (eds.) KI 2004. LNCS (LNAI), vol. 3238, pp. 226–240. Springer, Heidelberg (2004). doi:10.1007/978-3-540-30221-6_18

    Chapter  Google Scholar 

  30. Edelkamp, S., Kissmann, P.: Externalizing the multiple sequence alignment problem with affine gap costs. In: Hertzberg, J., Beetz, M., Englert, R. (eds.) KI 2007. LNCS (LNAI), vol. 4667, pp. 444–447. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74565-5_36

    Chapter  Google Scholar 

  31. Edelkamp, S., Kissmann, P.: Symbolic classification of general two-player games. In: Dengel, A.R., Berns, K., Breuel, T.M., Bomarius, F., Roth-Berghofer, T.R. (eds.) KI 2008. LNCS (LNAI), vol. 5243, pp. 185–192. Springer, Heidelberg (2008). doi:10.1007/978-3-540-85845-4_23

    Chapter  Google Scholar 

  32. Edelkamp, S., Kissmann, P., Jabbar, S.: Scaling search with pattern databases. In: MOCHART, pp. 49–64 (2008)

    Google Scholar 

  33. Edelkamp, S., Kissmann, P., Rohte, M.: Symbolic and explicit search hybrid through perfect hash functions - a case study in connect four. In: ICAPS (2014)

    Google Scholar 

  34. Edelkamp, S., Leue, S., Lluch-Lafuente, A.: Directed explicit-state model checking in the validation of communication protocols. Int. J. Softw. Tools Technol. 5(2–3), 247–267 (2004)

    Article  MATH  Google Scholar 

  35. Edelkamp, S., Sanders, P., Simecek, P.: Semi-external LTL model checking. In: CAV, pp. 530–542 (2008)

    Google Scholar 

  36. Edelkamp, S., Sulewski, D.: Flash-efficient LTL model checking with minimal counterexamples. In: SEFM, pp. 73–82 (2008)

    Google Scholar 

  37. Edelkamp, S., Sulewski, D.: Model checking via delayed duplicate detection on the GPU. Technical report 821, TU Dortmund (2008)

    Google Scholar 

  38. Edelkamp, S., Sulewski, D.: Efficient probabilistic model checking on general purpose graphics processors. In: SPIN (2010)

    Google Scholar 

  39. Edelkamp, S., Sulewski, D.: GPU exploration of two-player games with perfect hash functions. In: SOCS (2010)

    Google Scholar 

  40. Edelkamp, S., Sulewski, D.: External memory breadth-first search with delayed duplicate detection on the GPU. In: MOCHART, pp. 12–31 (2011)

    Google Scholar 

  41. Edelkamp, S., Sulewski, D., Yücel, C.: Perfect hashing for state space exploration on the GPU. In: ICAPS, pp. 57–64 (2010)

    Google Scholar 

  42. Edelkamp, S., Tang, Z.: Monte-carlo tree search for the multiple sequence alignment problem. In: SOCS, pp. 9–17 (2015)

    Google Scholar 

  43. Fredman, M.L., Komlós, J., Szemerédi, E.: Storing a sparse table with \(o(1)\) worst case access time. J. ACM 3, 538–544 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  44. Govindaraju, N.K., Gray, J., Kumar, R., Manocha, D.: GPUTeraSort: high performance graphics coprocessor sorting for large database management. In: SIGMOD, pp. 325–336 (2006)

    Google Scholar 

  45. Hansen, E., Zilberstein, S.: LAO*: a heuristic search algorithm that finds solutions with loops. Artif. Intell. 129, 35–62 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  46. Harris, M., Sengupta, S., Owens, J.D.: Parallel prefix sum (scan) with CUDA. In: Nguyen, H. (ed.) GPU Gems 3, pp. 851–876. Addison-Wesley, Salt Lake City (2007)

    Google Scholar 

  47. Hart, N., Nilsson, J., Raphael, B.: A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sci. Cybern. 4(2), 100–107 (1968)

    Article  Google Scholar 

  48. Helmert, M.: Decidability and undecidability results for planning with numerical state variables. In: AIPS, pp. 303–312 (2002)

    Google Scholar 

  49. Helmert, M., Domshlak, C.: Landmarks, critical paths, abstractions: what’s the difference anyway? In: ICAPS (2009)

    Google Scholar 

  50. Helmert, M., Haslum, P., Hoffmann, J.: Flexible abstraction heuristics for optimal sequential planning. In: ICAPS, pp. 176–183 (2007)

    Google Scholar 

  51. Hoare, C.A.R.: Algorithm 64: quicksort. Commun. ACM 4(7), 321 (1961)

    Article  Google Scholar 

  52. Hoffmann, J.: The metric FF planning system: translating “Ignoring the delete list” to numerical state variables. J. Artif. Intell. Res. 20, 291–341 (2003)

    MATH  Google Scholar 

  53. Hoffmann, J., Nebel, B.: Fast plan generation through heuristic search. J. Artif. Intell. Res. 14, 253–302 (2001)

    MATH  Google Scholar 

  54. Jabbar, S.: External memory algorithms for state space exploration in model checking and action planning. PhD thesis, TU Dortmund (2008)

    Google Scholar 

  55. Jabbar, S., Edelkamp, S.: I/O efficient directed model checking. In: Cousot, R. (ed.) VMCAI 2005. LNCS, vol. 3385, pp. 313–329. Springer, Heidelberg (2005). doi:10.1007/978-3-540-30579-8_21

    Chapter  Google Scholar 

  56. Jabbar, S., Edelkamp, S.: Parallel external directed model checking with linear I/O. In: Emerson, E.A., Namjoshi, K.S. (eds.) VMCAI 2006. LNCS, vol. 3855, pp. 237–251. Springer, Heidelberg (2005). doi:10.1007/11609773_16

    Chapter  Google Scholar 

  57. Jaychandran, G., Vishal, V., Pande, V.S.: Using massively parallel simulations, Markovian models to study protein folding: examining the Villin head-piece. J. Chem. Phys. 124(6), 164 903–164 914 (2006)

    Google Scholar 

  58. Kishimoto, A., Fukunaga, A., Botea, A.: On the scaling behavior of HDA*. In: SOCS (2010)

    Google Scholar 

  59. Kishimoto, A., Fukunaga, A.S., Botea, A.: Scalable, parallel best-first search for optimal sequential planning. In: ICAPS (2009)

    Google Scholar 

  60. Kocsis, L., Szepesvári, C.: Bandit based Monte-Carlo planning. In: ICML, pp. 282–293 (2006)

    Google Scholar 

  61. Korf, R.E.: Linear-space best-first search. Artif. Intell. 62(1), 41–78 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  62. Korf, R.E.: Finding optimal solutions to Rubik’s cube using pattern databases. In: AAAI, pp. 700–705 (1997)

    Google Scholar 

  63. Korf, R.E.:. Breadth-first frontier search with delayed duplicate detection. In: MOCHART, pp. 87–92 (2003)

    Google Scholar 

  64. Korf, R.E.: Best-first frontier search with delayed duplicate detection. In: AAAI, pp. 650–657 (2004)

    Google Scholar 

  65. Korf, R.E.: Minimizing disk I/O in two-bit breadth-first search. In: AAAI, pp. 317–324 (2008)

    Google Scholar 

  66. Korf, R.E., Felner, A.: Disjoint pattern database heuristics. In: Chips Challenging Champions: Games, Computers and Artificial Intelligence, pp. 13–26. Elsevier (2002)

    Google Scholar 

  67. Korf, R.E., Felner, A.: Recent progress in heuristic search: a case study of the four-peg towers of Hanoi problem. In: IJCAI, pp. 2324–2329 (2007)

    Google Scholar 

  68. Korf, R.E., Schultze, T.: Large-scale parallel breadth-first search. In: AAAI, pp. 1380–1385 (2005)

    Google Scholar 

  69. Korf, R.E., Zhang, W.: Divide-and-conquer frontier search applied to optimal sequence alignment. In: AAAI, pp. 910–916 (2000)

    Google Scholar 

  70. Korf, R.E., Zhang, W., Thayer, I., Hohwald, H.: Frontier search. J. ACM 52(5), 715–748 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  71. Kristensen, L., Mailund, T.: Path finding with the sweep-line method using external storage. In: ICFEM, pp. 319–337 (2003)

    Google Scholar 

  72. Krueger, J., Westermann, R.: Linear algebra operators for GPU implementation of numerical algorithms. ACM Trans. Graph. 22(3), 908–916 (2003)

    Article  Google Scholar 

  73. Kunkle, D., Cooperman, G.: Solving Rubik’s cube: disk is the new RAM. Commun. ACM 51(4), 31–33 (2008)

    Article  Google Scholar 

  74. Kupferschmid, S., Dräger, K., Hoffmann, J., Finkbeiner, B., Dierks, H., Podelski, A., Behrmann, G.: Uppaal/DMC - abstraction-based Heuristics for directed model checking. In: TACAS, pp. 679–682 (2007)

    Google Scholar 

  75. Mareš, M., Straka, M.: Linear-time ranking of permutations. In: Arge, L., Hoffmann, M., Welzl, E. (eds.) ESA 2007. LNCS, vol. 4698, pp. 187–193. Springer, Heidelberg (2007). doi:10.1007/978-3-540-75520-3_18

    Chapter  Google Scholar 

  76. Mehlhorn, K., Meyer, U.: External-memory breadth-first search with sublinear I/O. In: Möhring, R., Raman, R. (eds.) ESA 2002. LNCS, vol. 2461, pp. 723–735. Springer, Heidelberg (2002). doi:10.1007/3-540-45749-6_63

    Chapter  Google Scholar 

  77. Munagala, K., Ranade, A.: I/O-complexity of graph algorithms. In: SODA, pp. 687–694 (1999)

    Google Scholar 

  78. Myrvold, W., Ruskey, F.: Ranking and unranking permutations in linear time. Inf. Process. Lett. 79(6), 281–284 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  79. Phillips, J.C., Braun, R., Wang, W., Gumbart, J., Tajkhorshid, E., Villa, E., Chipot, C., Skeel, R.D., Kale, L., Schulten, K.: Scalable molecular dynamics with NAMD. J. Comp. Chem. 26, 1781–1802 (2005)

    Article  Google Scholar 

  80. Reinefeld, A., Schütt, T.: Out-of-core parallel frontier search with MapReduce. In: Mewhort, D.J.K., Cann, N.M., Slater, G.W., Naughton, T.J. (eds.) HPCS 2009. LNCS, vol. 5976, pp. 323–336. Springer, Heidelberg (2010). doi:10.1007/978-3-642-12659-8_24

    Chapter  Google Scholar 

  81. Reinefeld, A., Schütt, T., Döbbelin, R.: Very large pattern databases for heuristic search. In: Hariri, S., Keahey, K. (eds.), HPDC, pp. 803–809 (2010)

    Google Scholar 

  82. Richter, S., Helmert, M., Westphal, M.: Landmarks revisited. In: AAAI, pp. 975–982 (2008)

    Google Scholar 

  83. Rokicki, T., Kociemba, H., Davidson, M., Dethridge, J.: The diameter of the Rubik’s cube group is twenty. SIAM J. Discrete Math. 27(2), 1082–1105 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  84. Rosin, C.D.: Nested rollout policy adaptation for Monte-Carlo tree search. In: IJCAI, pp. 649–654 (2011)

    Google Scholar 

  85. Meyer, U., Sanders, P., Sibeyn, J. (eds.): Algorithms for Memory Hierarchies. LNCS, vol. 2625. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  86. Schaeffer, J., Björnsson, Y., Burch, N., Kishimoto, A., Müller, M.: Solving checkers. In: IJCAI, pp. 292–297 (2005)

    Google Scholar 

  87. Schaeffer, J., Burch, N., Bjrnsson, Y., Kishimoto, A., Müller, M., Lake, R., Lu, S.S.P.: Checkers is solved. Science 317(5844), 1518–1522 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  88. Schroedl, S.: An improved search algorithm for optimal multiple sequence alignment. J. Artif. Intell. Res. 23, 587–623 (2005)

    MathSciNet  MATH  Google Scholar 

  89. Schütt, T., Reinefeld, A., Maier, R.: MR-search: massively parallel heuristic search. Concurr. Comput.: Pract. Exp. 25(1), 40–54 (2013)

    Article  Google Scholar 

  90. Sturtevant, N.R.: External memory PDBs: initial results. In: SARA (2013)

    Google Scholar 

  91. Sturtevant, N.R., Rutherford, M.J.: Minimizing writes in parallel external memory search. In: IJCAI (2013)

    Google Scholar 

  92. Wijs, A.: What to do Next? Analysing and optimising system behaviour in time. PhD thesis, Vrije Universiteit Amsterdam (1999)

    Google Scholar 

  93. Zhou, R., Hansen, E.: Breadth-first heuristic search. In: ICAPS, pp. 92–100 (2004)

    Google Scholar 

  94. Zhou, R., Hansen, E.: Structured duplicate detection in external-memory graph search. In: AAAI, pp. 683–689 (2004)

    Google Scholar 

  95. Zhou, R., Hansen, E.: External-memory pattern databases using structured duplicate detection. In: AAAI (2005)

    Google Scholar 

  96. Zhou, R., Hansen, E.A.: Multiple sequence alignment using A*. In: AAAI (2002). Student abstract

    Google Scholar 

  97. Zhou, R., Hansen, E.A.: Edge partitioning in external-memory graph search. In: IJCAI, pp. 2410–2417 (2007)

    Google Scholar 

  98. Zhou, R., Hansen, E.A.: Parallel structured duplicate detection. In: AAAI, pp. 1217–1222 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fakultät Mathematik Und Informatik, Universität Bremen, Am Fallturm 1, 28359, Bremen, Germany

    Stefan Edelkamp

Authors
  1. Stefan Edelkamp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Edelkamp .

Editor information

Editors and Affiliations

  1. Kiel University , Kiel, Germany

    Lasse Kliemann

  2. Karlsruhe Institute of Technology , Karlsruhe, Germany

    Peter Sanders

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this chapter

Cite this chapter

Edelkamp, S. (2016). External-Memory State Space Search. In: Kliemann, L., Sanders, P. (eds) Algorithm Engineering. Lecture Notes in Computer Science(), vol 9220. Springer, Cham. https://doi.org/10.1007/978-3-319-49487-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-49487-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-49486-9

  • Online ISBN: 978-3-319-49487-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation