Skip to main content
SpringerLink
Log in

Qualitative State Space Abstraction

  • Chapter
  • First Online:
Qualitative Spatial Abstraction in Reinforcement Learning

Part of the book series: Cognitive Technologies ((COGTECH))

  • 864 Accesses

Abstract

In this chapter, we take a deeper look into the nature of abstraction. In particular, this chapter argues for the use of state space abstraction (Sect. 4.1) and presents a formal framework of abstraction and its different facets in Sect. 4.2. Based on this framework, a view on the interdependence of abstraction and representation follows in Sect. 4.3. Section 4.4 examines abstraction in agent control processes before applying them to reinforcement learning (Sect. 4.5).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Berendt, B., Barkowsky, T., Freksa, C., Kelter, S.: Spatial representation with aspect maps. In: Freksa, C., Habel, C., Wender, K.F. (eds.) Spatial Cognition: an Interdisciplinary Approach toRepresenting and Processing Spatial Knowledge, Lecture Notes in Artificial Intelligence, vol.1404, pp. 313–336. Springer, Berlin (1998)

    Google Scholar 

  • Bertel, S., Freksa, C., Vrachliotis, G.: Aspectualize and conquer in architectural design. In: Gero,J.S., Tversky, B., Knight, T. (eds.) Visual and Spatial Reasoning in Design III, pp. 255–279. Key Centre of Design, Computing and Cognition, University of Sydney (2004)

    Google Scholar 

  • Bertel, S., Vrachliotis, G., Freksa, C.: Aspect-oriented building design: Toward computer-aidedapproaches to solving spatial contraint problems in architecture. In: Allen, G.L. (ed.) AppliedSpatial Cognition: From Research to Cognitive Technology, pp. 75–102. Lawrence ErlbaumAssociates, Mahwah, NJ, USA (2007)

    Google Scholar 

  • Bittner, T., Smith, B.: A taxonomy of granular partitions. In: Montello, D. (ed.) Spatial InformationTheory: Cognitive and Computational Foundations of Geographic Information Science(COSIT), Lecture Notes in Computer Science, vol. 2205, pp. 28–43. Springer, Berlin (2001)

    Google Scholar 

  • Cohn, A.G., Hazarika, S.M.: Qualitative spatial representation and reasoning: An overview. FundamentaInformaticae 46(1–2), 1–29 (2001)

    MATH  MathSciNet  Google Scholar 

  • Degris, T., Sigaud, O., Wuillemin, P.H.: Learning the structure of factored Markov decision processesin reinforcement learning problems. In: Proceedings of the Twenty Third InternationalConference on Machine Learning (ICML), pp. 257–264. Pittsburgh, PA (2006)

    Google Scholar 

  • Dietterich, T.G.: State abstraction in MAXQ hierarchical reinforcement learning,. In: Solla, S.A., Leen, T.K., M¨uller, K.R. (eds.) Advances in Neural Information Processing Systems 12: Procedingsof the 1999 Conference, pp. 994–1000. MIT Press (2000b)

    Google Scholar 

  • Dylla, F., Frommberger, L., Wallgr¨un, J.O., Wolter, D., Nebel, B., W¨olfl, S.: SailAway: Formalizingnavigation rules. In: Proceedings of AISB Symposium on Spatial Reasoning and Communication. Edinburgh, UK (2007)

    Google Scholar 

  • Escrig, M.T., Toledo, F.: Autonomous robot navigation using human spatial concepts. InternationalJournal of Intelligent Systems 15(3), 165–196 (2000)

    Article  MATH  Google Scholar 

  • Forbus, K.D.: Qualitative Process Theory. Artificial Intelligence 24, 85–168 (1984)

    Article  Google Scholar 

  • Freksa, C.: Using orientation information for qualitative spatial reasoning. In: Frank, A.U., Campari,I., Formentini, U. (eds.) Theories and methods of spatio-temporal reasoning in geographicspace, pp. 162–178. Springer, Berlin (1992)

    Google Scholar 

  • Freksa, C., R¨ohrig, R.: Dimensions of qualitative spatial reasoning. In: Carret´e, N.P., Singh, M.G.(eds.) Proceedings of the 3rd IMACS International Workshop on Qualitative Reasoning andDecision Technologies (QUARDET), pp. 483–492. Barcelona, Spain (1993)

    Google Scholar 

  • Freksa, C., Zimmermann, K.: On the utilization of spatial structures for cognitively plausible andefficient reasoning. In: Proceedings of the IJCAI Workshop on Spatial and Temporal Reasoning,pp. 61–66. Chamb´ery, France (1993)

    Google Scholar 

  • Frommberger, L., Wolter, D.: Spatial abstraction: Aspectualization, coarsening, and conceptualclassification. In: Freksa, C., Newcombe, N.S., G¨ardenfors, P., W¨olfl, S. (eds.) Spatial CognitionVI: Reasoning, Action, Interaction: International Conference Spatial Cognition, LectureNotes in Artificial Intelligence, vol. 5248, pp. 311–327. Springer Verlag Berlin Heidelberg (2008)

    Google Scholar 

  • Galton, A., Meathrel, R.C.: Qualitative outline theory. In: Proceedings of the Sixteenth InternationalConference on Artificial Intelligence (IJCAI), pp. 1061–1066. Stockholm, Sweden (1999)

    Google Scholar 

  • Gantner, Z., Westphal, M., W¨olfl, S.: GQR – a fast reasoner for binary qualitative constraint calculi. In: Proceedings of the AAAI Workshop on Spatial and Temporal Reasoning. Chicago, IL(2008)

    Google Scholar 

  • Gutmann, J.S., Weigel, T., Nebel, B.: A fast, accurate and robust method for self-localization inpolygonal environments using laser range finders. Advanced Robotics 14(8), 651–667 (2001)

    Article  Google Scholar 

  • Herskovits, A.: Schematization. In: Olivier, P., Gapp, K.P. (eds.) Representation and Processing of Spatial Expressions, pp. 149–162. Lawrence Erlbaum Associates, Mahwah, NJ, USA (1998)

    Google Scholar 

  • Hobbs, J.R.: Granularity. In: Proceedings of the Ninth International Joint Conference on Artificial Intelligence (IJCAI), pp. 432–435. Los Angeles, CA, USA (1985)

    Google Scholar 

  • Klippel, A., Richter, K.F., Barkowsky, T., Freksa, C.: The cognitive reality of schematic maps. In: Meng, L., Zipf, A., Reichenbacher, T. (eds.) Map-based Mobile Services – Theories, Methods and Implementations, pp. 57–74. Springer, Berlin (2005)

    Google Scholar 

  • Kuipers, B.: The spatial semantic hierarchy. Artificial Intelligence 119, 191–233 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  • Mackaness, W.A., Chaudhry, O.: Generalization and symbolization. In: Shekhar, S., Xiong, H.(eds.) Encyclopedia of GIS. Springer (2008)

    Google Scholar 

  • Moratz, R.: Representing relative direction as binary relation of oriented points. In: Proceedings ofthe 17th European Conference on Artificial Intelligence (ECAI). Riva del Garda, Italy (2006)

    Google Scholar 

  • Moratz, R., Dylla, F., Frommberger, L.: A relative orientation algebra with adjustable granularity. In: Proceedings of the Workshop on Agents in Real-Time and Dynamic Environments (IJCAI05). Edinburgh, Scotland (2005)

    Google Scholar 

  • Moravec, H.P., Elfes, A.E.: High resolution maps from wide angle sonar. In: Proceedings of theIEEE International Conference on Robotics and Automation (ICRA). St. Louis, MO (1985)

    Google Scholar 

  • Porta, J.M., Celaya, E.: Reinforcement learning for agents with many sensors and actuators actingin categorizable environments. Journal of Artificial Intelligence Research 23, 79–122 (2005)

    MATH  Google Scholar 

  • Ravindran, B.: An algebraic approach to abstraction in reinforcement learning. Ph.D. thesis, Departmentof Computer Science, University of Massachusetts, Amherst MA (2004)

    Google Scholar 

  • Reynolds, S.I.: Adaptive resolution model-free reinforcement learning: Decision boundary partitioning. In: Proceedings of the Seventeenth International Conference on Machine Learning(ICML). Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  • Roberts, F.S.: Tolerance geometry. Notre Dame Journal of Formal Logic 14(1), 68–76 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  • Schiffer, S., Ferrein, A., Lakemeyer, G.: Qualitative world models for soccer robots. In: Proceedingsof theWorkshop on Qualitative Constraint Calculi: Application and Integration at KI 2006, pp. 3–14. Bremen, Germany (2006)

    Google Scholar 

  • Stell, J.G., Worboys, M.F.: Generalizing graphs using amalgamation and selection. In: G¨uting,R.H., Papadias, D., Lochovsky, F. (eds.) Advances in Spatial Databases: Proceedings of the 6thInternational Symposium on Spatial Databases (SSD), Lecture Notes in Computer Science, vol.1651, pp. 19–32. Springer-Verlag, Berlin Heidelberg (1999)

    Google Scholar 

  • Talmy, L.: How language structures space. In: Pick Jr., H.L., Acredolo, L.P. (eds.) Spatial Orientation:Theory, Research, and Application, pp. 225–282. Plenum, New York (1983)

    Google Scholar 

  • Thrun, S., Schwartz, A.: Finding structure in reinforcement learning. In: Tesauro, G., Touretzky,D., Leen, T. (eds.) Advances in Neural Information Processing Systems: Proceedings of the 1994 Conference, vol. 7. MIT Press, Cambridge, MA (1995)

    Google Scholar 

  • Uther, W.T.B., Veloso, M.M.: TTree: Tree-based state generalization with temporally abstract actions. In: Alonso, E., Kudenko, D., Kazakov, D. (eds.) Adaptive Agents and Multi-Agent Systems:Adaptation and Multi-Agent Learning, Lecture Notes in Artificial Intelligence, vol. 2636,pp. 260–290. Springer-Verlag Berlin Heidelberg (2003)

    Google Scholar 

  • Wallgr¨un, J.O., Frommberger, L., Wolter, D., Dylla, F., Freksa, C.: Qualitative spatial representationand reasoning in the SparQ-toolbox. In: Spatial Cognition V: Reasoning, Action, Interaction:International Conference Spatial Cognition 2006. Bremen, Germany (2007)

    Google Scholar 

  • Zadeh, L.A.: Fuzzy sets and information granularity. In: Gupta, M.M., Ragade, R.K., Yager, R.R.(eds.) Advances in Fuzzy Set Theory and Applications, pp. 3–18. North Holland PublishingCompany (1979)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Systems Group, Department of Mathematics and Informatics, University of Bremen, P.O. Box 330 440, 28334, Bremen, Germany

    Dr.-Ing. Lutz Frommberger

Authors
  1. Dr.-Ing. Lutz Frommberger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lutz Frommberger .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Frommberger, L. (2010). Qualitative State Space Abstraction. In: Qualitative Spatial Abstraction in Reinforcement Learning. Cognitive Technologies. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16590-0_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-16590-0_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-16589-4

  • Online ISBN: 978-3-642-16590-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation