Autonomous Agents and Multi-Agent Systems

, Volume 19, Issue 3, pp 332–377 | Cite as

A temporal logic-based planning and execution monitoring framework for unmanned aircraft systems

  • Patrick Doherty
  • Jonas Kvarnström
  • Fredrik Heintz
Article

Abstract

Research with autonomous unmanned aircraft systems is reaching a new degree of sophistication where targeted missions require complex types of deliberative capability integrated in a practical manner in such systems. Due to these pragmatic constraints, integration is just as important as theoretical and applied work in developing the actual deliberative functionalities. In this article, we present a temporal logic-based task planning and execution monitoring framework and its integration into a fully deployed rotor-based unmanned aircraft system developed in our laboratory. We use a very challenging emergency services application involving body identification and supply delivery as a vehicle for showing the potential use of such a framework in real-world applications. TALplanner, a temporal logic-based task planner, is used to generate mission plans. Building further on the use of TAL (Temporal Action Logic), we show how knowledge gathered from the appropriate sensors during plan execution can be used to create state structures, incrementally building a partial logical model representing the actual development of the system and its environment over time. We then show how formulas in the same logic can be used to specify the desired behavior of the system and its environment and how violations of such formulas can be detected in a timely manner in an execution monitor subsystem. The pervasive use of logic throughout the higher level deliberative layers of the system architecture provides a solid shared declarative semantics that facilitates the transfer of knowledge between different modules.

Keywords

Execution monitoring Planning Temporal action logic Reasoning about action and change Intelligent autonomous systems Unmanned aircraft systems 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alur, R., Feder, T., & Henzinger, T. A. (1991). The benefits of relaxing punctuality. In Proceedings of the 10th ACM symposium on principles of distributed computing (PODC-1991) (pp. 139–152). ACM Press: Montréal.Google Scholar
  2. 2.
    Alur, R., & Henzinger, T. A. (1992). Back to the future: Towards a theory of timed regular languages. In Proceedings of the 33rd IEEE symposium on foundations of computer science (FOCS-1992), IEEE (pp. 177–186). Pittsburgh: IEEE Computer Society Press.Google Scholar
  3. 3.
    Ambros-Ingerson, J., & Steel, S. (1988). Integrating planning, execution and monitoring. In Proceedings of the 7th national conference of artificial intelligence (AAAI-1988) (pp. 83–88). St. Paul: AAAI Press/The MIT Press.Google Scholar
  4. 4.
    Bacchus, F., & Kabanza, F. (1996). Planning for temporally extended goals. In Proceedings of the 13th national conference on artificial intelligence (AAAI-1996) (pp. 1215–1222). Portland: AAAI Press/The MIT Press.Google Scholar
  5. 5.
    Bacchus F., Kabanza F. (1998) Planning for temporally extended goals. Annals of Mathematics and Artificial Intelligence, 22: 5–27MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Barringer, H., Goldberg, A., Havelund, K., & Sen, K. (2004). Program monitoring with LTL in EAGLE. In Proceedings of the 18th international parallel and distributed processing symposium (IPDPS-2004) (264+ pp).Google Scholar
  7. 7.
    Barringer, H., Rydeheard, D., & Havelund, K. (2007). Runtime verification. Chap rule systems for run-time monitoring: From Eagle to RuleR (pp. 111–125). Berlin/Heidelberg: Springer.Google Scholar
  8. 8.
    Ben Lamine, K., & Kabanza, F. (2002). Reasoning about robot actions: A model checking approach. In Revised papers from the international seminar on advances in plan-based control of robotic agents (pp. 123–139). Springer.Google Scholar
  9. 9.
    Bjäreland, M. (2001). Model-based execution monitoring. PhD thesis, Linköpings universitet, Linköping Studies in Science and Technology, Dissertation no. 688.Google Scholar
  10. 10.
    Chien S., Knight R., Stechert A., Sherwood R., Rabideau G. (2000) Using iterative repair to improve the responsiveness of planning and scheduling. In: Chien S., Kambhampati S., Knoblock C.A.(eds) Proceedings of the 5th international conference on artificial intelligence planning systems (AIPS-2000).. AAAI Press, . Breckenridge, pp 300–307Google Scholar
  11. 11.
    Clarke, E. M., Grumberg, O., & Peled, D. A. (2000). Model checking. The MIT Press.Google Scholar
  12. 12.
    Coradeschi S., Saffiotti A. (2003) An introduction to the anchoring problem. Robotics and Autonomous Systems, 43(2–3): 85–96CrossRefGoogle Scholar
  13. 13.
    De Giacomo G., Reiter R., Soutchanski M. (1998) Execution monitoring of high-level robot programs. In: Cohn A.G., Schubert L.K., Shapiro S.C.(eds) Proceedings of the 6th international conference on principles of knowledge representation and reasoning (KR-1998).. Morgan Kaufmann, Trento, pp 453–465Google Scholar
  14. 14.
    Doherty P. (1994) Reasoning about action and change using occlusion. In: Cohn A.G.(eds) Proceedings of the 11th European conference on artificial intelligence (ECAI-1994).. John Wiley and Sons, Chichester, pp 401–405Google Scholar
  15. 15.
    Doherty, P. (2004). Advanced research with autonomous unmanned aerial vehicles. In D. Dubois, C. A. Welty, & M. A. Williams (Eds.), Proceedings on the 9th international conference on principles of knowledge representation and reasoning (KR-2004). Whistler: AAAI Press (Extended abstract for plenary talk).Google Scholar
  16. 16.
    Doherty, P. (2005). Knowledge representation and unmanned aerial vehicles. In A. Skowron, J. P. A. Barth籬 L. C. Jain, R. Sun, P. Morizet-Mahoudeaux, J. Liu, & N. Zhong (Eds.), Proceedings of the 2005 IEEE/WIC/ACM international conference on intelligent agent technology (IAT-2005) (pp. 9–16). Compiegne, France: IEEE Computer Society.Google Scholar
  17. 17.
    Doherty, P., Granlund, G., Kuchcinski, K., Sandewall, E., Nordberg, K., Skarman, E., & Wiklund, J. (2000). The WITAS unmanned aerial vehicle project. In W. Horn (Ed.), Proceedings of the 14th European conference on artificial intelligence (ECAI-2000) (pp. 747–755). Amsterdam: IOS Press.Google Scholar
  18. 18.
    Doherty P., Gustafsson J., Karlsson L., Kvarnström J. (1998) TAL: Temporal Action Logics—language specification and tutorial. Electronic Transactions on Artificial Intelligence, 2(3–4): 273–306MathSciNetGoogle Scholar
  19. 19.
    Doherty, P., Haslum, P., Heintz, F., Merz, T., Nyblom, P., Persson, T., & Wingman, B. (2004). A distributed architecture for autonomous unmanned aerial vehicle experimentation. In Proceedings of the 7th international symposium on distributed autonomous robotic systems (DARS-2004) (pp. 221–230). Toulouse, France.Google Scholar
  20. 20.
    Doherty P., Kvarnström J. (1999) TALplanner: An empirical investigation of a temporal logic-based forward chaining planner. In: Dixon C., Fisher M.(eds) Proceedings of the 6th international workshop on temporal representation and reasoning (TIME-1999).. IEEE Computer Society Press, Orlando, pp 47–54CrossRefGoogle Scholar
  21. 21.
    Doherty P., Kvarnström J. (2001) TALplanner: A temporal logic-based planner. Artificial Intelligence Magazine, 22(3): 95–102Google Scholar
  22. 22.
    Doherty, P., & Kvarnström, J. (2008). Temporal action logics. In V. Lifschitz, F. van Harmelen, & F. Porter (Eds.), The handbook of knowledge representation (Chap. 18). Elsevier.Google Scholar
  23. 23.
    Doherty, P., Łukaszewicz, W., & A. Szałas. (1995). Computing circumscription revisited: Preliminary report. In Proceedings of the 14th international joint conference on artificial intelligence (IJCAI-1995) (Vol. 2, pp. 1502–1508). Montréal: Morgan Kaufmann.Google Scholar
  24. 24.
    Doherty P., Łukaszewicz W., Szałas A. (1997) Computing circumscription revisited: A reduction algorithm. Journal of Automated Reasoning, 18: 297–336MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Doherty, P., & Meyer, J. J. C. (2007). Towards a delegation framework for aerial robotic mission scenarios. In M. Klusch, K. V. Hindriks, M. P. Papazoglou, & L. Sterling (Eds.), Proceedings of the 11th international workshop on cooperative information agents (CIA-2007), LNCS (pp. 5–26). Delft, The Netherlands: Springer.Google Scholar
  26. 26.
    Doherty, P., & Rudol, P. (2007). A UAV search and rescue scenario with human body detection and geolocalization. In 20th Australian joint conference on artificial intelligence (AI-2007), LNCS (pp. 1–13). Gold Coast, Queensland: Springer.Google Scholar
  27. 27.
    Drusinsky, D. (2003). Monitoring temporal rules combined with time series. In Proceedings of the computer aided verification conference (CAV-2003), LNCS (Vol. 2725, pp. 114–118). Springer.Google Scholar
  28. 28.
    Duranti, S., Conte, G., Lundström, D., Rudol, P., Wzorek, M., & Doherty, P. (2007). LinkMAV, a prototype rotary wing micro aerial vehicle. In Proceedings of the 17th IFAC symposium on automatic control in aerospace (ACA-2007). Toulouse, France.Google Scholar
  29. 29.
    Emerson, E. A. (1990). Temporal and modal logic. Handbook of theoretical computer science, volume b: Formal models and semantics (pp. 997–1072). MIT Press and Elsevier.Google Scholar
  30. 30.
    Fernández, J. L., & Simmons, R. G. (1998). Robust execution monitoring for navigation plans. In Proceedings of the 1998 IEEE/RSJ international conference on intelligent robots and systems (IROS-1998) (pp. 551–557). Victoria, BC, Canada.Google Scholar
  31. 31.
    Fichtner M., Grossmann A., Thielscher M. (2003) Intelligent execution monitoring in dynamic environments. Fundamenta Informaticae, 57(2–4): 371–392MATHMathSciNetGoogle Scholar
  32. 32.
    Fikes, R. (1971). Monitored execution of robot plans produced by STRIPS. In Proceedings of the IFIP congress (IFIP-1971) (pp. 189–194). Ljubljana, Yugoslavia.Google Scholar
  33. 33.
    Finkbeiner B., Sipma H. (2004) Checking finite traces using alternating automata. Formal Methods in System Design, 24(2): 101–127MATHCrossRefGoogle Scholar
  34. 34.
    Finzi, A., Ingrand, F., & Muscettola, N. (2004). Robot action planning and execution control. In Proceedings of the 4th international workshop on planning and scheduling for space (IWPSS-2004). Darmstadt, Germany.Google Scholar
  35. 35.
    Gat, E., Slack, M. G., Miller, D. P., & Firby, R. J. (1990). Path planning and execution monitoring for a planetary rover. In Proceedings of the 1990 IEEE international conference on robotics and automation (pp. 20–25). Cincinnati: IEEE Computer Society Press.Google Scholar
  36. 36.
    Gertler J. (1998) Fault detection and diagnosis in engineering systems. Marcel Dekker, New YorkGoogle Scholar
  37. 37.
    Ghallab, M. (1996). On chronicles: Representation, on-line recognition and learning. In L. C. Aiello, J. Doyle, & S. Shapiro (Eds.), Proceedings of the 5th international conference on principles of knowledge representation and reasoning (KR-1996) (pp. 597–607). San Francisco: Morgan Kaufmann.Google Scholar
  38. 38.
    Gustafsson J., Kvarnström J. (2004) Elaboration tolerance through object-orientation. Artificial Intelligence, 153: 239–285MATHCrossRefMathSciNetGoogle Scholar
  39. 39.
    Haigh, K. Z., & Veloso, M. M. (1998). Planning, execution and learning in a robotic agent. In R. Simmons, M. Veloso, & S. Smith (Eds.), Proceedings of the 4th international conference on artificial intelligence planning systems 1998 (AIPS-1998) (pp. 120–127). Pittsburgh: AAAI Press.Google Scholar
  40. 40.
    Harel D. (1987) Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8(3): 231–274MATHCrossRefMathSciNetGoogle Scholar
  41. 41.
    Heintz F., Doherty P. (2004) DyKnow: An approach to middleware for knowledge processing. Journal of Intelligent and Fuzzy Systems, 15(1): 3–13Google Scholar
  42. 42.
    Heintz, F., & Doherty, P. (2004b). Managing dynamic object structures using hypothesis generation and validation. In Proceedings of the AAAI workshop on anchoring symbols to sensor data.Google Scholar
  43. 43.
    Heintz F., Doherty P. (2006) DyKnow: A knowledge processing middleware framework and its relation to the JDL data fusion model. Journal of Intelligent and Fuzzy Systems, 17(4): 335–351Google Scholar
  44. 44.
    Heintz, F., Rudol, P., & Doherty, P. (2007). From images to traffic Behavior—a UAV tracking and monitoring application. In Proceedings of the 10th international conference on information fusion (pp. 1–8). Quebec: ISIF, IEEE, AES.Google Scholar
  45. 45.
    Karlsson L., Gustafsson J. (1999) Reasoning about concurrent interaction. Journal of Logic and Computation, 9(5): 623–650MATHCrossRefMathSciNetGoogle Scholar
  46. 46.
    Kavraki L.E., S̆vestka P., Latombe J., Overmars M.H. (1996) Probabilistic roadmaps for path planning in high dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4): 566–580CrossRefGoogle Scholar
  47. 47.
    Koubarakis, M. (1994). Complexity results for first-order theories of temporal constraints. In J. Doyle, E. Sandewall, & P. Torasso (Eds.), Proceedings of the 4th international conference on principles of knowledge representation and reasoning (KR-1994) (pp. 379–390). San Francisco: Morgan Kaufmann.Google Scholar
  48. 48.
    Kuffner, J. J., & LaValle, S. M. (2000). RRT-connect: An efficient approach to single-query path planning. In Proceedings of the IEEE international conference on robotics and automation (ICRA-2000) (pp. 995–1001). San Francisco, California, USA.Google Scholar
  49. 49.
    Kvarnström, J. (2002). Applying domain analysis techniques for domain-dependent control in TALplanner. In M. Ghallab, J. Hertzberg, & P. Traverso (Eds.), Proceedings of the sixth international conference on artificial intelligence planning and scheduling (AIPS-2002) (pp. 101–110). Menlo Park: AAAI Press.Google Scholar
  50. 50.
    Kvarnström, J. (2005). TALplanner and other extensions to temporal action logic. PhD thesis, Linköings universitet, Linköping Studies in Science and Technology, Dissertation no. 937.Google Scholar
  51. 51.
    Kvarnström J., Doherty P. (2000) TALplanner: A temporal logic based forward chaining planner. Annals of Mathematics and Artificial Intelligence, 30: 119–169MATHCrossRefGoogle Scholar
  52. 52.
    Lemai, S., & Ingrand, F. (2004). Interleaving temporal planning and execution in robotics domains. In Proceedings of the 19th national conference of artificial intelligence (AAAI-2004) (pp. 617–622). San Jose: AAAI Press.Google Scholar
  53. 53.
    Mantegazza, P., et al. (2000). RTAI: Real time application interface. Linux Journal, 72.Google Scholar
  54. 54.
    Markey N., Raskin J.F. (2006) Model checking restricted sets of timed paths. Theoretical Computer Science, 358(2–3): 273–292MATHCrossRefMathSciNetGoogle Scholar
  55. 55.
    Markey, N., & Schnoebelen, P. (2003). Model checking a path. CONCUR 2003–concurrency theory (pp. 251–265).Google Scholar
  56. 56.
    Merz, T. (2004). Building a system for autonomous aerial robotics research. In Proceedings of the 5th IFAC symposium on intelligent autonomous vehicles (IAV-2004). Lisbon, Portugal: Elsevier.Google Scholar
  57. 57.
    Merz, T., Duranti, S., & Conte, G. (2004). Autonomous landing of an unmanned aerial helicopter based on vision and inertial sensing. In Proceedings of the 9th international symposium on experimental robotics (ISER-2004). Singapore.Google Scholar
  58. 58.
    Merz, T., Rudol, P., & Wzorek, M. (2006). Control system framework for autonomous robots based on extended state machines. In Proceedings of the international conference on autonomic and autonomous systems (ICAS-2006).Google Scholar
  59. 59.
    Myers K. (1999) CPEF: Continuous planning and execution framework. AI Magazine, 20(4): 63–69Google Scholar
  60. 60.
    Pettersson O. (2005) Execution monitoring in robotics: A survey. Robotics and Autonomous Systems, 53(2): 73–88CrossRefMathSciNetGoogle Scholar
  61. 61.
    Pettersson, P. O. (2006). Sampling-based path planning for an autonomous helicopter. Licentiate thesis, Linköpings universitet, Linköping.Google Scholar
  62. 62.
    Rosu G., Havelund K. (2005) Rewriting-based techniques for runtime verification. Automated Software Engineering, 12(2): 151–197CrossRefMathSciNetGoogle Scholar
  63. 63.
    Rudol, P., & Doherty, P. (2008). Human body detection and geolocalization for UAV human body detection and geolocalization for UAV search and rescue missions using color and thermal imagery. In Proceedings of the IEEE aerospace conference (pp. 1–8).Google Scholar
  64. 64.
    Rudol, P., Wzorek, M., Conte, G., & Doherty, P. (2008). Micro unmanned aerial vehicle visual servoing for cooperative indoor exploration. In Proceedings of the IEEE aerospace conference.Google Scholar
  65. 65.
    Simmons, R., & Apfelbaum, D. (1998). A task description language for robot control. In Proceedings of the 1998 IEEE/RSJ international conference on intelligent robots and systems (IROS-1998) (pp. 1931–1937). Victoria, BC, Canada.Google Scholar
  66. 66.
    Thati P., Rosu G. (2005) Monitoring algorithms for metric temporal logic specifications. Electronic Notes in Theoretical Computer Science, 113: 145–162CrossRefGoogle Scholar
  67. 67.
    Washington, R., Golden, K., & Bresina, J. (2000). Plan execution, monitoring, and adaptation for planetary rovers. Electronic Transactions on Artificial Intelligence, 5(17).Google Scholar
  68. 68.
    Weyhrauch R. (1980) Prolegomena to a theory of mechanized formal reasoning. Artificial Intelligence, 13(1–2): 133–170MATHCrossRefMathSciNetGoogle Scholar
  69. 69.
    Wilkins D., Lee T., Berry P. (2003) Interactive execution monitoring of agent teams. Journal of Artificial Intelligence Research, 18: 217–261MATHGoogle Scholar
  70. 70.
    Wzorek, M., & Doherty, P. (2008). A framework for reconfigurable path planning for autonomous unmanned aerial vehicles. Journal of Applied Artificial Intelligence (Forthcoming).Google Scholar
  71. 71.
    Wzorek, M., Conte, G., Rudol, P., Merz, T., Duranti, S., & Doherty, P. (2006). From motion planning to control—a navigation framework for an autonomous unmanned aerial vehicle. In Proceedings of the 21st Bristol international unmanned air vehicle systems (UAVS) conference. University of Bristol Department of Aerospace Engineering.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Patrick Doherty
    • 1
  • Jonas Kvarnström
    • 1
  • Fredrik Heintz
    • 1
  1. 1.Department of Computer and Information ScienceLinköpings UniversitetLinköpingSweden

Personalised recommendations