Skip to main content
SpringerLink
Log in

A temporal logic-based planning and execution monitoring framework for unmanned aircraft systems

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Research with autonomous unmanned aircraft systems is reaching a new degree of sophistication where targeted missions require complex types of deliberative capability integrated in a practical manner in such systems. Due to these pragmatic constraints, integration is just as important as theoretical and applied work in developing the actual deliberative functionalities. In this article, we present a temporal logic-based task planning and execution monitoring framework and its integration into a fully deployed rotor-based unmanned aircraft system developed in our laboratory. We use a very challenging emergency services application involving body identification and supply delivery as a vehicle for showing the potential use of such a framework in real-world applications. TALplanner, a temporal logic-based task planner, is used to generate mission plans. Building further on the use of TAL (Temporal Action Logic), we show how knowledge gathered from the appropriate sensors during plan execution can be used to create state structures, incrementally building a partial logical model representing the actual development of the system and its environment over time. We then show how formulas in the same logic can be used to specify the desired behavior of the system and its environment and how violations of such formulas can be detected in a timely manner in an execution monitor subsystem. The pervasive use of logic throughout the higher level deliberative layers of the system architecture provides a solid shared declarative semantics that facilitates the transfer of knowledge between different modules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Alur, R., Feder, T., & Henzinger, T. A. (1991). The benefits of relaxing punctuality. In Proceedings of the 10th ACM symposium on principles of distributed computing (PODC-1991) (pp. 139–152). ACM Press: Montréal.

  2. Alur, R., & Henzinger, T. A. (1992). Back to the future: Towards a theory of timed regular languages. In Proceedings of the 33rd IEEE symposium on foundations of computer science (FOCS-1992), IEEE (pp. 177–186). Pittsburgh: IEEE Computer Society Press.

  3. Ambros-Ingerson, J., & Steel, S. (1988). Integrating planning, execution and monitoring. In Proceedings of the 7th national conference of artificial intelligence (AAAI-1988) (pp. 83–88). St. Paul: AAAI Press/The MIT Press.

  4. Bacchus, F., & Kabanza, F. (1996). Planning for temporally extended goals. In Proceedings of the 13th national conference on artificial intelligence (AAAI-1996) (pp. 1215–1222). Portland: AAAI Press/The MIT Press.

  5. Bacchus F., Kabanza F. (1998) Planning for temporally extended goals. Annals of Mathematics and Artificial Intelligence, 22: 5–27

    Article  MATH  MathSciNet  Google Scholar 

  6. Barringer, H., Goldberg, A., Havelund, K., & Sen, K. (2004). Program monitoring with LTL in EAGLE. In Proceedings of the 18th international parallel and distributed processing symposium (IPDPS-2004) (264+ pp).

  7. Barringer, H., Rydeheard, D., & Havelund, K. (2007). Runtime verification. Chap rule systems for run-time monitoring: From Eagle to RuleR (pp. 111–125). Berlin/Heidelberg: Springer.

  8. Ben Lamine, K., & Kabanza, F. (2002). Reasoning about robot actions: A model checking approach. In Revised papers from the international seminar on advances in plan-based control of robotic agents (pp. 123–139). Springer.

  9. Bjäreland, M. (2001). Model-based execution monitoring. PhD thesis, Linköpings universitet, Linköping Studies in Science and Technology, Dissertation no. 688.

  10. Chien S., Knight R., Stechert A., Sherwood R., Rabideau G. (2000) Using iterative repair to improve the responsiveness of planning and scheduling. In: Chien S., Kambhampati S., Knoblock C.A.(eds) Proceedings of the 5th international conference on artificial intelligence planning systems (AIPS-2000).. AAAI Press, . Breckenridge, pp 300–307

    Google Scholar 

  11. Clarke, E. M., Grumberg, O., & Peled, D. A. (2000). Model checking. The MIT Press.

  12. Coradeschi S., Saffiotti A. (2003) An introduction to the anchoring problem. Robotics and Autonomous Systems, 43(2–3): 85–96

    Article  Google Scholar 

  13. De Giacomo G., Reiter R., Soutchanski M. (1998) Execution monitoring of high-level robot programs. In: Cohn A.G., Schubert L.K., Shapiro S.C.(eds) Proceedings of the 6th international conference on principles of knowledge representation and reasoning (KR-1998).. Morgan Kaufmann, Trento, pp 453–465

    Google Scholar 

  14. Doherty P. (1994) Reasoning about action and change using occlusion. In: Cohn A.G.(eds) Proceedings of the 11th European conference on artificial intelligence (ECAI-1994).. John Wiley and Sons, Chichester, pp 401–405

    Google Scholar 

  15. Doherty, P. (2004). Advanced research with autonomous unmanned aerial vehicles. In D. Dubois, C. A. Welty, & M. A. Williams (Eds.), Proceedings on the 9th international conference on principles of knowledge representation and reasoning (KR-2004). Whistler: AAAI Press (Extended abstract for plenary talk).

  16. Doherty, P. (2005). Knowledge representation and unmanned aerial vehicles. In A. Skowron, J. P. A. Barth籬 L. C. Jain, R. Sun, P. Morizet-Mahoudeaux, J. Liu, & N. Zhong (Eds.), Proceedings of the 2005 IEEE/WIC/ACM international conference on intelligent agent technology (IAT-2005) (pp. 9–16). Compiegne, France: IEEE Computer Society.

  17. Doherty, P., Granlund, G., Kuchcinski, K., Sandewall, E., Nordberg, K., Skarman, E., & Wiklund, J. (2000). The WITAS unmanned aerial vehicle project. In W. Horn (Ed.), Proceedings of the 14th European conference on artificial intelligence (ECAI-2000) (pp. 747–755). Amsterdam: IOS Press.

  18. Doherty P., Gustafsson J., Karlsson L., Kvarnström J. (1998) TAL: Temporal Action Logics—language specification and tutorial. Electronic Transactions on Artificial Intelligence, 2(3–4): 273–306

    MathSciNet  Google Scholar 

  19. Doherty, P., Haslum, P., Heintz, F., Merz, T., Nyblom, P., Persson, T., & Wingman, B. (2004). A distributed architecture for autonomous unmanned aerial vehicle experimentation. In Proceedings of the 7th international symposium on distributed autonomous robotic systems (DARS-2004) (pp. 221–230). Toulouse, France.

  20. Doherty P., Kvarnström J. (1999) TALplanner: An empirical investigation of a temporal logic-based forward chaining planner. In: Dixon C., Fisher M.(eds) Proceedings of the 6th international workshop on temporal representation and reasoning (TIME-1999).. IEEE Computer Society Press, Orlando, pp 47–54

    Chapter  Google Scholar 

  21. Doherty P., Kvarnström J. (2001) TALplanner: A temporal logic-based planner. Artificial Intelligence Magazine, 22(3): 95–102

    Google Scholar 

  22. Doherty, P., & Kvarnström, J. (2008). Temporal action logics. In V. Lifschitz, F. van Harmelen, & F. Porter (Eds.), The handbook of knowledge representation (Chap. 18). Elsevier.

  23. Doherty, P., Łukaszewicz, W., & A. Szałas. (1995). Computing circumscription revisited: Preliminary report. In Proceedings of the 14th international joint conference on artificial intelligence (IJCAI-1995) (Vol. 2, pp. 1502–1508). Montréal: Morgan Kaufmann.

  24. Doherty P., Łukaszewicz W., Szałas A. (1997) Computing circumscription revisited: A reduction algorithm. Journal of Automated Reasoning, 18: 297–336

    Article  MATH  MathSciNet  Google Scholar 

  25. Doherty, P., & Meyer, J. J. C. (2007). Towards a delegation framework for aerial robotic mission scenarios. In M. Klusch, K. V. Hindriks, M. P. Papazoglou, & L. Sterling (Eds.), Proceedings of the 11th international workshop on cooperative information agents (CIA-2007), LNCS (pp. 5–26). Delft, The Netherlands: Springer.

  26. Doherty, P., & Rudol, P. (2007). A UAV search and rescue scenario with human body detection and geolocalization. In 20th Australian joint conference on artificial intelligence (AI-2007), LNCS (pp. 1–13). Gold Coast, Queensland: Springer.

  27. Drusinsky, D. (2003). Monitoring temporal rules combined with time series. In Proceedings of the computer aided verification conference (CAV-2003), LNCS (Vol. 2725, pp. 114–118). Springer.

  28. Duranti, S., Conte, G., Lundström, D., Rudol, P., Wzorek, M., & Doherty, P. (2007). LinkMAV, a prototype rotary wing micro aerial vehicle. In Proceedings of the 17th IFAC symposium on automatic control in aerospace (ACA-2007). Toulouse, France.

  29. Emerson, E. A. (1990). Temporal and modal logic. Handbook of theoretical computer science, volume b: Formal models and semantics (pp. 997–1072). MIT Press and Elsevier.

  30. Fernández, J. L., & Simmons, R. G. (1998). Robust execution monitoring for navigation plans. In Proceedings of the 1998 IEEE/RSJ international conference on intelligent robots and systems (IROS-1998) (pp. 551–557). Victoria, BC, Canada.

  31. Fichtner M., Grossmann A., Thielscher M. (2003) Intelligent execution monitoring in dynamic environments. Fundamenta Informaticae, 57(2–4): 371–392

    MATH  MathSciNet  Google Scholar 

  32. Fikes, R. (1971). Monitored execution of robot plans produced by STRIPS. In Proceedings of the IFIP congress (IFIP-1971) (pp. 189–194). Ljubljana, Yugoslavia.

  33. Finkbeiner B., Sipma H. (2004) Checking finite traces using alternating automata. Formal Methods in System Design, 24(2): 101–127

    Article  MATH  Google Scholar 

  34. Finzi, A., Ingrand, F., & Muscettola, N. (2004). Robot action planning and execution control. In Proceedings of the 4th international workshop on planning and scheduling for space (IWPSS-2004). Darmstadt, Germany.

  35. Gat, E., Slack, M. G., Miller, D. P., & Firby, R. J. (1990). Path planning and execution monitoring for a planetary rover. In Proceedings of the 1990 IEEE international conference on robotics and automation (pp. 20–25). Cincinnati: IEEE Computer Society Press.

  36. Gertler J. (1998) Fault detection and diagnosis in engineering systems. Marcel Dekker, New York

    Google Scholar 

  37. Ghallab, M. (1996). On chronicles: Representation, on-line recognition and learning. In L. C. Aiello, J. Doyle, & S. Shapiro (Eds.), Proceedings of the 5th international conference on principles of knowledge representation and reasoning (KR-1996) (pp. 597–607). San Francisco: Morgan Kaufmann.

  38. Gustafsson J., Kvarnström J. (2004) Elaboration tolerance through object-orientation. Artificial Intelligence, 153: 239–285

    Article  MATH  MathSciNet  Google Scholar 

  39. Haigh, K. Z., & Veloso, M. M. (1998). Planning, execution and learning in a robotic agent. In R. Simmons, M. Veloso, & S. Smith (Eds.), Proceedings of the 4th international conference on artificial intelligence planning systems 1998 (AIPS-1998) (pp. 120–127). Pittsburgh: AAAI Press.

  40. Harel D. (1987) Statecharts: A visual formalism for complex systems. Science of Computer Programming, 8(3): 231–274

    Article  MATH  MathSciNet  Google Scholar 

  41. Heintz F., Doherty P. (2004) DyKnow: An approach to middleware for knowledge processing. Journal of Intelligent and Fuzzy Systems, 15(1): 3–13

    Google Scholar 

  42. Heintz, F., & Doherty, P. (2004b). Managing dynamic object structures using hypothesis generation and validation. In Proceedings of the AAAI workshop on anchoring symbols to sensor data.

  43. Heintz F., Doherty P. (2006) DyKnow: A knowledge processing middleware framework and its relation to the JDL data fusion model. Journal of Intelligent and Fuzzy Systems, 17(4): 335–351

    Google Scholar 

  44. Heintz, F., Rudol, P., & Doherty, P. (2007). From images to traffic Behavior—a UAV tracking and monitoring application. In Proceedings of the 10th international conference on information fusion (pp. 1–8). Quebec: ISIF, IEEE, AES.

  45. Karlsson L., Gustafsson J. (1999) Reasoning about concurrent interaction. Journal of Logic and Computation, 9(5): 623–650

    Article  MATH  MathSciNet  Google Scholar 

  46. Kavraki L.E., S̆vestka P., Latombe J., Overmars M.H. (1996) Probabilistic roadmaps for path planning in high dimensional configuration spaces. IEEE Transactions on Robotics and Automation, 12(4): 566–580

    Article  Google Scholar 

  47. Koubarakis, M. (1994). Complexity results for first-order theories of temporal constraints. In J. Doyle, E. Sandewall, & P. Torasso (Eds.), Proceedings of the 4th international conference on principles of knowledge representation and reasoning (KR-1994) (pp. 379–390). San Francisco: Morgan Kaufmann.

  48. Kuffner, J. J., & LaValle, S. M. (2000). RRT-connect: An efficient approach to single-query path planning. In Proceedings of the IEEE international conference on robotics and automation (ICRA-2000) (pp. 995–1001). San Francisco, California, USA.

  49. Kvarnström, J. (2002). Applying domain analysis techniques for domain-dependent control in TALplanner. In M. Ghallab, J. Hertzberg, & P. Traverso (Eds.), Proceedings of the sixth international conference on artificial intelligence planning and scheduling (AIPS-2002) (pp. 101–110). Menlo Park: AAAI Press.

  50. Kvarnström, J. (2005). TALplanner and other extensions to temporal action logic. PhD thesis, Linköings universitet, Linköping Studies in Science and Technology, Dissertation no. 937.

  51. Kvarnström J., Doherty P. (2000) TALplanner: A temporal logic based forward chaining planner. Annals of Mathematics and Artificial Intelligence, 30: 119–169

    Article  MATH  Google Scholar 

  52. Lemai, S., & Ingrand, F. (2004). Interleaving temporal planning and execution in robotics domains. In Proceedings of the 19th national conference of artificial intelligence (AAAI-2004) (pp. 617–622). San Jose: AAAI Press.

  53. Mantegazza, P., et al. (2000). RTAI: Real time application interface. Linux Journal, 72.

  54. Markey N., Raskin J.F. (2006) Model checking restricted sets of timed paths. Theoretical Computer Science, 358(2–3): 273–292

    Article  MATH  MathSciNet  Google Scholar 

  55. Markey, N., & Schnoebelen, P. (2003). Model checking a path. CONCUR 2003–concurrency theory (pp. 251–265).

  56. Merz, T. (2004). Building a system for autonomous aerial robotics research. In Proceedings of the 5th IFAC symposium on intelligent autonomous vehicles (IAV-2004). Lisbon, Portugal: Elsevier.

  57. Merz, T., Duranti, S., & Conte, G. (2004). Autonomous landing of an unmanned aerial helicopter based on vision and inertial sensing. In Proceedings of the 9th international symposium on experimental robotics (ISER-2004). Singapore.

  58. Merz, T., Rudol, P., & Wzorek, M. (2006). Control system framework for autonomous robots based on extended state machines. In Proceedings of the international conference on autonomic and autonomous systems (ICAS-2006).

  59. Myers K. (1999) CPEF: Continuous planning and execution framework. AI Magazine, 20(4): 63–69

    Google Scholar 

  60. Pettersson O. (2005) Execution monitoring in robotics: A survey. Robotics and Autonomous Systems, 53(2): 73–88

    Article  MathSciNet  Google Scholar 

  61. Pettersson, P. O. (2006). Sampling-based path planning for an autonomous helicopter. Licentiate thesis, Linköpings universitet, Linköping.

  62. Rosu G., Havelund K. (2005) Rewriting-based techniques for runtime verification. Automated Software Engineering, 12(2): 151–197

    Article  MathSciNet  Google Scholar 

  63. Rudol, P., & Doherty, P. (2008). Human body detection and geolocalization for UAV human body detection and geolocalization for UAV search and rescue missions using color and thermal imagery. In Proceedings of the IEEE aerospace conference (pp. 1–8).

  64. Rudol, P., Wzorek, M., Conte, G., & Doherty, P. (2008). Micro unmanned aerial vehicle visual servoing for cooperative indoor exploration. In Proceedings of the IEEE aerospace conference.

  65. Simmons, R., & Apfelbaum, D. (1998). A task description language for robot control. In Proceedings of the 1998 IEEE/RSJ international conference on intelligent robots and systems (IROS-1998) (pp. 1931–1937). Victoria, BC, Canada.

  66. Thati P., Rosu G. (2005) Monitoring algorithms for metric temporal logic specifications. Electronic Notes in Theoretical Computer Science, 113: 145–162

    Article  Google Scholar 

  67. Washington, R., Golden, K., & Bresina, J. (2000). Plan execution, monitoring, and adaptation for planetary rovers. Electronic Transactions on Artificial Intelligence, 5(17).

  68. Weyhrauch R. (1980) Prolegomena to a theory of mechanized formal reasoning. Artificial Intelligence, 13(1–2): 133–170

    Article  MATH  MathSciNet  Google Scholar 

  69. Wilkins D., Lee T., Berry P. (2003) Interactive execution monitoring of agent teams. Journal of Artificial Intelligence Research, 18: 217–261

    MATH  Google Scholar 

  70. Wzorek, M., & Doherty, P. (2008). A framework for reconfigurable path planning for autonomous unmanned aerial vehicles. Journal of Applied Artificial Intelligence (Forthcoming).

  71. Wzorek, M., Conte, G., Rudol, P., Merz, T., Duranti, S., & Doherty, P. (2006). From motion planning to control—a navigation framework for an autonomous unmanned aerial vehicle. In Proceedings of the 21st Bristol international unmanned air vehicle systems (UAVS) conference. University of Bristol Department of Aerospace Engineering.

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, Linköpings Universitet, 581 83, Linköping, Sweden

    Patrick Doherty, Jonas Kvarnström & Fredrik Heintz

Authors
  1. Patrick Doherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonas Kvarnström
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fredrik Heintz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonas Kvarnström.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Doherty, P., Kvarnström, J. & Heintz, F. A temporal logic-based planning and execution monitoring framework for unmanned aircraft systems. Auton Agent Multi-Agent Syst 19, 332–377 (2009). https://doi.org/10.1007/s10458-009-9079-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-009-9079-8

Keywords

Search

Navigation