Abstract
Autonomous robots may one day be allowed to fly or to drive around in large numbers, but this will require their makers and programmers to show that the most critical parts of their software are robust and reliable. Moreover, autonomous robots embed onboard deliberation functions. This is what makes them autonomous but open for new challenges. There are many approaches to consider for the V&V of AR software, e.g. write high-level specifications and derive them through correct implementation, deploy and develop new or modified V&V formalisms to program robotics components, etc. One should note that learned models aside, most models used in deliberation functions are already amenable to formal V&V. Thus, we rather focus on functional-level components or modules and propose an approach that relies on an existing robotics specification and implementation framework (Gen oM), in which we harness existing well-known formal V&V frameworks (UPPAAL, BIP, FIACRE-TINA). Gen oM was originally developed by roboticists and software engineers who wanted to clearly and precisely specify how a reusable, portable, middleware-independent, functional component should be specified and implemented. As a result, Gen oM has a rigorous specification and a clear semantics of the implementation, and it provides a template mechanism to synthesize code that opens the door to automatic formal-model synthesis and formal V&V (offline and online). This bottom-up approach, which starts from components implementation, is more modest than top-down ones which aim at a larger and more global view of the problem. Yet, it gives encouraging results on real implementations on which one can build more complex high-level properties to be then verified and validated offline but also with online monitors.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Y. Abdeddaim, E. Asarin, M. Gallien, F. Ingrand, C. Lesire, M. Sighireanu, Planning robust temporal plans: A comparison between CBTP and TGA approaches, in Proceedings of the International Conference on Automated Planning and Scheduling (2007). https://hal.archives-ouvertes.fr/hal-00157935
T. Abdellatif, J. Combaz, J. Sifakis, Model-based implementation of real-time applications, in International Conference on Embedded Software (2010). http://dl.acm.org/citation.cfm?id=1879052
D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, D. Mané, Concrete Problems in AI Safety (2016). http://arxiv.org/abs/1606.06565v2
B.D. Argall, S. Chernova, M.M. Veloso, B. Browning, A survey of robot learning from demonstration. Rob. Auton. Syst. 57(5), 469–483 (2009)
G. Bardaro, A. Semprebon, M. Matteucci,A use case in model-based robot development using AADL and ROS, in ACM/IEEE Workshop on Robotics Software Engineering (ACM Press, New York, 2018), pp. 9–16. https://doi.org/10.1007/978-3-319-10783-7_13. http://dl.acm.org/citation.cfm?doid=3196558.3196560
G. Behrmann, A. David, K.G. Larsen, A Tutorial on Uppaal 4.0. Technical Report, Department of Computer Science, Aalborg University, Denmark (2006). https://www.uppaal.com/uppaal-tutorial.pdf
S. Ben Rayana, M. Bozga, S. Bensalem, J. Combaz, RTD-finder - A tool for compositional verification of real-time component-based systems, in International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2016). http://link.springer.com/chapter/10.1007/978-3-662-49674-9_23
S. Bensalem, L. de Silva, F. Ingrand, R. Yan, A verifiable and correct-by-construction controller for robot functional levels. J. Softw. Eng. Rob. 1(2), 1–19 (2011). http://arxiv.org/abs/0908.0221v1
S. Bensalem, K. Havelund, A. Orlandini, Verification and validation meet planning and scheduling. Int. J. Softw. Tools Technol. Trans. 16(1), 1–12 (2014). https://doi.org/10.1007/s10009-013-0294-x. http://link.springer.com/10.1007/s10009-013-0294-x
A. Benveniste, G. Berry, The synchronous approach to reactive and real-time systems. Proc. IEEE 79(9), 1270–1282 (1991)
A. Benveniste, P. Caspi, S. Edwards, N. Halbwachs, P. Le Guernic, R. de Simone, The synchronous languages 12 years later. Proc. IEEE 91, 64–83 (2003). https://dblp.org/rec/journals/pieee/BenvenisteCEHGS03
B. Berthomieu, M. Diaz, Modeling and verification of time-dependent systems using time petri nets. IEEE Trans. Softw. Eng. 17(3), 259–273 (1991). http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcAuth=mekentosj&SrcApp=Papers&DestLinkType=FullRecord&DestApp=WOS&KeyUT=A1991FE66100005
B. Berthomieu, J.P. Bodeveix, P. Farail, M. Filali, H. Garavel, P. Gaufillet, F. Lang, F. Vernadat, Fiacre: An intermediate language for model verification in the topcased environment, in Embedded Real-Time Software and Systems, HAL - CCSD, Toulouse (2008). http://hal.inria.fr/docs/00/26/24/42/PDF/Berthomieu-Bodeveix-Farail-et-al-08.pdf
D. Bjørner, K. Havelund,40 Years of Formal Methods - Some Obstacles and Some Possibilities? FM (2014). https://dblp.org/rec/conf/fm/BjornerH14
J. Bohren, S. Cousins, The SMACH high-level executive. IEEE Rob. Autom. Mag. 17(4), 18–20 (2010). https://doi.org/10.1109/MRA.2010.938836. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5663871
F. Boussinot, R. de Simone, The ESTEREL language. Proc. IEEE 79 , 1293–1304 (1991)
D. Brugali, Model-Driven Software Engineering in Robotics. IEEE Rob. Autom. Mag. 22(3), 155–166 (2015). https://doi.org/10.1109/MRA.2015.2452201. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7254324
H. Bruyninckx, Open robot control software: The OROCOS project, in IEEE International Conference on Robotics and Automation (2001)
F. Cassez, C. Pagetti, O.H. Roux, A timed extension for ALTARICA. Fundam. Inform. 62, 291–332 (2004). https://dblp.org/rec/journals/fuin/CassezPR04
A. Cavalcanti, Formal methods for robotics: RoboChart, RoboSim, and more, in Formal Methods: Foundations and Applications (Springer International Publishing, Cham, 2017), pp. 3–6. https://doi.org/10.1145/1592434.1592436. http://link.springer.com/10.1007/978-3-319-70848-5_1
G. Cicala, A. Khalili, G. Metta, L. Natale, S. Pathak, L. Pulina, A. Tacchella, Engineering approaches and methods to verify software in autonomous systems, in International Conference on Intelligent Autonomous Systems (2016).http://link.springer.com/chapter/10.1007/978-3-319-08338-4_121
J. Claßen, G. Röger, G. Lakemeyer, B. Nebel, Platas—integrating planning and the action language golog. KI-Künstliche Intell. 26(1), 61–67 (2012). http://link.springer.com/article/10.1007/s13218-011-0155-2
D. Come, J. Brunel, D. Doose, Improving code quality in ROS packages using a temporal extension of first-order logic, in IEEE International Conference on Robotic Computing (IEEE, Piscataway, 2018), pp. 1–8. https://doi.org/10.1109/IRC.2018.00010. http://ieeexplore.ieee.org/document/8329874/
H. Costelha, P.U. Lima, Robot task plan representation by Petri Nets: modelling, identification, analysis and execution. Auton. Rob. 33(4), 337–360 (2012). https://doi.org/10.1142/3376. http://link.springer.com/10.1007/s10514-012-9288-x
S. Dal Zilio, B. Berthomieu, D. Le Botlan, Latency analysis of an aerial video tracking system using fiacre and tina, in FMTV Verification Challenge of WATERS 2015, LAAS-VERTICS (2015). http://arxiv.org/abs/1509.06506v1
A. David, K.G. Larsen, A. Legay, M. Mikučionis, D.B. Poulsen, UPPAAL SMC tutorial. Int. J. Softw. Tools Technol. Trans. 17, 1–19 (2015). https://doi.org/10.1007/s10009-014-0361-y. http://dx.doi.org/10.1007/s10009-014-0361-y
A. Desai, T. Dreossi, S.A. Seshia, Combining model checking and runtime verification for safe robotics, in International Conference on Runtime Verification RV (2017). https://dblp.org/rec/conf/rv/DesaiDS17
S. Dhouib, S. Kchir, S. Stinckwich,T. Ziadi, M. Ziane, RobotML, a domain-specific language to design, simulate and deploy robotic applications, in IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (2012). http://link.springer.com/chapter/10.1007/978-3-642-34327-8_16
D. Doose, C. Grand, C. Lesire, MAUVE runtime: A component-based middleware to reconfigure software architectures in real-time, in IEEE International Conference on Robotic Computing (IEEE, Piscataway, 2017), pp. 208–211. https://doi.org/10.1109/IRC.2017.47. http://ieeexplore.ieee.org/document/7926540/
V. D’Silva, D. Kroening, G. Weissenbacher, A survey of automated techniques for formal software verification. IEEE Trans. Comput. Aided Design Integr. Circuits Syst. 27(7), 1165–1178 (2008). https://doi.org/10.1109/TCAD.2008.923410. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4544862
B. Espiau, K. Kapellos, M. Jourdan, Formal verification in robotics: Why and how?, in International Symposium on Robotics Research (1996). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.54.3091&rep=rep1&type=pdf
P. Feth, M.N. Akram, R. Schuster, O. Wasenmüller, Dynamic Risk Assessment for Vehicles of Higher Automation Levels by Deep Learning (2018). http://arxiv.org/abs/1806.07635v1
M. Foughali, Formal Verification of the Functional Layer of Robotic and Autonomous Systems. PhD Thesis, LAAS/CNRS, 2018
M. Foughali, B. Berthomieu, S. Dal Zilio, P.E. Hladik, F. Ingrand, A. Mallet, Formal verification of complex robotic systems on resource-constrained platforms, in FormaliSE @ The International Conference on Software Engineering ICSE (ACM Press, New York, 2018), pp. 2–9. https://doi.org/10.1016/S1571-0661(05)80435-9. https://hal.laas.fr/hal-01778960
N. Gobillot, F. Guet, D. Doose, C. Grand, C. Lesire, L. Santinelli, Measurement-based real-time analysis of robotic software architectures, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, Piscataway, 2016), pp. 3306–3311. https://doi.org/10.1109/IROS.2016.7759509. https://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=7759509&contentType=Conference+Publications
M. Guerra, D. Efimov, G. Zheng, W. Perruquetti, Avoiding local minima in the potential field method using input-to-state stability. Control Eng. Pract. 55(C), 174–184 (2016). https://doi.org/10.1016/j.conengprac.2016.07.008. http://dx.doi.org/10.1016/j.conengprac.2016.07.008
J. Guiochet, M. Machin, H. Waeselynck, Safety-critical advanced robots: A survey. Rob. Auton. Syst. 94, 43–52 (2017). http://www.sciencedirect.com/science/article/pii/S0921889016300768
D. Hähnel, W. Burgard, G. Lakemeyer, GOLEX—bridging the gap between logic (GOLOG) and a real robot, in KI Advances in Artificial Intelligence (Springer, Berlin, 1998), pp. 165–176
R. Halder, J. Proença, N. Macedo, A. Santos, Formal verification of ros-based robotic applications using timed-automata, in IEEE/ACM International FME Workshop on Formal Methods in Software Engineering (FormaliSE) (2017). https://dblp.org/rec/conf/icse/HalderPMS17
M. Herrb, Pocolibs: POsix COmmunication LIbrary. Technical Report, LAAS-CNRS (1992). https://git.openrobots.org/projects/pocolibs/gollum/index
P.E. Hladik, Hippo. Technical Report, LAAS-CNRS (2020). https://redmine.laas.fr/projects/genom3-fiacre-template/gollum/hippo
J. Huang, C. Erdogan, Y. Zhang, B. Moore, Q. Luo, A. Sundaresan, G. Rosu, ROSRV: Runtime verification for robots, in Runtime Verification (Springer, Cham, 2014). http://link.springer.com/chapter/10.1007/978-3-319-11164-3_20
G. Infantes, M. Ghallab, F. Ingrand, Learning the behavior model of a robot. Auton. Rob. 30, 1–21 (2010). https://homepages.laas.fr/felix/publis-pdf/arj10.pdf
F. Ingrand, M. Ghallab, Deliberation for autonomous robots: a survey. Artif. Intell. 247, 10–44 (2017). https://doi.org/10.1016/j.artint.2014.11.003. http://dx.doi.org/10.1016/j.artint.2014.11.003
A. Kai, K. Hölldobler, B. Rumpe, A. Wortmann, Modeling robotics software architectures with modular model transformations. J. Softw. Eng. Rob. 8(1), 3–16 (2017). https://doi.org/10.6092/JOSER. https://www.google.com/
J. Kober, J.A. Bagnell, J. Peters, Reinforcement learning in robotics: a survey. Int. J. Rob. Res. 32, (2013). https://doi.org/10.1177/0278364913495721. http://ijr.sagepub.com/content/early/2013/08/22/0278364913495721.abstract
P. Koopman, M. Wagner, Challenges in autonomous vehicle testing and validation. SAE Int. J. Trans. Safety 4(1), 15–24 (2016). https://doi.org/10.4271/2016-01-0128. http://papers.sae.org/2016-01-0128/
D. Kortenkamp, R.G. Simmons, Robotic systems architectures and programming, in Handbook of Robotics, ed. by B. Siciliano, O. Khatib (Springer, Berlin, 2008), pp. 187–206
H. Kress-Gazit, T. Wongpiromsarn, U. Topcu, Correct, reactive, high-level robot control. IEEE Rob. Autom. Mag. 18(3), 65–74 (2011). https://doi.org/10.1109/MRA.2011.942116. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6016593
C. Lesire, F. Pommereau, ASPiC: An acting system based on skill petri net composition, in IEEE/RSJ International Conference on Intelligent Robots and Systems (2018), pp. 1–7
H.J. Levesque, R. Reiter, Y. Lesperance, F. Lin, R.B. Scherl, GOLOG: A logic programming language for dynamic domains. J. Logic Program. 31(1), 59–83 (1997). http://www.sciencedirect.com/science/article/pii/S0743106696001215
W. Li, A. Miyazawa, P. Ribeiro, A. Cavalcanti, J. Woodcock, J. Timmis, From formalised state machines to implementations of robotic controllers, in Distributed Autonomous Robotic Systems (Springer, Cham, 2018), pp. 1–14.
A. Lotz, A. Hamann, I. Lütkebohle, D. Stampfer, Modeling Non-Functional Application Domain Constraints for Component-Based Robotics Software Systems (2016). http://arxiv.org/abs/1601.02379
M. Luckcuck, M. Farrell, L. Dennis, C. Dixon, M. Fisher, Formal Specification and Verification of Autonomous Robotic Systems: A Survey (2018). http://arxiv.org/abs/1807.00048v1
A. Mallet, C. Pasteur, M. Herrb, S. Lemaignan, F. Ingrand, GenoM3: Building middleware-independent robotic components, in IEEE International Conference on Robotics and Automation (2010), pp. 4627–4632. https://doi.org/10.1109/ROBOT.2010.5509539. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5509539
W. Meng, J. Park, O. Sokolsky, S. Weirich, I. Lee, Verified ROS-based deployment of platform-independent control systems, in NASA Formal Methods (Springer International Publishing, Cham, 2015), pp. 248–262. https://doi.org/10.1007/978-3-319-17524-9_18. http://link.springer.com/10.1007/978-3-319-17524-9_18
A. Miyazawa, P. Ribeiro, W. Li, A. Cavalcanti, J. Timmis, Automatic property checking of robotic applications, in IEEE/RSJ International Conference on Intelligent Robots and Systems (2017). http://dblp.org/rec/conf/iros/Miyazawa0LCT17
C. Mühlbacher, S. Gspandl, M. Reip, G. Steinbauer, Improving dependability of industrial transport robots using model-based techniques, in IEEE International Conference on Robotics and Automation (2016), pp. 3133–3140. https://doi.org/10.1109/ICRA.2016.7487480. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=7487480
A. Nordmann, N. Hochgeschwender, D. Wigand, S. Wrede, A survey on domain-specific modeling and languages in robotics. J. Softw. Eng. Rob. 7(1), 1–25 (2016). https://scholar.google.com/
M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler, A.Y. Ng, ROS: an open-source Robot Operating System, in IEEE International Conference on Robotics and Automation (2009)
P. Ribeiro, A. Miyazawa, W. Li, A. Cavalcanti, J. Timmis, Modelling and Verification of Timed Robotic Controllers, in International Conference on Integrated Formal Methods (2017). http://dblp.org/rec/conf/ifm/0002MLCT17
K.Y. Rozier, Specification - The biggest bottleneck in formal methods and autonomy, in Verified Software: Theories, Tools, and Experiments (2016). https://doi.org/10.1007/978-3-319-48869-1. http://link.springer.com/chapter/10.1007/978-3-319-48869-1_2
Z. Saigol, Extending automotive certification processes to handle autonomous vehicles, in RoboSoft: Software Engineering for Robotics (Springer, Berlin, 2020)
C. Schlegel, Composition, separation of roles and model-driven approaches as enabler of a robotics software ecosystem, in RoboSoft: Software Engineering for Robotics (Springer, Berlin, 2020)
C. Schlegel, T. Hassler, A. Lotz, A. Steck, Robotic software systems: From code-driven to model-driven designs, in International Conference on Advanced Robotics (IEEE, Piscataway, 2009), pp. 1–8. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5174736
S.A. Seshia, D. Sadigh, S.S. Sastry, Towards Verified Artificial Intelligence (2016). http://arxiv.org/abs/1606.08514v3
R.G. Simmons, C. Pecheur, Automating model checking for autonomous systems, in AAAI Spring Symposium on Real-Time Autonomous Systems (2000)
D. Simon, R. Pissard-Gibollet, S. Arias, ORCCAD, a framework for safe robot control design and implementation, in Control Architecture for Robots (2006). https://hal.inria.fr/inria-00385258
D. Socci, P. Poplavko, S. Bensalem, M. Bozga, Modeling mixed-critical systems in real-time BIP, in 1st Workshop on Real-Time Mixed Criticality Systems (2013). https://hal.archives-ouvertes.fr/hal-00867465/
A. Sorin, L. Morten, J. Kjeld, U.P. Schultz, Rule-based dynamic safety monitoring for mobile robots. J. Softw. Eng. Rob. 7(1), 120–141 (2016). https://scholar.google.fr/
T. Sotiropoulos, H. Waeselynck, J. Guiochet, F. Ingrand, Can robot navigation bugs be found in simulation? An exploratory study, in IEEE International Conference on Software Quality, Reliability and Security (2017). https://dblp.org/rec/conf/qrs/SotiropoulosWGI17
H. Täubig, U. Frese, C. Hertzberg, C. Lüth, S. Mohr, E. Vorobev, D. Walter Guaranteeing functional safety: design for provability and computer-aided verification. Auton. Rob. 32(3), 303–331 (2011). https://doi.org/10.1007/s10514-011-9271-y. http://www.springerlink.com/index/10.1007/s10514-011-9271-y
C.J. Tomlin, I. Mitchell, A.M. Bayen, M. Oishi, Computational techniques for the verification of hybrid systems. Proc. IEEE 91(7), 986–1001 (2003). https://doi.org/10.1109/JPROC.2003.814621. http://ieeexplore.ieee.org/document/1215682/
V. Verma, A.K. Jónsson, C. Pasareanu, M. Iatauro, Universal executive and PLEXIL: engine and language for robust spacecraft control and operations, in American Institute of Aeronautics and Astronautics Space, AIAA Space Conference (2006). http://scholar.google.com/scholar?q=related:IpQ407u5_qsJ:scholar.google.com/&hl=en&num=20&as_sdt=0,5
F. Vicentini, M. Askarpour, M.G. Rossi, D. Mandrioli, Safety Assessment of Collaborative Robotics Through Automated Formal Verification. IEEE Trans. Rob. 36(1), 42–61 (2020). https://doi.org/10.1109/TRO.2019.2937471. https://ieeexplore.ieee.org/document/8844289/
B.C. Williams, M.D. Ingham, Model-based programming of intelligent embedded systems and robotic space explorers. Proc IEEE Special Issue Model. Design Embedded Softw. 91(1), 212–237 (2003)
K.W. Wong, H. Kress-Gazit, Robot operating system (ROS) introspective implementation of high-level task controllers. J. Softw. Eng. Rob. 8(1), 1–13 (2017). https://doi.org/10.6092/JOSER. http://joser.unibg.it/index.php/joser/issue/view/9
J. Woodcock, P.G. Larsen, J. Bicarregui, J.S. Fitzgerald, Formal methods: Practice and experience. ACM Comput. Surveys 41(4) (2009). https://dblp.org/rec/journals/csur/WoodcockLBF09
N. Yakymets, S. Dhouib, H. Jaber, A. Lanusse, Model-driven safety assessment of robotic systems, in IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, Piscataway, 2013), pp. 1137–1142. https://doi.org/10.1109/IROS.2013.6696493. http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6696493&contentType=Conference+Publications
Acknowledgements
This work has been supported by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 825619 (AI4EU) and the Artificial and Natural Intelligence Toulouse Institute – Institut 3iA (ANITI) under grant agreement No: ANR-19-PI3A-0004.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Ingrand, F. (2021). Verification of Autonomous Robots: A Roboticist’s Bottom-Up Approach. In: Cavalcanti, A., Dongol, B., Hierons, R., Timmis, J., Woodcock, J. (eds) Software Engineering for Robotics. Springer, Cham. https://doi.org/10.1007/978-3-030-66494-7_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-66494-7_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66493-0
Online ISBN: 978-3-030-66494-7
eBook Packages: Computer ScienceComputer Science (R0)