Evaluation Methodology for Cooperative ADAS Utilizing Simulation and Experiments

  • Sebastian BittlEmail author
  • Dominique Seydel
  • Jakob Pfeiffer
  • Josef Jiru
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 921)


Wireless vehicular networks are to be deployed in both Europe and the USA within upcoming years. Such networks introduce a new promising source of information about vehicular environments to be used by cooperative advanced driver assistance systems (ADAS). However, development and evaluation of such cooperative ADAS is still challenging. Hence, we introduce a novel methodology for their development and evaluation processes. It is applied to evaluate the fulfillment of requirements on position accuracy information within exchanged messages. Such requirements are only roughly defined and not sufficiently evaluated in field tests. This holds especially for Global Navigation Satellite Systems (GNSS) optimized for maximum integrity of obtained positions. Such configuration is required to increase robustness and reliability of safety critical ADAS. We find that pure GNSS-based positioning cannot fulfill position accuracy requirements of studied ADAS in most test cases.


VANET ETSI ITS ADAS Positioning Evaluation 



This work was supported by the project “Möglichkeiten und Grenzen des Multi-GNSS RAIM für zukünftige Safety-of-Life Anwendungen” (Multi RAIM II), funded by the German Federal Ministry of Economics and Technology (BMWi) and administered by the Project Management Agency for Aeronautics Research of the German Space Agency (DLR) in Bonn, Germany (grant no. 50NA1313). The authors want to thank Hanno Beckmann, Kathrin Frankl and Bernd Eissfeller from UAF Munich for their support during work on the presented topics.


  1. 1.
    Memorandum of Understanding for OEMs within the CAR 2 CAR Communication Consortium on Deployment Strategy for cooperative ITS in Europe, v 4.0102, June 2011Google Scholar
  2. 2.
    Intelligent Transport Systems (ITS); V2X Applications; Part 1: Road Hazard Signalling (RHS) application requirements specification, August 2013Google Scholar
  3. 3.
    Intelligent Transport Systems (ITS); V2X Applications; Part 2: Intersection Collision Risk Warning (ICRW) application requirements specification, August 2013Google Scholar
  4. 4.
    Intelligent Transport Systems (ITS); V2X Applications; Part 3: Longitudinal Collision Risk Warning (LCRW) application requirements specification, November 2013Google Scholar
  5. 5.
    Intelligent Transport Systems (ITS); Users and applications requirements; Part 2: Applications and facilities layer common data dictionary, September 2014Google Scholar
  6. 6.
    Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Local Dynamic Map (LDM), September 2014Google Scholar
  7. 7.
    Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 2: Specification of Cooperative Awareness Basic Service, November 2014Google Scholar
  8. 8.
    Intelligent Transport Systems (ITS); Vehicular Communications; GeoNetworking; Part 4: Geographical Addressing and Forwarding for Point-to-Point and Point-to-Multipoint Communications; Sub-part 1: Media-Independent Functionality, July 2014Google Scholar
  9. 9.
    Intelligent Transport Systems; Vehicular Communications; Basic Set of Applications; Use case definitions, August 2015Google Scholar
  10. 10.
    The Collaborative GNSS Encyclopaedia (2016). Accessed Aug 2016
  11. 11.
    Broggi, A., et al.: High performance multi-track recording system for automotive applications. Int. J. Automot. Technol. 13(1), 123–132 (2012)CrossRefGoogle Scholar
  12. 12.
    Strasser, B., et al.: Networking of test and simulation methods for the development of advanced driver assistance systems (ADAS). In: 4. Tagung Sicherheit durch Fahrerassistenz (2010)Google Scholar
  13. 13.
    Beck, K.: Test Driven Development: By Example. Addison-Wesley Longman, Boston (2002)Google Scholar
  14. 14.
    Behrisch, M., Bieker, L., Erdmann, J., Krajzewicz, D.: SUMO - simulation of urban mobility: an overview. In: The Third International Conference on Advances in System Simulation, pp. 63–68, October 2011Google Scholar
  15. 15.
    Berger, C., Rumpe, B.: Engineering autonomous driving software. In: Experience from the DARPA Urban Challenge, pp. 243–272. Springer (2012).
  16. 16.
    Bittl, S.: Towards solutions for current security related issues in ETSI ITS. In: Mendizabal, J., et al. (eds.) Nets4Cars/Nets4Trains/Nets4Aircraft 2016. LNCS, vol. 9669, pp. 136–148. Springer, Cham (2016). Scholar
  17. 17.
    Bittl, S., Gonzalez, A.A., Myrtus, M., Beckmann, H., Sailer, S., Eissfeller, B.: Emerging attacks on VANET security based on GPS time spoofing. In: IEEE Communications and Network Security Conference, pp. 344–352, September 2015Google Scholar
  18. 18.
    Blackman, S.S.: Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp. Electron. Syst. Mag. 19(1), 5–18 (2004)CrossRefGoogle Scholar
  19. 19.
    Chrisofakis, E., Junghanns, A., Kehrer, C., Rink, A.: Simulation-based development of automotive control software with Modelica. In: 8th International Modelica Conference (2011)Google Scholar
  20. 20.
    Christopher, T.: Analysis of dynamic scenes: application to driving assistance. Ph.D. thesis, Institute National Polytechnique de Grenoble-INPG (2009)Google Scholar
  21. 21.
    Cohda Wireless: MK4a V2X Evaluation Kit, June 2013. Accessed July 2016
  22. 22.
    Hanzlik, A.: Simulation-based application software development in time-triggered communication systems. Int. J. Softw. Eng. Appl. 4(2), 75–92 (2013)Google Scholar
  23. 23.
    Harding, J., et al.: Vehicle-to-vehicle communications: readiness of V2V technology for application. Technical report DOT HS 812 014, Washington, DC. National Highway Traffic Safety Administration, August 2014Google Scholar
  24. 24.
    Hofmann-Wellenhof, B., Lichtenegger, H., Wasle, E.: GNSS - Global Navigation Satellite Systems: GPS, GLONASS, Galileo, and More. Springer, Heidelberg (2007). Scholar
  25. 25.
    Huckle, T., Schneider, S.: Numerische Methoden: Eine Einführung für Informatiker, Naturwissenschaftler, Ingenieure und Mathematiker, vol. 2. Springer, Heidelberg (2006). Scholar
  26. 26.
    Joerer, S., Segata, M., Bloessl, B., Cigno, R.L., Sommer, C., Dressler, F.: To crash or not to crash: estimating its likelihood and potentials of beacon-based IVC systems. In: IEEE Vehicular Networking Conference, pp. 25–32 (2012)Google Scholar
  27. 27.
    Kaplan, E., Hegarty, C.: Understanding GPS: Principles and Applications. Artech House, Norwood (2005)Google Scholar
  28. 28.
    Kluge, T.: Library for Cubic Spline interpolation in C++ (2011). Accessed Aug 2016
  29. 29.
    Lytrivis, P., Thomaidis, G., Amditis, A.: Cooperative path prediction in vehicular environments. In: 11th International Conference on Intelligent Transportation Systems, pp. 803–808 (2008)Google Scholar
  30. 30.
    Rondinone, M., et al.: iTETRIS: a modular simulation platform for the large scale evaluation of cooperative ITS applications. Simul. Model. Pract. Theory 34, 99–125 (2013)CrossRefGoogle Scholar
  31. 31.
    Madas, D., et al.: On path planning methods for automotive collision avoidance. In: Intelligent Vehicles Symposium, pp. 931–937 (2013)Google Scholar
  32. 32.
    Misra, P., Enge, P.: Global Positioning System: Signals, Measurements, and Performance. Ganga-Jamuna Press, Lincoln (2001)Google Scholar
  33. 33.
    Riley, G.F., Henderson, T.R.: The NS-3 network simulator. In: Wehrle, K., Günes, M., Gross, J. (eds.) Modeling and Tools for Network Simulation, pp. 15–34. Springer, Heidelberg (2010). Scholar
  34. 34.
    Roscher, K., Bittl, S., Gonzalez, A.A., Myrtus, M., Jiru, J.: ezCar2X: rapid-prototyping of communication technologies and cooperative ITS applications on real targets and inside simulation environments. In: 11th Conference Wireless Communication and Information, pp. 51–62, October 2014Google Scholar
  35. 35.
    Roscher, K., Jiru, J., Gonzalez, A., Heidrich, W.: ezCar2X: a modular software framework for rapid prototyping of C2X applications. In: 9th ITS European Congress, June 2013Google Scholar
  36. 36.
    Schuenemann, B.: V2X simulation runtime infrastructure VSimRTI: an assessment tool to design smart traffic management systems. Comput. Netw. 55(14), 3189–3198 (2011)CrossRefGoogle Scholar
  37. 37.
    Seydel, D., et al.: An evaluation methodology for VANET applications combining simulation and multi-sensor experiments. In: 2nd International Conference on Vehicular Intelligent Transport Systems, pp. 213–224, April 2016Google Scholar
  38. 38.
    Sharp, G., Lin, P.C., Komsuoglu, H.: Ground Truth Measurement System (2003). Accessed Aug 2016
  39. 39.
    Sommer, C., German, R., Dressler, F.: Bidirectionally coupled network and road traffic simulation for improved IVC analysis. IEEE Trans. Mob. Comput. 10(1), 3–15 (2011)CrossRefGoogle Scholar
  40. 40.
    Stiller, C., Färber, G., Kammel, S.: Cooperative cognitive automobiles. In: IEEE Intelligent Vehicles Symposium (2007)Google Scholar
  41. 41.
    Streit, R.L., Luginbuhl, T.E.: Probabilistic multi-hypothesis tracking. Technical report, Naval Underwater Systems Center Newport RI (1995)Google Scholar
  42. 42.
    Takasu, T.: RTKLIB: Open Source Program Package for RTK-GPS. FOSS4G (2009)Google Scholar
  43. 43.
    Tan, H.S., Huang, J.: DGPS-based vehicle-to-vehicle cooperative collision warning: engineering feasibility viewpoints. IEEE Trans. Intell. Transp. Syst. 7(4), 415–428 (2006)CrossRefGoogle Scholar
  44. 44.
    Voigtländer, P.: ADTF: framework for driver assistance and safety systems. In: FISTA World Automotive Congress (2008)Google Scholar
  45. 45.
    Winner, H., Hakuli, S., Wolf, G.: Handbuch Fahrerassistenzsysteme: Grundlagen, Komponenten und Systeme für aktive Sicherheit und Komfort. Vieweg+Teubner (2009)Google Scholar
  46. 46.
    Yan, G., Olariu, S., Weigle, M.C.: Providing VANET security through active position detection. Comput. Commun. Mobil. Protoc. ITS/VANET 31(12), 2883–2897 (2008)Google Scholar
  47. 47.
    Ziegler, J., Bender, P., Dang, T., Stiller, C.: Trajectory planning for bertha - a local, continuous method. In: Intelligent Vehicles Symposium, pp. 450–457 (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sebastian Bittl
    • 1
    Email author
  • Dominique Seydel
    • 2
  • Jakob Pfeiffer
    • 2
  • Josef Jiru
    • 2
  1. 1.HU BerlinBerlinGermany
  2. 2.Fraunhofer ESKMunichGermany

Personalised recommendations