Skip to main content

Continuous Experimentation for Automotive Software on the Example of a Heavy Commercial Vehicle in Daily Operation

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 12292)


As the automotive industry focuses its attention more and more towards the software functionality of vehicles, techniques to deliver new software value at a fast pace are needed. Continuous Experimentation, a practice coming from the web-based systems world, is one of such techniques. It enables researchers and developers to use real-world data to verify their hypothesis and steer the software evolution based on performances and user preferences, reducing the reliance on simulations and guesswork. Several challenges prevent the verbatim adoption of this practice on automotive cyber-physical systems, e.g., safety concerns and limitations from computational resources; nonetheless, the automotive field is starting to take interest in this technique. This work aims at demonstrating and evaluating a prototypical Continuous Experimentation infrastructure, implemented on a distributed computational system housed in a commercial truck tractor that is used in daily operations by a logistic company on public roads. The system comprises computing units and sensors, and software deployment and data retrieval are only possible remotely via a mobile data connection due to the commercial interests of the logistics company. This study shows that the proposed experimentation process resulted in the development team being able to base software development choices on the real-world data collected during the experimental procedure. Additionally, a set of previously identified design criteria to enable Continuous Experimentation on automotive systems was discussed and their validity confirmed in the light of the presented work.


  • Software engineering
  • Software architecture
  • Continuous Experimentation
  • Cyber-physical systems
  • Automotive

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. 1.

  2. 2.


  1. ISO 21448:2019: “Road vehicles - safety of the intended functionality”. Accessed 04 Nov 2019

  2. ISO 26262-1:2011: “Road vehicles - functional safety”. Accessed 04 Nov 2019

  3. Tesla financials & accounting information. Accessed 31 Jan 2020

  4. Auer, F., Felderer, M.: Current state of research on continuous experimentation: a systematic mapping study. In: 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 335–344. IEEE (2018)

    Google Scholar 

  5. Cioroaica, E., Kuhn, T., Buhnova, B.: (Do not) trust in ecosystems. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 9–12 (2019)

    Google Scholar 

  6. Fagerholm, F., Guinea, A.S., Mäenpää, H., Münch, J.: The right model for continuous experimentation. J. Syst. Softw. 123, 292–305 (2017)

    CrossRef  Google Scholar 

  7. Giaimo, F., Andrade, H., Berger, C.: The automotive take on continuous experimentation: a multiple case study. In: 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 126–130. IEEE (2019).

  8. Giaimo, F., Berger, C.: Design criteria to architect continuous experimentation for self-driving vehicles. In: 2017 IEEE International Conference on Software Architecture (ICSA), pp. 203–210. IEEE (2017).

  9. Giaimo, F., Berger, C., Kirchner, C.: Considerations about continuous experimentation for resource-constrained platforms in self-driving vehicles. In: Lopes, A., de Lemos, R. (eds.) ECSA 2017. LNCS, vol. 10475, pp. 84–91. Springer, Cham (2017).

    CrossRef  Google Scholar 

  10. Gupta, S., et al.: Top challenges from the first practical online controlled experiments summit. ACM SIGKDD Explor. Newsl. 21(1), 20–35 (2019)

    CrossRef  Google Scholar 

  11. Hiller, M.: Thoughts on the future of the automotive electronic architecture (2016). Accessed 22 Oct 2019

  12. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014).

    CrossRef  Google Scholar 

  13. Mattos, D.I., Bosch, J., Olsson, H.H.: Challenges and strategies for undertaking continuous experimentation to embedded systems: industry and research perspectives. In: Garbajosa, J., Wang, X., Aguiar, A. (eds.) XP 2018. LNBIP, vol. 314, pp. 277–292. Springer, Cham (2018).

    CrossRef  Google Scholar 

  14. Holmström Olsson, H., Bosch, J.: Post-deployment data collection in software-intensive embedded products. In: Herzwurm, G., Margaria, T. (eds.) ICSOB 2013. LNBIP, vol. 150, pp. 79–89. Springer, Heidelberg (2013).

    CrossRef  Google Scholar 

  15. Ros, R., Runeson, P.: Continuous experimentation and a/b testing: a mapping study. In: Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering, RCoSE 2018, pp. 35–41. ACM, New York (2018)

    Google Scholar 

  16. Ross, P.E.: Tesla reveals its crowdsourced autopilot data. Accessed 31 Jan 2020

  17. Smith, B.W.: SAE levels of driving automation (2013). Accessed 31 Jan 2020

  18. SAE J3016: Taxonomy and definitions for terms related to on-road automated motor vehicles. Society of Automotive Engineers, Warrendale, PA (2014)

    Google Scholar 

  19. Wieringa, R.J.: Design Science Methodology for Information Systems and Software Engineering. Springer, Heidelberg (2014).

    CrossRef  Google Scholar 

Download references


This work was supported by the project Highly Automated Freight Transports (AutoFreight), funded by Vinnova FFI [2016-05413].

Author information

Authors and Affiliations


Corresponding author

Correspondence to Federico Giaimo .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Giaimo, F., Berger, C. (2020). Continuous Experimentation for Automotive Software on the Example of a Heavy Commercial Vehicle in Daily Operation. In: Jansen, A., Malavolta, I., Muccini, H., Ozkaya, I., Zimmermann, O. (eds) Software Architecture. ECSA 2020. Lecture Notes in Computer Science(), vol 12292. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58922-6

  • Online ISBN: 978-3-030-58923-3

  • eBook Packages: Computer ScienceComputer Science (R0)