Mastering the Data Pipeline for Autonomous Driving

Abstract

Autonomous driving is at hand, for some at least. Others are still struggling to produce basic ADAS functions efficiently. What is the difference between the two? It is the way in which the data is treated and used. The companies on the front line realized long ago that data plays a key and central role in the progress and development processes must be adapted accordingly. Those companies that have not adapted their processes are still struggling to catch up and are wasting time and resources.

This article discusses the key aspects and stages of data-driven development and points out the most common bottlenecks. It does not make sense to focus on just one part of the data-driven development pipeline and neglect the others. Only harmonized improvements along the entire pipeline will allow for faster progress. Inconsistencies in formats and interfaces are the most common source of project delays. Therefore, we provide a perspective from the start of the data pipeline to the application of the selected data in the training and validation processes and on to the new start of the cycle. We address all parts of the data pipeline including data logging, ingestion, management, analysis, augmentation, training, and validation using open-loop methods.

The integrated pipeline for the continuous development of machine-learningbased functions without inefficiencies is the final goal, and the technologies presented here describe how to achieve it.