Automated Minibuses: State of the Art and Improvements Through AVENUE

Abstract

This chapter looks at advances in automated vehicle (AV) technology, focusing on the central role of AVs in revolutionizing public transport as part of the AVENUE project. It traces the path of AV development, highlighting NAVYA’s pioneering contributions and the significant technological advances that have been achieved. The chapter describes the evolution of NAVYA technology and the disruption of the transport ecosystem during the ambitious goals of the AVENUE project. The project’s mission to integrate automated minibuses into the public transport ecosystem is critically analyzed, highlighting operational innovations, the integration of on-demand door-to-door services, and the comprehensive management of an AV ecosystem. Technology, legal, and regulatory challenges and continuous improvements in AV software and hardware are discussed, emphasizing NAVYA’s influential position in the European AV landscape. The chapter concludes with a reflection on the implications of the AVENUE project for future technological directions, emphasizing the wider implications for automated mobility and the need to navigate the complex interplay of innovation, regulation, and public acceptance.

NAVYA assets were taken over on April 23, 2023, by Gama (Gaussin Macnica Mobility joint venture, 51% owned by Gaussin and 49% by Macnica)

1 Introduction

The idea of AVENUE project was triggered in 2017 when the technology companies started to challenge the car original equipment manufacturers (OEMs) on their ground stating that the software will take control of the vehicles and change the way cars will be driven in the future. While car OEMs were working in a progressive V cycle way adding features one after the other to their cars, software companies disrupted this process with an out of the box thinking, where they considered cars as connected computers on wheel driven by a software that is developed in fast delivery AGILE process with AI model.

NAVYA (founded in 2014 from a group, Induct, which had launched a driverless electric minibus called Navia for Navigation par Intelligence Artificielle) took part of this disruption, as a pioneer by introducing the first commercial automated shuttle in 2015. The Autonom^® Shuttle was built without a steering wheel nor pedals to be SAE Level 4 by design. The strategy aimed to introduce shared and public automated shuttles in urban areas, thereby reducing reliance on cars for short trips, alleviating traffic congestion, freeing up parking spaces, and reducing greenhouse gas emissions. NAVYA became the reference in this automated public transport shuttle area showcasing its technology around the world, for public transport operators, advanced cities, universities and research around the world. At that time NAVYA was deploying the Autonom^® Shuttle in geofenced areas with 1–2 vehicles per site with round trips of 1–2 km.

The AVENUE concept was to raise the challenge and to integrate the automated minibus in the public transport systems. By putting the services in the heart of the project, the technology should solve more operational situations, go into mixed traffic, connect to public transport infrastructure, bring new adapted and disruptive mobility services, and deliver smooth and seamless riding like any other public transport service. Among all, the on-demand (1) combined with a door-to-door (2) approach in public transport where a passenger can use a mobile application to order a ride and be hailed was the major disruption.

To raise this challenge, NAVYA developed advanced technologies for its automated driving system (ADS) and established open-standard connectivity from the shuttles to external IT systems, enhancing operational excellence for these new mobility services.

This chapter will detail these elements and what was done to make the AVENUE project one of the major automated vehicle (AV) projects in Europe for the European Commission. AVENUE paved the way for competition and changed the perspective on automated driving, not only in terms of technology but also in the realms of services and even in the legal and regulatory framework.

2 Automated Driving Context before Starting AVENUE

Before the AVENUE project in 2018, the hype on automated driving was predicting the arrival of driverless cars by 2022.

Technology leaders like Google, with a dedicated subsidiary WAYMO, planned huge investments and forecast that robotaxis will cover major cities around the world in 2022 with at least 3000 taxis per major city. Uber was stating that they will leverage their margin by reducing the cost of human drivers, and they started investing billions in the technology. Lyft, Apple, and many other tech giants jumped in the competition pushing car leaders to react and create dedicated departments for advanced driver assistance systems (ADAS) and autonomous driving system (ADS). All the ecosystem was looking to the future with the robotaxi point of view.

Although NAVYA was working on robotaxi vision following the trend, the core business was to focus on L4 automated shuttles for shared and public transport. The idea was to bring as quick as possible these shuttles on the market by managing the risk one after the other. This was also driven by the lack of investment. As you aim for higher performance, you will require a greater financial investment, potentially accelerating your go-to market timeline by several years. NAVYA’s practical approach was to begin implementing automated driving in restricted geofenced areas, whether private or public, with limited features, low speeds, and minimal interactions with other road users. This strategy aimed to minimize safety risks, reduce costs, and address issues one by one. Once you begin, you can gradually scale up to higher performance, increase interactions, and add features incrementally. This incremental approach to the operational design domain (ODD) would enable AV shuttles to enter the market quickly, albeit with limited capabilities, while continuously building data and knowledge to help the technology learn, mature, and advance. This would allow NAVYA to start selling and engaging investments.

With the AVENUE project, the idea was to put in place several shuttles on open roads of several cities in Europe, connect them to the public transport network, and create new mobility services. One of the major concepts of AVENUE was to demonstrate a door-to-door on-demand service covering a large geofenced area with public and shared transport, rather than just having fixed bus stops and shuttles driving in a round trip. The vehicle can pick you up from where you are and drop you to where you want to go, and the same time it hails other passengers on the road as a normal public transport. It is a mix between taxi and public transport. The difference with a robotaxi is that a 15-passenger vehicle will pick up additional passengers along the route, even if their destinations differ. However, both pickups and drop-offs must be within the shuttle’s range (ride pooling).

Such a challenge needs to manage the whole AV system and ecosystem and not only the automated driving technology of the vehicle. It needs technologies to drive the vehicle, technologies to assist and secure the passengers, and technologies to receive and manage missions to a fleet vehicle. All this should be done safely with respect to the legal framework in place.

From 2018 to 2022, within the 4 years of AVENUE project, high technological and operational investment was made to develop such technologies and to reach the target fixed by AVENUE. To better understand the technological and operational leap done, we need to put the project in the technological and economical context of 2018 to allow a clear vision. Then we will draw the technological advancements made during AVENUE project. We will finish by a projection action triggered during AVENUE and will build the technological long tale to show market needs.

2.1 Market Projection

Before 2018, automated driving was at the top of the hype cycle for emerging technologies published by Panetta (2017). All major players and OEMs were projecting to have a prominent market in the coming years. The scale of the market projections was extreme, exceeding $60 billion by 2030 (Frost & Sullivan, 2018).

All actors wanted to be part of this and to take advantage. The AV technology, with the help of the advancement of artificial intelligence, built the ambition that it could be done fast and revenues can be available within a few years’ time frame. This race to the tech market pushed actors to overcommunicate on their potentials, ambitions, and vision of the future, which lead to overinvestments and overexpectations. In its 2019 report, McKinsey and Company (2019) identified that “Since 2010, investors have poured $220 billion into more than 1,100 companies across ten technology clusters. Investors invested the first $100 billion of these funds by mid-2016 and the rest thereafter.” Within a total of 120 billion investments in mobility, in the 24-month time frame (2016–2018), automated vehicle technologies were the most rapidly increasing technology sector. Today, mid-2023, the picture is much different. Expectations are even higher, but within longer time frames and with a more realistic approach, as clearly reported in the 2023 report of Deichmann et al. (2023). It is anticipated that automated driving will trigger even more revenues and will also benefit to ADAS market. Both ADS and ADAS are correlated and can benefit one to the other, and many technologies and features of ADS can be used for ADAS.

2.2 Automated Driving

Just as a reminder, when talking about driving autonomy, the standard SAE J3016 of the SAE International and ISO is the reference, with five levels of driving automation. Full automated driving System (ADS) is seen at Levels 4 and 5. Vehicles do not have driving wheels nor pedals, there is no driver, and the ADS system will fully drive the vehicle.

The advanced driver assistance system (ADAS) encompasses vehicles operating at Level 3 and below. In these levels, cars assist the drivers during the driving process. Even in Level 3, where the software can handle the vehicle in various scenarios, there will come a point when the system cannot handle the situation anymore and will request that the driver take control of the vehicle. At that moment, the system disengages technically passing full control to the driver. Level 3 automation introduces ambiguity by potentially misleading the driver about onboard responsibilities—whether the driver or the system is primarily responsible. This raises questions about legal liability in case of accidents during the transition from system control to driver control. A notable instance highlighting this issue is the well-known 2018 car accident involving an Uber self-driving car, which struck and killed a pedestrian (Said, 2018).

2.3 The Landscape of Automated Mobility

Significant technological players initiated an exceptional disruption within the traditional original equipment manufacturer (OEM) car market. While traditional OEMs were gradually incorporating ADAS features, advancing from level to level, tech giants such as Google/WAYMO, UBER, LIFT, BAIDU, and others recognized that technology was the true game changer. Instead of incrementally moving from L0 to L5, they took a more direct approach to working on SAE L5, even if, in practice, it resembled more of an L3 or L4. Their perspective envisions the future of automobiles as robustly connected computers on wheels, embodying the tech giants’ vision, rather than mere vehicles with software components, as seen in the OEM’s perspective.

The tech giants harnessed cutting-edge sensor technologies, including lidars, radars, and other advanced sensor systems. Additionally, they leveraged state-of-the-art AI technologies, whether in the realm of computer vision or AI prediction, to enhance the development of perception, trajectory planning, and driving capabilities within their advanced driver assistance systems (ADS). These companies were also well equipped with high-performance computing power, which they employed to a substantial extent. New emerging comers also followed this trend of top-down computer on wheels’ approach.

In addition to those concentrating on vehicle or driving system development, a comprehensive ecosystem was also established, featuring numerous participants operating across all facets of the automated driving market. This includes players involved in hardware, electronic control units (ECUs), sensors, feature provisioning, AI solutions, fleet management, localization technologies, mapping, networked transport of RTCM via internet protocol (NTRIP) data, and more.

2.4 NAVYA before 2018

In 2015, a small French startup, NAVYA, took the pioneering step of introducing commercial automated shuttles for public transportation. At a time when most market competitors were engaged in the race for driverless personal cars, NAVYA shifted its focus to public transport. They recognized the numerous advantages that automated driving can offer to cities, rather than to individual drivers, such as reduced traffic congestion. Their goal was to develop automated shuttles capable of accommodating up to 15 passengers, offering a shared and public transport service that encourages people to leave their personal vehicles at home for short rides. By adopting this innovative approach, NAVYA emerged as one of the pioneers that significantly influenced the global landscape of automated vehicles.

2.4.1 Hardware

As NAVYA investments were not at the same level as OEM or tech giants, NAVYA focused on solving the risk step by step, developing one feature after the other, solving ODD incrementally, and leveraging speed accordingly. No OEM, nor any industrial provider, was ready to build a vehicle platform able to be automative ready to receive the PCs and automated driving system. Thus, NAVYA built its own factory and its own mechanical platform called the Autonom^® Shuttle (Fig. 3.1, NAVYA ARMA specifications). The conception was targeted to be SAE L4 ready by design without steering wheel, without pedals, without car mirrors, and able to scale to full SAE L4 (Fig. 3.2, NAVYA ARMA sensor overview).

Fig. 3.1

NAVYA ARMA© specifications

Fig. 3.2

NAVYA automation positioning in the market, based on SAE J3016© levels of driving automation (Illustration by the author based on SAE J3016©, 2021)

NAVYA successfully integrated lidars, sensors, and various components (Fig. 3.3, NAVYA ARMA sensor overview) that were readily available from the market, even though these parts had not yet reached mass production levels with the highest automotive industry standards, mainly because such standards were nonexistent in the market at that time. The majority of industry players utilized identical components, including the same lidars and sensor sets, for their prototypes and small-scale production runs (Fig. 3.3).

Fig. 3.3

NAVYA ARMA© sensor overview

2.4.2 Software

During that period, NAVYA demonstrated that their shuttle could be completely driven and operated by embedded software connected to a central supervision center. They employed localization technologies that relied on pre-mapped environments, utilizing global navigation satellite system real-time kinematic (GNSS RTK) technologies, among others. The maps used were basic 2D representations, and the commissioning and mapping technologies were relatively straightforward, typically involving simple round trips and linear routes. These systems effectively operated within a limited operational design domain (ODD).

The shuttle’s primary routes consisted of straightforward journeys, such as round trips, covering short distances, typically about 1 or 2 km. There were minimal interactions, and if traffic lights were encountered, they were typically managed by the onboard operator.

2.4.3 Services

To put shuttles in operation, a common work with the PTO is done to define the trajectory and the route. A radio GNSS base is installed on the site, open to the sky and covering the geofenced area. This base will get several satellite signals and will use GNSS RTK technologies to send correction to the shuttle giving precise 10-cm positioning. Subsequently, NAVYA implemented an in-house mapping hardware and software solution called mapping and localization measurement system (MMS). This system had the capability to be installed on any car, allowing the digitalization and creation of highly accurate route maps with precise positioning. Furthermore, it collected all the necessary data to digitally recreate the surrounding environment. The map was sent to the R&D for cleaning and taking out parked vehicles, pedestrians, dynamic elements, etc., to keep only the road and structural elements of the environment. A clean map was built by the R&D, and the commissioning teams can start working with the Autonom Shuttle^® and a dedicated in-house tool on site to create and build the virtual rail, called the path. The path is what the shuttle will use to follow. It is built block by block, and each has its own parameters related to the real road (speed, intersection, curves, passenger crossing, etc.). Once the path is built, it is inserted in the vehicle system and put on the server. The shuttle can then use it to start the blank testing without passengers, driving and following the path and of course handling and reacting to all dynamic environments on the road. This will allow to fine-tune the path and to adjust also all the real driving parameters to have a smooth ride. NAVYA accomplished all these tasks using internally developed tools. Each of these steps was essential in establishing the mobility service they aimed to provide.

2.5 NAVYA Ecosystem

The success of NAVYA was based not only on the driving technology alone but is the whole system that makes it happen. NAVYA delivers to cities a full solution and service to put in place automated shuttles, to train the PTO, and to allow daily operation. The service value chain is public transport authority (PTA), to public transport operator (PTO), and to NAVYA AV system shuttle and supervision. NAVYA provides the technology to the PTO that will help him to operate the daily service for the PTA or the city.

From NAVYA shuttles, the value chain extends to the automotive industry, encompassing the suppliers of each hardware element of the shuttle. These hardware suppliers provide various elements, including fundamental components such as static or mechanical parts, batteries, battery management systems, sensors, and low-level command control systems, among others. Many of these suppliers lack prior experience in the field of automated vehicles, and the requirement for achieving a high level of safety in each component has not yet become common practice in their factories or production processes. Notably, basic elements like sensors remain relatively fragile, and their maturity is not yet at a stage where failure rates and false positives are optimal. Nonetheless, they are considered safe enough for deployment in vehicles under specific controlled conditions.

The entire ecosystem is collectively learning about its needs. Sensor suppliers require real data from manufacturers to enhance their products, while manufacturers seek greater reliability from the sensors. Public transport operators (PTOs) demand high-quality service and uptime from shuttle suppliers. These demands cascade down to the suppliers of various systems and subsystems. However, it is challenging to achieve a high level of safety as defined by Automotive Safety Integrity Level (ASIL) based on the regulatory standards outlined in automotive ISO 26262 and still make it a mass-market product. Everyone involved is on a learning curve as they work together to address these challenges and refine the ecosystem. Therefor even if the ADS of the shuttle would be the best of bread technology on the market, if the hardware subsystem is not reliable, the driving experience is not smooth. If sensors are providing false positive, stating there is an obstacle where there is none or just a leaf falling, the ADS will trigger harsh brakes to avoid accidents. That is why perfect correlation between hardware and software should always be harmonized. Moreover, to sell these shuttles on the market, it should be balanced in an accessible eco-financial equation.

2.6 Legal Boundaries

In 2015, when NAVYA released its first commercial automated shuttles called The Autonom Shuttle^® ARMA, no legal institution in any country had a good knowledge of such vehicles and technology. None was ready to deliver the authorization to put them on open road. Each vehicle on open road should have a registration plate, and the vehicle should be homologated by the authorities based on the related legal framework of its category (M1, M2, etc.). As the NAVYA shuttle does not have steering wheels nor pedals, it is seen as an unclassified vehicle. Authorities took into account that it is an innovative vehicle, an experimental vehicle, and they delivered partial homologation with a registration plate limited in the time of testing, e.g., 1 or 2 years, if the vehicle can prove that it is safe enough to operate on open road under certain conditions.

NAVYA had to operate on a country-by-country basis, collaborating closely with government authorities or entities representing the government. Their goal was to assist these authorities in comprehending the technology, the vehicle, and how to address the challenges posed by the absence of traditional elements such as a steering wheel, pedals, left and right rearview mirrors, bidirectional driving, and the associated lighting systems. This cooperative effort aimed to ensure the smooth integration of their automated shuttle technology into each region’s regulatory framework and infrastructure.

In the absence of a standardized legal framework, the response to automated driving initiatives varied from one country to another. In nearly 20 countries, NAVYA took the pioneering step of being the first to introduce automated driving on open roads, where regulations and acceptance of this innovative technology were largely uncharted territory. In some, it took more than 1 year and half to get the registration plate, convincing the authorities that the AV shuttle is safe and secured for its innovation purpose on the dedicated road. Once the authorities gained the knowledge and established the procedures for partial homologation in collaboration with NAVYA, it paved the way for competitors to enter the country more swiftly. This scenario played out in countries like Denmark, as exemplified by the AVENUE project. Through the joint efforts of NAVYA, the local partner, AMobility (trademarked Holo), and AVENUE, we became the first to introduce an automated vehicle (AV) shuttle on open roads in Denmark, creating a precedent for others to follow.

3 Technology Improvements Through AVENUE

3.1 A Global View

One specificity of the AVENUE project was that the AV operations started from day one. Four different cities in Europe put in operation a total of ten automated shuttles and connected them to their public transport system. AV shuttles were based on the technology available in 2018 and the operational design domain that NAVYA shuttles could manage at that time. The PTOs used these deployments to advance their process, build the IT, learn from operational deployment, and prepare to scale and reach the target built by AVENUE to have a smooth and seamless on-demand door-to-door service. Although some regions had prior experience with AV shuttle experimentation, embracing this technology for regular use represented a significant advancement and a substantial request for progress in terms of regulatory and operational readiness. The other project partners were able to get a deep dive into the technology, to understand the current limits and potential, and to ideate to support the delivery of all targeted services of the project.

To put in place automated shuttles services, a global view of the technology was needed to understand the enhancements at all levels. The AV shuttle operational capabilities were the starting point. If the ODD of the site is above these capabilities, the service is due to fail. For example, the AV shuttle cannot go faster than 25 km per hour due to safety issues. If in the route defined by the operator there is a part at 50 km/h, even a short one, the shuttle will not be able to follow, and it is better to decide not to build the service.

3.2 NAVYA Software

Even though NAVYA is known as a manufacturer of automated shuttles, its core competencies are the ADS software and related technologies. NAVYA does conceive and build the vehicle and architect the whole set of sensors. The ADS, along with its related technologies, comprises the software suite that governs the automated shuttle, encompassing its full range of functionalities. It supports all aspects of the aforementioned system, from deploying to operating a mobility service using such automated shuttles. The software of NAVYA can be seen as three major parts: NavyaDrive^®, NavyaOperate^®, and public API (Fig. 3.4).

Fig. 3.4

NAVYA ADS overview

NavyaDrive^® is an embedded software installed on the computing system in each vehicle and allows to operate the vehicle. Internally, every new iteration of the NavyaDrive^® software is referred to as a “Setup,” which is linked to a specific version number.

NavyaOperate^® is an off-board software installed on the cloud and allows to supervise and manage vehicles. NAVYA supervision center team uses this software to supervise sites, as well as to take actions on the vehicle like for maintenance, analysis, and other non-driving actions.

Navya API is a software installed on a cloud system that allows a third party to receive and send information to one or several vehicles. No direct connectivity to vehicles is allowed to third parties due to security and cybersecurity issues.

3.3 Automotive New Release Process

The NavyaDrive^® software is the main intelligence embedded in the shuttle. It is installed in the computing systems and connected to sensors from one side and to mechanical hardware from the other. The connection to mechanical hardware manages low-level command control system. It sends and receives all information to fully control vehicle dynamics. As AV shuttles are not yet full in industrial production with volumes, the providers made some evolution of existing hardware and mechanical parts to allow the use in AV. New software updates of these parts are made regularly, correcting bugs or adding some features. For this purpose, shuttles, even if they are all based on the same ARMA model, do have differences between each other.

For each new software version of NavyaDrive^®, a comprehensive series of testing and validation procedures is essential to ensure compatibility and compliance with the various versions of the ARMA shuttles that have received authorization from regulatory authorities. This rigorous process is crucial to enable NavyaDrive^® to fully utilize the capabilities of any deployed ARMA shuttle, offering a safe and reliable automated driving solution. With each new release of the NavyaDrive^® software and setup, it is imperative to adhere to automotive standards during the testing and validation process. This ensures that no unintended regressions are introduced from previous versions, that all new features function as intended, and that the ARMA shuttles maintain the requisite safety standards to be deployed on the road. This meticulous process guarantees the reliability and safety of each software update. Adding a new feature or correcting even a small bug should go through this long process.

The AVENUE project played a crucial role in enhancing the standardization of this process. It ensured that all AVENUE shuttles adhered to the same service standards set for the project. Prior to this, NAVYA had been focused on promptly addressing client issues to maintain the continuity of their mobility services. In response to client-specific concerns, they often corrected bugs and released software updates after limited local testing, ensuring compatibility with the client’s hardware version. However, this approach led to significant segmentation of the NavyaDrive software and introduced some unintended issues for other clients. Standardizing the process through projects like AVENUE helped streamline software development and ensure consistency in service across all deployments.

Due to the growing diversity in hardware among different clients, the approach of addressing client-specific issues had led to an accumulation of software discrepancies. With the AVENUE project involving ten vehicles and four public transport operators (PTOs), each facing unique scenarios and operational design domains (ODDs), the testing and validation process had to be meticulously fine-tuned and adapted to ensure that each software setup could run seamlessly on the various ARMA shuttles under different conditions.

While this approach improved overall software consistency, it occasionally resulted in frustration for the PTOs. The time required to go through this thorough process and reintroduce the service with new features could lead to operational challenges and delays. However, the aim was to strike a balance between software consistency and efficient service delivery. Each software setup iteration is a heavy specification, design, and validation process that takes several months to make sure that no safety issues are released to final customers. Fig. 3.5 (NAVYA software update and validation process) describes the software specification tasks and the test and validation process.

Fig. 3.5

NAVYA software update and validation process

AVENUE operations commenced right from the project’s inception, with both software and hardware versions already at an advanced stage. Over the course of the 4.5 years of the AVENUE project and due to the continuous evolution of hardware providers, several updates to both hardware and software became necessary. The AVENUE shuttles initially utilized version 3.X of the software and eventually progressed to the final release at version 7.X, reflecting the dynamic nature of the technology and the ongoing development efforts within NAVYA’s research and development, driven by Agile development methodology. Above is a timeline of different major setups of NavyaDrive^® during the AVENUE project (Fig. 3.6).

Fig. 3.6

NAVYA software version setups

3.4 NavyaDrive^® Evolutions

3.4.1 The Operating System

In order to have a powerful and highly reactive driving system, NAVYA proceeded to refactoring the software architecture and migrate all from Windows to Linux operating system.

This migration was a major step to scale the NavyaDrive^® system to higher level, bring more features, provide higher stability and performance, and structure the upcoming setups. A better homogeneity was also established with other connected hardware parts of shuttles like routers and other elements.

Enhancements at all levels were made to allow agility and resilience to cybersecurity attacks. The cybersecurity was not simple to manage when you are system dependent like on Windows. With Linux NAVYA did enhance the signature of PC images, the control of the access, the governance of the profiles management, and the cyphering and certificate management.

Several PTOs and authority assessors quickly noticed these changes by comparing not only the driving but also by comparing the cybersecurity audits done before and after this update.

This phase was difficult at several levels starting from the refactoring of the software till the installation on the ARMA shuttles. It must be done with the presence of a NAVYA engineer on site for each shuttle. It needs to travel to the site with the computing system, replace the previous ones with the new ones, update the software parameters depending on each site specificities, and run blank test to ensure non-regression, and only then can it go back to normal service.

3.4.2 Over-the-Air Update

With the introduction of the over-the-air (OTA) feature in the latest software setup, the previously time-consuming and on-site installations for updates have become obsolete. This innovation enables software updates to be delivered through 3G or 4G networks, simplifying the process and enhancing safety compliance with EU regulations. Furthermore, OTA updates have accelerated the management capabilities of NAVYA’s remote supervision center. It allows for remote diagnostics, investigation of events, retrieval of logs, assistance to onboard operators in various situations, vehicle maintenance diagnostics, and more, making the operation and maintenance of AV shuttles significantly more efficient and streamlined.

Now for each new setup, NAVYA can activate this feature remotely, transfer the new software, and check the integrity and health of software before putting back the AV shuttle in the hands of the PTO. It can additionally customize the display screens to include personalized elements such as sounds, animated images, icons, and more for PTOs.

3.4.3 On-Demand Service

In AVENUE objectives, integrating on-demand/door-to-door system within the public transport service would be a game changer for urban mobility. This was a major technical challenge. A transformation of how the AV shuttle should conceive its route. The state of art of all AV shuttles at that time was to follow a virtual rail with a round trip. The shuttle starts in point A to finish at point D and then do the reverse from D to A. It should pass by bus station B and bus station C. This track is followed in a regular way according to a timetable, passing always by the same stops at the same time. Everything is done without any alterations or interaction with the passengers, except for stopping at the bus station.

In the AVENUE project, the on-demand system operates differently from traditional public transport services with fixed routes and timetables. Instead, any passenger can utilize a mobile application or an interactive bus stop to select any destination within a geofenced area. They can then wait until the AV shuttle arrives and pick them up. During the journey, the AV shuttle might make slight adjustments to its route to pick up another passenger heading in the same direction, even if they have a different final destination. This approach provides a more flexible and personalized transportation service.

This service requests several elements:

• An IT architecture where the IT system is connected to a fleet of vehicles covering the geofenced area

• An application, mobile, or interactive outdoor screen, connected to the system that a passenger can use request a travel

• A fleet management system to receive the travel orders, detect the nearest AV shuttle, and send the mission

• AV shuttles that can receive and operate the travel missions, sending back reel time information for tracking

NAVYA supported building the system architecture with technical partners and focused on developing the parts related to AV shuttle and external connectivity. Many steps were needed to build this flexible innovative system, described below.

The driving route structure was previously built with a fixed virtual rail (Fig. 3.7, fixed route/rail approach), called path, which was prepared by the commissioning team and with a dedicated map. The path is constructed with fixed stops. Once ready, the path is put inside the software of the AV shuttle. The driving system follows this path, guiding the shuttle accordingly. It executes the relevant driving instructions while also considering all new dynamic and static surrounding elements, such as pedestrians, traffic conditions, and other vehicles. If the shuttle gets out of its path, the embedded security system of the NavyaDrive^® will stop the shuttle. So if there is any change to hail another passenger out of the path, it will immediately stop the shuttle. The idea of changing dynamically the defined path during the operation was at the time inconceivable (Fig. 3.8).

Fig. 3.7

Fixed route/rail approach (Navya©)

Fig. 3.8

Fixed route as seen by the vehicle (Navya©)

The structure of the driving system was refactored to take into account step by step these changes. The first step was to allow the onboard operator to define the pickup/destination points of the travel within a fixed path by using the onboard tactical screen, dashboard user interface (DUI).

A mission management system was built to prepare and control the way the driving system should treat pickup/destination requests. Tracking the mission is taken in account to follow, send information, and acknowledge the status of the missions. A mission can be modified, canceled, or replaced with a new one by the system. However, the system does not permit changes to the current driving if they introduce any safety risks. Additional features were added to allow the management of the events for the ride. The event management system is described later. It allows to receive missions with additional actions like opening the door and rolling the ramp once arrived. Such flexibility will enhance the service to be used easily by disabled passenger or elderly.

The next step in the development process involved creating a comprehensive off-board system, which was hosted on a cloud-based platform. This system is designed to receive mission instructions from external sources, verify them, and then transmit them to the shuttles. This ensures better control over external connections and prevents any compromise to driving safety. It will facilitate the creation of new functionalities for end users from external systems. Therefore, all security, cybersecurity features, access control, and stack management should be done to prevent hacking the driving system.

A secured API was created for the partners so they can connect from external IT system. The connectivity structure involves third parties interfacing with the NAVYA cloud, which then acts as an intermediary to communicate with the AV shuttles. This configuration ensures that direct connections from third parties to the shuttles are not permitted, enhancing security and control over the communication channels.

The software architecture described above enables partners to efficiently manage the fleet of shuttles on one side and oversee passenger applications on the other. This dual functionality streamlines the operation and coordination of the AV shuttle service, providing control over both the vehicles and the passenger experience. This step allowed the first usage of on-demand service in a round trip and provided a first connectivity between the passenger application and the shuttle connected to the fleet management system for the AVENUE sites.

To extend the coverage across the entire geofenced area, enabling the shuttle to deviate from round trips and pick up passengers anywhere within this zone, it is essential to have the capability to dynamically construct the driving path. This necessitates an adaptation in NavyaDrive^®’s approach, shifting from predefined static paths to the construction of dynamic routes in real time. A representative site can serve as a testing ground to develop and refine this new path structure, allowing for greater flexibility in shuttle operations.

The site of Belle Idée in Switzerland (Fig. 3.9, the Belle-Idee site in Geneva) is the best use case to do so. It covers 38 acres, 9.2 km of roads, 24 buildings, 6 parking, and several points of interest, served by an extensive road mesh.

Fig. 3.9

The Belle-Idee site in Geneva

The ambition was that a passenger can request a pickup on his mobile app from any place to any destination within this area.

The whole area was mapped, and a clean and structural HD map was built for this purpose. NAVYA has always been innovative and active in developing its own technology of mapping and building a bench of advanced tools to do so. AVENUE helped enhancing such tools that allowed better knowledge and perception of the environment and brought a higher-level of localization.

Instead of building a path on a round trip, all the roads of the area were covered with a network of paths with different specificities as shown in Fig. 3.10 (Belle-idee routes and virtual stops).

Fig. 3.10

Belle-Idee routes and virtual stops

The NavyaDrive^® navigation system underwent a significant transformation. Instead of relying on a single static path to follow, it now possesses the capability to dynamically construct the path by assembling pieces of pre-mapped and prepared routes. This approach ensures that the navigation system can adapt and construct the optimal route for driving by piecing together predefined path segments as needed.

For the door-to-door service, instead of fixed stops, virtual stops were used. Almost 73 virtual stops (Fig. 3.10) were decided with the PTO to allow safe boarding and dropping of the passengers in the most probable points of interest that are usually used. No physical bus stop was built. When a user selects a pickoff, there is always a virtual stop next to his/her position, and he/she was advised to go to this spot for safety issues.

The mission management system also was enhanced to allow:

• Add/cancel missions dynamically

• Dynamic rerouting

• Refuse any mission that could lead to dangerous maneuvers like changing at the last minute and then turning left and right when already engaged

• Status and event management

A specific driving “Mode” in NavyaDrive^® system was built to allow AVENUE partners to connect and start engaging mission sending for the on-demand service.

The system was first tested, trained, and fine-tuned by starting the operations without passengers on board. Connection to local supervision center was installed. Once validated, it was open to all passengers (Fig. 3.11).

Fig. 3.11

The TPG control center, passenger app, and vehicle on the site

The Belle-Idée site was at the forefront of implementing the on-demand system and incorporating all the latest features with the most recent setup, marking a significant milestone during the final year of the AVENUE project. Other sites activated the on-demand service and put it in operation, each differently, and with different application or fleet management provider. All were using the same innovation brought by NAVYA, and all demonstrated that such integration is a game changer for public transport. In Lyon, the deployment site was at Parc Olympic de Lyon (Parc OL) (Fig. 3.12, Lyon Parc OL deployment site) and in Copenhagen at the Slagelse site (Fig. 3.13, Copenhagen Slagelse site).

Fig. 3.12

Lyon Parc OL deployment site (Keolis ©)

Fig. 3.13

Copenhagen Slagelse site (Holo ©)

3.4.4 V2X Traffic Light Management

The AVENUE project played a pivotal role in enhancing the driving experience by integrating vehicle-to-everything (V2X) technology. V2X technology was employed consistently, particularly at traffic light intersections, and occasionally in managing challenging turn situations. This technological integration improved overall traffic flow and maneuverability. As a reminder, in Europe traffic light goes from green to orange to red where only green allows to go on. In Europe the orange light appears just before the red light. Decisions are sometimes not easy to take when the vehicle is about to pass a green light and at the last minute the light changes to orange. Even normal drivers brakes aggressively when it happens, without stating those who accelerate instead. Previously NAVYA shuttles impulse harsh braking in such situations leading to uncomfortable situation for passenger and cars behind.

NAVYA built a decision diagram handling every phase of the traffic light, green to orange to red to green, taking into account the time to change to the next light, the traffic conditions, and the shuttle speed. All these elements together are considered to handle the right deceleration/acceleration profiles.

NAVYA’s experience with V2X technology revealed that errors could occur, and at times, V2X might not transmit the correct signals. Although such instances were rare, they were a reality. To address the critical issue of avoiding running a red light, NAVYA developed a redundant solution that leverages camera-based recognition to assess the status of traffic lights in various lighting conditions. This approach is built on artificial intelligence models and was tested during the AVENUE project but subsequently integrated into production to enhance safety and reliability.

To ensure a smooth and safe driving experience, the vehicle employs an estimation system to determine the time remaining before a traffic light changes to green or red. Depending on the situation, the shuttle can take one of the following two actions: Frist is anticipatory deceleration. If the light is changing from green to orange to red, the shuttle applies the appropriate deceleration profile in advance to optimize passenger comfort. It also informs the onboard operator about the approaching change in signal. Second is speed regulation. In cases where the light is transitioning from red to green, the vehicle regulates its speed to ensure a safe and controlled crossing of the intersection when the signal turns green. These actions are designed to enhance the overall driving experience, ensuring smooth and efficient intersection navigation.

In specific situation like flashing orange light on all the traffic lights of the intersection, meaning the driver should be careful and follow the normal priority rules if there is no sign, the ARMA shuttle stops, disengages, and informs the operator to take over: “traffic light is not manageable.” In this case, the operator can then either take the lead in manual driving to cross the intersection or wait until the shuttle receives readable information to cross safely. NAVYA added new diagnostics and errors code, on flashing orange lights and on missing V2X signals.

3.4.5 V2X Solution for Complex Situations

The site in France, near Lyon, NAVYA and the PTO (Keolis) faced two major driving difficulties. First the shuttle must go through an important roundabout where more than 40,000 vehicles drive in a day. And second the shuttle must do an important turn left where traffic is difficult. The solution facing these two situations was to use an interactive V2X system with external partners.

For the roundabout, active traffic lights with V2X are installed on each of its entry. Before arriving to the roundabout, the ARMA shuttle sends information to the traffic light controller and requests the priority on the traffic. The first traffic light in front of the shuttles will take control and will allow the entry of the shuttle in the roundabout. The other traffic lights on the roundabout will turn red progressively one after the other on each entry of the roundabout and will block other vehicles from getting in the path of the shuttle. Once the shuttle passes through, the traffic lights behind get back to green to unleash the traffic allowing other vehicles to get in. In this way the shuttle will have a fluid traffic and will ensure a safe and smooth ride.

As for the left turn difficulty, the ARMA shuttle will send information to the traffic light controller when it is near the left turn. The shuttle will take the left lane, and the traffic light will turn to green smoothly and to red for the vehicles on the other side of the road. Therefore, the road will be clear, and the shuttle can engage safely the left turn without stopping. This V2X management was the world’s first demonstration of how infrastructure and automated shuttles can actively interact to ensure secure traffic. Major developments were made on the NavyaDrive^® software and the low level stack of V2X to ensure compatibility and non-regression.

3.4.6 Driving Enhancement

Safe and smooth driving is the essence of driving activity, especially when it comes to automated driving. Many elements do interact and are associated to deliver such results including all hardware and software enhancements. As there are many done during AVENUE, it is difficult to go through all of them here, but we can get a peak on some.

The acceleration and the deceleration profiles are the major ones. Sudden and aggressive braking or acceleration can lead to accidents, cause discomfort for passengers due to jerky movements, and also lead to the disapproval from other road users and vehicles. As a result, the service can be stopped by the PTO if it does not deliver customer satisfaction at all levels, including comfort. The acceleration and deceleration profiles of the NavyaDrive^® were enhanced from a setup to another to provide smooth driving. The purpose of this algorithm was to ensure the right trade-off between safety and comfort for both passengers inside and other road users outside. Acceleration was boosted upon departure, and intersections were efficiently navigated with increased acceleration. As for the deceleration and braking, they are softer, and the harsh brakes and jerks were minimized. The braking distances were modified, depending on the deceleration profile criteria.

“Adaptive cruise control” was also added to the NavyaDrive^®. The ADS detects and evaluates the speed of the vehicle in front and adapts its acceleration and deceleration profiles accordingly. Adjustment of the speed of the shuttle on the speed of the vehicle in front was done automatically. This prevents the shuttle to always deploy different acceleration and deceleration profile or to do several braking. This is also valuable in traffic jams, a kind of traffic jam feature for ADAS.

Obstacle tracking system allows to anticipate the obstacle trajectory and to anticipate the entry of an obstacle in the path of the ARMA shuttle. This results in better behavior in several situations like on intersection and passenger crossing. For example, when the shuttle stops at a passenger crossing, without a traffic light, the shuttle tracks the movement of the passenger. It detects when the passenger is out of its path and when the passenger trajectory is getting farther away. So when the passenger passes the shuttle front lane, the shuttle gets back in movement carefully even if the passenger is always on the crossing. The same thing happens when the shuttle is driving and detects a moving obstacle on the left that is drifting and comes in the alert zone in front or around the shuttle.

Overtaking is a major difficulty for automated driving. Overtaking in all situations, expected or unexpected ones, and at any speed is not yet delivered by any ADS shuttle system by the time this document is written. Nevertheless, in some situations several OEMs have incorporated it in ADAS.

NAVYA during the AVENUE project made enhancement on this topic, and some overtaking was achieved. Overtaking of small objects (Fig. 3.14), objects parked half on the pavement and half on the road, and not well parked vehicles, were possible to avoid to stop or disengage. A maximal lateral distance of possible overtaking was set to keep safety first.

Fig. 3.14

Overtaking a static obstacle

Steering control was enhanced, and a new software was implemented to improve responsiveness of the steering.

Driving alignment (trajectory tracking): Maintaining precise trajectory and adhering to correct lines or curves are a fundamental aspect of manual driving. In the realm of automated driving, this aspect becomes even more critical and requires constant monitoring. It is important to recognize that deviations from the intended path can occur not only due to vehicle performance but also as a result of road conditions, such as potholes, uneven road surfaces, curves, road grip, and overall stability.

A trajectory tracker is of a high importance to keep safe driving, and NAVYA enhanced the trajectory tracker within the AVENUE project. This module improved accuracy of trajectory with higher control of the jerk. The jerk became a major measurement element of the comfort of the passenger and the smooth driving. Curve management is easy to build in software, but when hardware is involved, a 100% alignment is not simple. The hardware reactivity, curve angle, and speed while driving can lead to some misalignment. All drivers notice this curve’s lateral deviation upon inertia while driving.

NAVYA developed a more precise trajectory tracking allowing a smooth driving on curves and better radius alignment on the road, reducing lateral deviation. This enhanced the safety in curves and prevented incidents due to out of path alerts (Fig. 3.15).

Fig. 3.15

NAVYA trajectory tracker

Driving alignment (line/lane check): Transitions from “MANUAL DRIVING” mode to “AUTO DRIVING” are sometimes difficult to manage depending on its position toward the road. While starting a journey, the shuttle should be correctly aligned within its path, with a centimeter-wise precision. A human-machine interface (HMI) has been developed to assist the operator in this peculiar situation (Fig. 3.16).

Fig. 3.16

Median detection for confident departure feature

The perception viewer (Fig. 3.17) provides the operator with the exact position of the vehicle: lateral alignment and angular alignment error (offset angle and centimeter gap with the path).

Fig. 3.17

Perception viewer

Vehicle behavior improvements: As per the legal framework in effect during the AVENUE project, an onboard operator had to be present inside each AV shuttle when it was on open roads. The onboard operator must keep hands on the steering controls and remain standing to maintain an unobstructed view of the road ahead. This requirement was in accordance with the Vienna Convention, which mandates that every driving vehicle must have a driver on board. These regulations were implemented to ensure safety and align with established international driving standards. Until the change of this convention, which was done in 2022 as described in the chapter “Governance Impact Assessment” from Lionel Binz later in this book, in AV shuttle the driver is the onboard operator even if he does not do the driving operations.

The shuttle occasionally needs to disengage the ADS and allow the onboard safety operator to take over manual driving. This is necessary for situations where the ADS faces technological constraints, such as overtaking or reversing, or when the AV shuttle must exit the parking area and be positioned at the starting point of its automated route. Similarly, at the end of service, the shuttle may need to be manually driven back to the parking area or to a maintenance center. These maneuvers are to be done manually by the onboard operator with the steering controls. During manual driving, the ADS software and safety control system are deactivated. The onboard operator controls totally all the driving. Most of the time, incidents and minor accidents occur during such situations.

Based on the requests of several PTOs, a new driving mode was introduced to allow the onboard operator to drive safely the AV shuttle. The assisted manual driving (AMD) feature assists the operator during all manual driving. In a later iteration activation became optional, but once activated, speed limitation, obstacle detection, and Autonomous Emergency Braking were enabled. With this mode also, a buzzer inside the vehicle alerts the onboard operator and the passengers on the presence of an obstacle (Fig. 3.18).

Fig. 3.18

Information process to the onboard operator

In manual driving mode, certain onboard operators provided feedback regarding the risk of passengers falling within the shuttle due to the detection of nearby obstacles, prompting the shuttle to initiate abrupt stops. As a result, NAVYA ceased to restrict the vehicle’s speed in AMD mode, and braking procedures were adjusted accordingly. The decision to stop or continue driving remains with the onboard operator. Additionally, a new tone signal alerts the onboard operator to nearby obstacles, with frequency indicating proximity: the more frequent the tone, the closer the obstacle.

• Obstacle in the “close” zone = continuous tone signal

• Obstacle in the “middle” zone = very regular tone signal

• Obstacle in the “far” zone = irregular tone signal

Blinker management: A more operational blinker management was set up to allow better interaction with other vehicles and road users, especially during departure and arrival to stations. They stay activated on stations. Such feedback from the onboard operators and from AVENUE users allowed NAVYA to enhance such elements that cannot be easily seen in development or test and validation phases (Fig. 3.19).

Fig. 3.19

NAVYA blinking management

Blackbox improvements: ARMA shuttles have been recording all their activities for a long time. Still enhancements were made within AVENUE by reducing significantly the recording time of the data and adding more data to allow a clearer view and to replay certain situations afterward. Reducing time of recording reduces the use of the CPU power and keeps the compute power to the NavyaDrive^® system.

3.5 Supervision Improvements and NavyaOperate©

Automated public transport services, like automated metro, are connected to a control tower and a supervision center. All the vehicles are connected, facilitating the exchange of various types of information with the supervision center:

• Telematic information sent from the vehicles to the supervision center. This is the upstream where all details of the shuttles are viewed on a dashboard. It includes video stream inside and outside the vehicles, the position, speed, events, bus stops, unexpected stops, alerts, etc. All these information are needed to operate a daily service and are provided to the PTO either via the NavyaOperate^® system or via a partner API. It allows access to the shuttle states and events, current and passed, with time stamps and associated status codes.

• Technical information exchange in both upstream and downstream. This level of exchange is under the control of NAVYA supervision and maintenance center. It allows to access the NavyaDrive^® software of the shuttle, only under maintenance access and in stopped situation. The supervision center can get logs, diagnosis of the vehicle’s vital information, and hardware details. It also allows the maintenance and the supervision teams to update over-the-air the software of the shuttles with a downstream channel.

• Mission management. This two-way stream is the one developed for the AVENUE on-demand system that allows a third-party connection to the shuttles and to send missions or event triggering. The upstream provides acknowledgment and tracking of the missions.

Numerous enhancements where done during the AVENUE project to allow a complete integration of the shuttles in the AVENUE technical architecture and to provide the complete on-demand service.

The AVENUE project emphasized the importance of having a richer dataset to facilitate more straightforward analysis. This dataset encompassed various components, including historical diagrams, diagnostic information, detailed event logs; highly technical data such as velocity, obstacle events, as well as comprehensive descriptions; and maintenance-related details like temperature and heat data. Additionally, it involved real-time streaming, complete mission information, and the ability to remotely trigger various actions such as honking, buzzing, controlling lights, stops, doors, and ramps, and monitoring interior vehicle information. The dataset also encompassed GNSS signals, hit ratios, and other signals, together with detailed site-specific information and synchronization with onboard data like bus stops, stop names, and more. They were made available, making easier the analysis. Some of this information was also put in the record management system.

There were three supervision modes developed:

• “PARTNERS” supervision mode: this mode is reserved only for PTOs that have access to the Navya API. PTO can send missions to the ARMA shuttle.

• “STAND-ALONE” supervision mode: through this feature, some other supervision solutions can send commands to the vehicle: STOP, GO, and OPEN AND CLOSE DOORS.

• “METRO” supervision mode: the supervision can now ask the vehicle to leave a station remotely in “METRO” mode.

3.6 Navya API

The connectivity between the shuttles and remote applications of third parties is due to the standardized public API put in place and enhanced during AVENUE. It is a major outcome of AVENUE. The architecture developed goes through the NAVYA cloud system before connecting to the shuttles as described earlier. This is not only for security reasons but also for functional and technical performance across all sites, all connected partners, and all vehicles in operation or not, avoiding to impede the main task of the shuttle, which is driving smoothly and safely. No partner can connect directly to a shuttle. All must pass through the NAVYA cloud system.

The API evolved a lot during the AVENUE project, and the connectivity protocols were simplified and standardized across the 4.5 years of the project. At the beginning only the partner Bestmile was able to connect, send, and receive information with a dedicated closed protocol. To add a new connection, it would take several months with this architecture. NAVYA changed the protocol to RESTful HTTP API with WebSockets, and with this, any new partner was able to connect its application within days. That is how three new AVENUE partners were able to quickly connect their fleet management system (FMS) and mobile application applications at all sites.

Also a training and support program was put in place to allow the new partners to onboard quickly. Before any connection to the real AV fleet, a simulation program with the digital twin site was developed and installed to train the FMS and to be sure that all sending and receiving are working. Once the FMS is working correctly with the simulator, a real testing phase with people on site is applied to be sure of real integration, and continuity is done between simulation and the real site.

With the AVENUE project, this public API got through different versions bringing many features like:

• Change of the exchange protocol to have a standard secured HTTP and a new architecture of WebSocket and associated low-level services

• Security and certificate management

• Access per vehicle per several vehicles per site

• Access mode management

• Enrich data that should be collected by the NavyaDrive^® and then shared to the NAVYA cloud

• Video live streaming

• In-vehicle monitoring and information sharing

• Share event data

• Share control non-driving action on the shuttle (buzz, light, doors, ramp, sound, screens, etc.)

• Share diagnosis data (engine temperature, GNSS signals, lights and blinkers, batterie status, etc.)

• Mission management create, cancel, change, and track.

• Event triggering with mission management

Opening a new feature in the public API is more than just providing data through the connectivity and putting the API interface on the server. For each and every new feature or data delivered, development is needed on the NavyaDrive^® software in the shuttle to collect, clean, secure, and send the data. And for each development on the NavyaDrive^®, a new setup must be made, and the full test and validation process must be applied to be sure that there is no regression and no impact on the safety in operation.

The AVENUE project fed NAVYA with new features and ideas to enhance the API for fleet management systems, the supervision, and the shepherding features.

3.7 HMI and Experience Enhancement

3.7.1 Operator User Interface

Inside each shuttle, a screen is installed to replace the vehicle dashboard of a standard vehicle. This screen must respect legal regulations in place like for normal vehicle. and several mandatory elements are to be followed. This screen is called the dashboard user interface (DUI) (Fig. 3.20).

Fig. 3.20

NAVYA© DUI interface

Several functions are to be support by the DUI:

• Provide and display of mandatory visual and control elements to the safety driver

• Provide tools to the safety driver to perceive, analyze, ad control the shuttle

• Provide passengers tools to define which bus stops are requested and to follow the status of the trip

As described earlier, in some cases the onboard operator needs to manually drive the shuttle. The DUI is also to help him in these maneuvers and to provide a clear vision. It helps to show the exact perception view as the NavyaDrive^® is seeing it. Many advancements were done on the DUI based on the feedbacks provided by the PTOs of AVENUE during the operations. These AVENUE outcomes were important elements that could not be seen without feedback from the ground, from the people handling the real daily operation of the shuttles.

Below are some evolutions made on the DUI (example in Fig. 3.25):

• Authentication process.

• System status overview.

• Service mode control.

• Support-assisted manual driving mode.

• Diagnosis and analysis.

• Track and acknowledge any system event.

• Display help page.

• Alignment with regulation related to vehicle dashboard.

• A clear perception viewer indicating how NavyaDrive^® is perceiving the environment.

• Several elements were added to the perception viewer like: positioning of the environment and all objects seen by the vehicles sensors, obstacles that are blocking the vehicle advancement clearly highlighted when needed, and dynamic point visibility.

• Visual color enhancement to clearly indicate elements like lidar hit ratio and % level.

• Better positioning of some buttons used by the onboard operator like “Navigator,” “States,” “GO,” and “STOP” buttons.

• Door states displayed inside the GO button.

Asked by the PTO and for the comfort of the safety operators, a night mode of the DUI was added (Fig. 3.26). Three main possible choices are available: day mode, automatic mode (display of the night mode according to the time of the sunrise and sunset in the geographical area), and night mode.

Adding to the above, a new event management system was developed within NavyaDrive^® during the AVENUE project. This allows to track events for the supervision center. For a better management between the onboard operator on site and the supervision center, an event viewer was added to the DUI allowing him/her to better diagnose the vehicle status (Fig. 3.21). An interactive interface helps to deep dive in the events, and solutions are directly proposed delivering more independency of the onboard operator.

Fig. 3.21

NAVTYA© DUI day mode selection and vehicle status

Software and display changes in the “EVENTS” section (Fig. 3.22):

• 1. Icon indicating the severity of the event.

• 2. A short description of the event is automatically proposed.

• 3. A padlock indicates the impact of the event.

• 4. Detailed information by clicking on the event.

• 5. Procedure to be followed by the operator to solve the detected event.

Fig. 3.22

NAVYA© DUI event reporting

3.7.2 Event Triggering System

A real-time information with audio announcement was another outcome of AVENUE project. The AVENUE PTOs from different countries and with different languages are used to have inside their buses an information system for the passenger: visual indication of next stops, full itinerary display, audio announcements, etc.

NAVYA designed this software to have the capability to automatically initiate actions in response to event statuses. The event manager functions by alerting other systems when a particular event occurs, enabling those systems to execute corresponding actions. For instance, when the vehicle reaches a bus stop, the system can trigger the opening of doors and the deployment of the ramp. This real-time information display provides all the necessary passenger information, ensuring a smooth and well-informed experience.

Additionally, the system can also make adjustments to various supplementary elements within the vehicle and its interior. This includes modifications such as regulating ambient lighting, controlling low beams, making adjustments to the dashboard user interface (DUI), and deploying the ramp, among other features.

3.7.3 In-Vehicle Audio Announcements (UI)

The event triggering system is used also to automatically trigger audio announcement upon events, combining prerecorded messages as well as text-to-speech system. Audio announcements aim of course to assist disabled people. This can be done in multilanguage.

This system was first tested during the Dubai World Challenge for Self-Driving Transport for its first edition in 2019. Arabic and English languages were used to announce the bus stops and other information.

3.7.4 Interactive Interface for Passengers (UI)

Based on the advancements made on the DUI stated previously for the onboard operator and the real-time information system, the internal screen was enhanced for the passengers. It was designed to assist users along their journey:

• Visualize the top-view map of the operation site

• Visualize the real-time position of the vehicle

• Visualize the list of available stations

• Visualize the current itinerary

• Visualize the estimated time of arrival

• Select a desired station when on-demand service is enabled

• Select the desired interface language

• Trigger the departure/stop of the vehicle

• Display help page

In AVENUE, all shuttles were ARMA shuttles, and all visual enhancements stated were done for DUI screen that is used for the onboard operator and the passengers. One outcome of AVENUE is to add to the next generation of NAVYA shuttles, called EVO, and a screen dedicated to passenger information. EVO will have a large lateral display facing the entry of the shuttle above the lateral window. This display will be used for all internal and real-time information. Such system was conceived, produced, and tested in a prototype during the Dubai World Challenge stated above. All visual and sound information were displayed and triggered in Arabic and in English.

3.7.5 External Sound (UI)

Usually, NAVYA shuttles are operating in mixed traffic in cities and peri-urban areas where other vehicles are driving. Horns, sounds, and other external elements are of normal usage for these environments.

In AVENUE, the site of Belle-Idée is a hospital. Three ARMA shuttles are covering the whole area of the hospital. Cars, buses, and pedestrians are encountered, but all respect this quiet environment. NAVYA worked with the PTO to get all recommendations and adjustments needed to adapt the shuttles accordingly to this quiet environment. The features that can, in a way or another, disturb or make noise were deactivated, like busing when an obstacle is detected. These adjustments can be turned on and off easily on site.

3.7.6 External Screen and Human-Machine Interface (HMI)

The interaction with external road users is necessary. The ARMA shuttle can include two dedicated screens. Each displays information through the front and back window. The event management described above allows to trigger displays on these screens upon the perception of the environment or to interact with the road users. For example, the system can display animations when pedestrians are crossing ahead, indicating to them that the vehicle has detected them and is stopping. Alternatively, it can show information to vehicles behind, informing them of various actions of the ARMA shuttle vehicle, such as boarding or disembarking passengers, turning left or right, displaying a stop or yield sign in front of the shuttle, indicating proximity to a traffic light, signaling manual driving, or alerting to the detection of an obstacle just ahead, etc. (Fig. 3.23).

Fig. 3.23

Images and animated figures can be personalized for the PTO

3.8 Other Enhancements

3.8.1 Hardware Enhancement

NAVYA is enhancing the hardware constantly through commercial projects, upon customer feedback, and of course the needs of the AVENUE sites. AVENUE brought added value to this process as within AVENUE acceleration was requested in areas that were not explored before. The shuttle itself, mechanical parts, brakes and steering box, ECU and electronics, electrical network, sensor architecture, the safety concept, low-level control, etc., all were in constant enhancement, and AVENUE contributed to the assessment of needs. Several versions of ARMA shuttles went through this constant enhancement for AVENUE PTOs. It is difficult to highlight hardware enhancement here. Sometimes, small hardware details block the authorization to deploy a service on open road, and hardware enhancements must be made.

One specific issue was raised for the Copenhagen deployments. One mandatory point was requested and hardware changes in the structure of the vehicle so that the vehicles should handle the safety of wheelchairs according to the Danish law. In order to fix the wheelchairs, Q-straint has been installed in the shuttles. Q-straint works simply by having four mounted points in the floor, with seatbelts that can be hooked to the wheelchair. The seatbelts retract automatically and lock when necessary. When a wheelchair has to board the shuttle, the onboard operator mounts the four seatbelt heads on the floor. This means that the floor is empty when riding without a wheelchair. When carrying a wheelchair user, the three foldable seats cannot be used, and there is room for eight additional passengers (Fig. 3.24).

Fig. 3.24

Attaching a wheelchair (Holo©)

3.8.2 Mapping, Commissioning, and Tools

A whole set of tools, not directly related to driving the shuttles or supervising the operation, are needed and contribute a lot to deliver and AV service. The different levels of complexity of AVENUE sites contributed a lot in the enhancements of all these tools.

The MMS tool, related to the acquisition of the lidar cloud point and all environmental data of the road, was enhanced and got higher performance of lidars, sensors, and localization parts. A dedicated hardware and software tool was built to easily operate the system on site by the commissioning engineers on site.

HD mapping was developed to provide a higher level of precision in the perception of the environment and its knowledge. It brought higher localization precision while driving, and a notable enhancement was seen once the sites were mapped in HD mapping and updated. This enhancement is not only related to the acquisition of data. A full set of mapping tools and features were built by NAVYA engineers to manage this map, to rebuild this digital twin, to clean it, and to identify structural elements like driving zone, the road, etc.

Related to the commissioning, a dedicated path construction tool was built and enhanced to get the most of the HD map and to build the path with the easiest and the fastest way. Commissioning engineers can then quickly put a site in operation.

Also diagnosis tools accelerated the way the maintenance people can understand, correct, and put back shuttles in operation.

All these tools are fundamental to the AV ecosystem to deploy mobility service.

3.8.3 Additional Tool Enhancements

A complete set of additional tools was required for other tasks, such as creating the path and shuttle maintenance. Having more than 13 shuttles in the project across 6 cities greatly contributed to the enhancement of these tools.

Enhanced perception viewer was developed for commission engineers and maintenance team to help them in their task. It provides a 360° view of the surroundings, the same view as the shuttle sees it. Such feature reduced the time to detect and solve issues from within the shuttle. Several information were delivered like the lidar map, perception, wheel angles, obstacles display in different angles, etc.

It also allowed to calibrate sensors with a specific process to follow and be sure that sensors are perfectly aligned (Fig. 3.25).

Fig. 3.25

NAVYA© DUI 360 perception

Event registry, as stated previously, is logged, aggregated, and managed through this software feature.

Log management is made simpler to allow the supervision center to isolate and retrieve information faster. The supervision center and R&D had also easier tools to analyze the different diagrams retrieved.

Deploy, sync, and store the configuration files for faster commissioning and maintenance updates.

Tool mode was created for maintenance and deployment training purposes, with richer information (site name, vehicle technical details, hardware id, temperature of elements, etc.).

NavyaDrive^® boot indication guides users through various startup stages, conducting autochecks and diagnostics, and displaying relevant information on the proper functioning of each step (Fig. 3.26).

Fig. 3.26

NAVYA© booting status

4 Beyond Avenue

Developing profitable business models for automated shuttles requires substantial investment in research and development. Operational design domains (ODDs) vary, and with each step toward enhancing new features, there is a significant increase in the level of required resources and workload. Initially, many AVENUE public transport operators (PTOs) prioritized higher speeds, but as operational services were implemented, it became evident that speed was secondary to reliability and safety. Increasing speed affects all aspects of the software, sensors, and hardware components. It necessitates a more extensive capability to detect obstacles at greater distances, improve analysis and prediction, react more quickly, and ensure the reliability and safety of hardware at all levels.

Therefore, the hardware driving platform, an SAE L4 drive-by-wire system, is a crucial component. At present, no provider offers a fully homologated platform that closely aligns with automotive standards such as ISO 26262, and includes hardware redundancy features like brakes and steering. Additionally, this platform must incorporate a low-level security concept to ensure the vehicle can safely stop when the software system detects dangerous maneuvers or encounters issues that cause a loss of control.

Several universities, startups, and engineers demonstrated pieces of software that can control longitudinal and latitudinal maneuvers of a vehicle. However, all require a vehicle, and none can bring such software to market without having safe, homologated hardware. Producing such a vehicle cannot be done for very small series. It should be done in an industrial way and will require very high financial and resource investment that can be justified by high volumes (price/volume). All experts and OEM agree on this, and that is why, today, historical OEMs are working on incrementing features of the ADAS to go toward SAE L3 before getting to SAE L4.

Nevertheless, some are taking the challenge and are starting to invest in such drive-by-wire platform, and in CES 2022 some announced that they will have such vehicle by 2026. NAVYA has already started this process during the AVENUE project by building partnership with a bus OEM, Bluebus. This collaboration is made to build the first automated minbus SAE L4 ready and aligned with legal framework that is to be put in place. Since 2023 the hardware was out of Bluebus factory, and NAVYA connected the NavyaDrive^® software in a few weeks and made first automated L4 tests without a driver on board. This L4 bus will officially hit the market in 2025 and will scale the automated vehicles for public transport to another level.

5 Conclusion

AVENUE was a game changer in the AV landscape in Europe and toward all the AV community. It changed the way the community and decision-makers are seeing the automated vehicles.

I remember when, in 2019, NAVYA launched the Autonom^® Shuttle^® in Brussels, right under the European Commission’s headquarters, the Berlaymont. As the community experienced it, everyone saw the potential of this technology for public transport. It became clear that it is not just software, but an entire ecosystem capable of supporting the local socio-economic systems of EU countries.

This change paradigm from vehicle centric to ecosystem was needed, and the EU started to work on the legal framework. NAVYA was integrated in this legal work from the beginning, and the ADS act was released in 2022.

The work done in this area by the French government, where NAVYA was active since 2018, inspired and supported the ADS Act. The French government was the first to release in 2022 a full set of regulation on AV SAE L4, sowing the ecosystem dynamics. Just 1 month before, Germany released a part of its legal framework.

All are based on a double authorization as stated in other chapters: A type apporval of the automated vehicle to be put on the open road on one hand and an authorization of the AV service for a specific area with a specific ODD associated to the vehicle on the other. One delivered at a global level, the other at local level.

Before AVENUE all PTOs were waiting to receive their automated shuttles to put in place a service. But once they receive it, none was ready to put in place, and all stumbled on two elements: the legal authorization and how to manage the daily operations with their non-experienced team.

The role of public transport operators (PTOs) traditionally involves managing buses and drivers, overseeing driver competencies, handling client complaints, and maintaining schedules and frequencies. When traditional buses experience mechanical failures, their repair centers can manage these issues, as their personnel are trained for such maintenance. Similarly, transitioning from conventional fuel-powered buses to electric ones often requires minimal changes, as the drivers remain the same, with some necessary training.

However, the situation is markedly different when it comes to automated shuttles. Managing and maintaining automated shuttles present unique challenges that differ from traditional vehicles, necessitating specialized expertise and considerations.

Automated shuttles are digital objects, controlled by computers. At that time, they were experimental objects and were not industrialized with a high level of after-sales processes, local after-sales ecosystems in all countries, or trained repair centers. No driving competencies are needed to handle them but different expertise. Many public transport operators (PTOs) are not yet fully familiar with the digitalization of their profession and the related processes. Transitioning from traditional diesel-powered buses with fixed timetables to automated buses that don’t require traditional driving skills poses a significant challenge. The move towards fully digitalized and connected buses, which operate without fixed timetables and instead offer an on-demand service, represents a disruptive shift.

In this context, when an automated shuttle encounters issues, the onboard operator from the PTO must, rather than calling his maintenance technician, engage in a digital process with the NAVYA supervision center. This process involves providing a detailed description and obtaining logs. Unlike conventional buses that rely on mechanical maintenance, solutions for automated shuttles are predominantly digital in nature. They may entail immediate digital troubleshooting or involve software updates that follow a dedicated process.

For instance, there were cases where certain onboard operators believed they could address incidents by attempting to manage or access the shuttle’s computers. However, their efforts often resulted in exacerbating the issues rather than resolving them. In some instances, after attempting to independently rectify shuttle incidents, they resorted to manual driving to maneuver the shuttle. Yet, instead of moving forward, the bidirectional shuttle unexpectedly moved backward within the garage, ultimately colliding with a wall and causing damage to its lidars and sensors.

Also the authorities were not ready. They do not know what is this technological vehicle and in which legal “box” should they put it. They knew that it is game changer in the industry, but they wanted to take more time to understand it, to control all aspects of it to avoid any human or material damage, and to put all safety elements in a well-defined regulation.

The choice of the site, where the AV vehicle should drive, is an important part and role of the PTO. If the site’s ODD is not fully aligned with the automated shuttles’ capabilities, daily operations will struggle, and operational issues will start. It was seen in AVENUE as some of the sites that were decided on paper were not the most successful ones and brought many legal and daily complexity.

A shift in mindset was required among all partners, from public transport operators (PTO) to technical partners. AVENUE demonstrated that their “vehicle-centric” vision, solely focused on the level of driving autonomy of the shuttle, is not the optimal approach. Merely possessing an automated shuttle does not suffice to establish a safe and high-quality daily mobility service. The PTO must undertake several tasks: selecting suitable sites with the appropriate operational design domains (ODD) according to the shuttle capabilities, preparing and training staff, digitizing all systems, establishing connections with the supervision center, and developing operational processes. These processes, such as pre-operation checklists, procedures for starting and ending the service, incident management protocols, and passenger handling guidelines, are crucial components of a robust and sustainable AV service.

AVENUE allowed to put the focus on this ecosystem and all actions needed for that. A comprehensive vision was established, urging all partners to consider the “big picture” within urban planning and the public transport network, rather than focusing solely on AV technology. This encompassed questions on how AV shuttles should be integrated into Mobility as a Service (MaaS), identifying the external technologies required for integration, and evaluating in which use cases AV shuttles were more beneficial to citizens compared to traditional buses. AV shuttle should be, as any other transport vehicle, part of the whole mobility service of cities, bringing added value to all citizens.

Based on this knowledge and this “big picture” vision, the AVENUE team built a very ambitious follower project, ULTIMO project, to integrate AV in MaaS, in 3 major EU cities with at least 15 AV shuttles per site. This project of around 55 million Euros was the winning proposal of a major EU call and received funding from the European Commission in Horizon Europe program.

NAVYA significantly influenced all partners, the European community, cooperative connected and automated mobility (CCAM), competitors, researchers, and notably the European Commission. It demonstrated that even a smaller entity can make a difference and contribute to the establishment of an ecosystem that challenges industry giants. Many startups were created following NAVYA models. Big PTOs created dedicated departments, and today some actors of car giants created a strategic plan based on AV shuttles to produce in mass production around 2026.

We all hope that AV shuttles can become a regular part of our daily transportation, and every partner in AVENUE, along with NAVYA teams, hold strong confidence and take great pride in being part of this transformative disruption.