AIoT Pipelines

Abstract

Pipelines have become an important concept in many development organizations, especially from a DevOps perspective. This chapter introduces the concept of AIoT pipelines and discusses pipeline aggregations.

Pipelines have become an important concept in many development organizations, especially from a DevOps perspective. This chapter introduces the concept of AIoT pipelines and discusses pipeline aggregations.

1 Definition

There are a number of different definitions for the pipeline concept. On the technical level, a good example is the popular development support tool git, which provides a set of tools to allow flexible creation of pipelines to automate the continuous integration process. On the methodological level, for example, the Scaled Agile Framework (SAFe) introduces the concept of Continuous Delivery Pipelines (CDP) as the automation workflows and activities required to move a new piece of functionality from ideation to release. A SAFe pipeline includes Continuous Exploration (CE), Continuous Integration (CI), Continuous Deployment (CD), and Release on Demand. This makes sense in principle.

The Digital Playbook is also based on the concept of pipelines. An AIoT pipeline helps move a new functionality through the cycle from ideation and design to release, usually in a cyclic approach, meaning that the released functionality can enter the same pipeline at the beginning to be updated in a subsequent release. The assumption is that AIOT pipelines are usually bound to a particular AIoT technical platform, e.g., edge AI, edge SW, cloud AI, cloud SW, smartphone apps, etc. Each AIoT pipeline usually has an associated pipeline team with skills specific to the pipeline and the target platform (Fig. 20.1).

Fig. 20.1

AIoT pipeline – definition

2 Pipeline Aggregations

Due to the complexity of many AIoT initiatives, it can make sense to logically aggregate pipelines. This is something that many technical tools with built-in pipeline support such as git are providing out of the box. From the point of view of the target platform, the aggregation concept also makes sense. Take, for example, an edge pipeline that aggregates edge AI components, edge software components, and potentially even custom edge hardware into a higher-level edge component. On the organizational level, this can mean that a higher-level pipeline organization aggregates a number of pipeline teams. For example, the edge pipeline team consists of an edge AI and an edge software team.

This way of looking at an organization can be very helpful to manage complexity. It is important to note that careful alignment of the technical and organizational perspectives is required. Usually, it is best to create a 1:1 mapping between technical pipelines, target platforms and pipeline teams.

Figure 20.2 shows an edge pipeline that aggregates three pipelines, namely edge AI, edge HW and edge SW. The combined output of the three lower-level pipelines is combined into integrated edge components.

Fig. 20.2

AIoT pipelines aggregates

3 AIoT Pipelines & Feature-Driven Development

Technical pipelines are useful for managing and – at least partially – automating the creation of new functionalities within a single technology platform. However, many functional features in an AIoT system will require support from components on a number of different platforms. Take, for example, the function to activate a vacuum robot via the smartphone. This feature will require components on the smartphone, the cloud and the robot itself. Each of these platforms is managed by an individual pipeline. It is now important to orchestrate the development of the new feature across the different pipelines involved. This is best done by assigning new features to feature teams, which work across pipelines and pipeline teams. There are a number of different ways this can be done, e.g., by making the pipeline teams the permanent home of technology experts in a particular domain and then creating virtual team structures for the feature teams that get the required experts from the technical pipelines teams assigned for the duration of the development of the particular feature. Another approach can be to permanently establish the feature teams and look at the technical pipeline teams more as a loose grouping. Unfortunately, different technology stacks and cross-technology features tend to require dealing with some kind of organizational matrix structure, which must be addressed one way or another. There are some examples of how other organizations are looking at this, e.g., the famous Spotify model. The Digital Playbook does not make any assumptions about how this is addressed in detail but recommends the combination of pipelines/pipelines teams on the one hand, and features/features teams on the other (Fig. 20.3).

Fig. 20.3

AIoT features

Jan Bosch is Professor at Chalmers University and Director of the Software Center: There are two different ways in which you’re going to organize. In the component-based organizational model, you have the overall system architecture and assign teams to the different components and subsystems. The alternative model is a feature teams model; you have teams pick work items from the backlog. That team can then touch any component in the system and make all the changes they need to make to deliver their features. That is, in general, my preferred approach, but it is an important caveat. The companies that do this in an embedded systems context are associating the required skills typically with work items in the backlog. They say whatever team picks this up has to have at least these skills to deliver on this feature successfully. So it is not that any team can pick any work item.

4 Holistic AIoT DevOps

The pipeline concept must be closely aligned with DevOps. DevOps is a well-established set of practices that combine software development and IT operations. In more traditional organizations, these two functions used to be in different silos, which often caused severe problems and inefficiencies. DevOps focuses on removing these frictions between development and operations teams by ensuring that developer and operations experts are working in close alignment across the entire software development lifecycle, from coding to testing to deployment.

An AIoT initiative will have to look at DevOps beyond the more or less well-established DevOps for software. One reason is that AI development usually requires a different DevOps approach and organization. This is usually referred to as MLops. Another reason is that the highly distributed nature of an AIoT system usually requires that concepts such as Over the Air Updates be included, which is another complexity usually not found in cloud-centric DevOps organizations. All of these aspects will be addressed in the AIoT DevOps section in more detail.

In addition to the DevOps focused on continuous delivery of new features and functionalities, an AIoT organization will usually also need to look explicitly at security and potentially functional safety, as well as reliability and resilience. These different aspects will have to be examined through the cloud and edge software perspective, as well as the AI perspective. The Digital Playbook builds on existing concepts such as DevSecOps (an extension of DevOps to also cover security) to address these issues specifically from an AIoT point of view (Fig. 20.4).

Fig. 20.4

AIoT pipelines + DevOps

5 Managing Different Speeds of Development

One of the biggest challenges in most AIoT projects is managing the different speeds of development that can usually be found. For example, hardware and manufacturing-related topics usually move much slower (i.e., months) than software or AI development (weeks). In some cases, one might even have to deal with elements that change on a daily basis, e.g., automatically retrained AI models. To address this, one must carefully consider the organizational setup. Often, it can make sense to allow these different topics to evolve at their own speed, e.g., by allowing a different sprint regime for different pipelines that produce AIoT artifacts and components at different speeds. An overview is given in the figure following. Please note that there is often no straightforward answer for dealing with AIoT elements that require either very long or very short iterations. For example, for very slow moving elements, one can choose very long sprints. Alternatively, one can have all teams work with a similar spring cadence but allow the slower moving topics to deliver non-deployable artifacts, e.g., updated planning and design documents, etc. Similarly, for very fast moving elements the strict sprint cadence might be too rigid, so it could be better to allow them to be worked on and released ad hoc. For example, like automatically retrained AI models, this makes perfect sense since for an automated process no sprint planning seems required.

However, there is a key prerequisite for this to work: dependencies between artefacts and components from the different AIoT pipelines have to be carefully managed from a dependency point of view. In general, it is OK for fast moving artefacts to depend on slower moving artefacts, but not the other way around – otherwise the evolution of the fast moving artefacts will have a negative impact on the slower moving artefacts. These dependencies can be of a technical nature (e.g., call dependencies between software components, or deployment dependencies between hardware and software) or of a more organizational nature (e.g., procurement decisions). The technical dependencies and how to deal with them will be discussed in more detail in the Data/Functional Viewpoint of the Product/Solution Design. Finally, the Agile V-Model is introduced later as an option to manage product development teams in these types of situations (Fig. 20.5).

Fig. 20.5

Managing different speeds of development

Jan Bosch from Chalmers University and the Software Center: This is a key question: How do you do a release? There are companies in the earliest development stage that do heartbeat-based releases; every component releases every third or every fourth week at the end of the agile sprints. You release all the new versions of the components simultaneously, so that is one way. However, this requires a high level of coordination between the different groups who are building different subsystems in different parts of the system. This is why many companies aim to reach a state where continuous integration and testing of the overall system is so advanced that any of the components in the system can release at any point in time, as long as they have passed the test cases. Then, the teams can start to operate on different heartbeats. Some of the leading cloud companies are now releasing multiple times a day. This should also be the goal for an AIoT system: frequent releases, early validation, less focus on dependency management between different teams.