1 Introduction

Software has become an integral part of many products and services. Software development methodologies have evolved with a change in software technologies and customer needs (Solinski and Petersen 2016). During the 2000s, there has been a shift from traditional development methods towards agile methods to respond to fast-changing environments and volatile customer requirements (Petersen and Wohlin 2010). Agile methods have been shown to be more flexible in responding to changes in requirements than traditional methods (Erickson et al. 2005). Along with increased flexibility, agile adoption has provided other benefits such as improved productivity, predictability, and team morale (Vijayasarathy and Turk 2008).

The agile methods were originally developed for small and co-located teams. However, the potential benefits provided and their popularity in small-scale settings have motivated many large organizations to adopt their use, creating a need for scaling the methods to larger contexts. Scaling agile methods poses several challenges related, e.g., to communication, and coordination, as well as the integration of non-development units like marketing, and human resources (Dikert et al. 2016). As an aid in solving scaling issues by providing "blueprints" of what agile at scale can look like, practitioners and consultants have introduced several scaling frameworks. These include the Scaled Agile Framework (SAFe) 2018d, Large Scale Scrum (LeSS) 2016, and Disciplined Agile Delivery (DAD) 2012.

The adoption of agile methods and agile scaling frameworks has increased significantly in recent years among large organizations (Version One 2019). According to the latest non-scientific survey conducted by Digital.ai (earlier known as VersionOne) Version One (2019), SAFe was found to be the most popular framework, with 37 percent of respondent organizations reporting adopting it. A scientific survey on software development approaches also indicated the predominance of SAFe over LeSS and DAD Kuhrmann et al (2017). Both practitioners and researchers have indicated a strong interest in understanding SAFe usage and adoption in practice (Moe et al. 2016; Moe and Dingsøyr 2017; Conboy and Carroll N 2019; Ciancarini et al. 2022).

A multi-vocal literature review reported a lack of scientific research in the domain of SAFe (Putta et al. 2018). According to it, most of the published information on SAFe comes from grey literature sources, with the largest number of reports being provided by the organization behind the framework, indicating a significant risk of publication bias. Thus, not surprisingly, the grey literature was inclined to focus more on presenting the benefits of SAFe rather than the challenges. More recent studies (Conboy and Carroll N 2019; Uludağ et al. 2022; Edison et al. 2021) confirm this lack of sound empirical research on SAFe adoption. Given its importance in practice, we think that significantly more attention should be given to independent, in-depth studies of SAFe adoption to help provide solid advice for practitioners considering its adoption and researchers wanting to understand its role in implementing agile at scale.

In this paper, we help fill this gap in the current research by presenting a case study of the SAFe transformation in a large financial corporation. We study the reasons for the transformation, the activities in the transformation process, and the challenges and initial benefits received.

We described a small part of the transformation process in a conference paper describing the formation of the Agile Release Trains (ARTs) and the related challenges (Putta et al. 2019). However, this paper discusses the whole transformation in-depth, including the formation of the release trains, and contains details not included in the previous conference publication.

The structure of the paper is as follows: The next section presents information on SAFe and summarizes the primary studies from both grey literature and scientific literature on SAFe. The third section presents the research methodology. The fourth section contains our results. The fifth section presents the discussions and limitations. Finally, the last section presents the conclusions and suggestions for future work.

2 Background

In this section, we first present the importance of studying large-scale agile transformations. Next, we give an overview of the Scaled Agile Framework, SAFe, by explaining the different levels, key practices, and roles and its roadmap of implementation. Next, we discuss existing studies of the framework and its adoption.

2.1 Large-scale Agile Development

Since the creation of the agile manifesto in the year 2000, agile approaches to software development have more or less become the norm in both the private and public sectors. Reported benefits of agile are many, e.g., higher productivity, faster feedback, and better quality (Petersen and Wohlin 2010). The principles and practices of agile are well-suited for small and co-located teams, as this was the initial focus of the methods. However, due to the benefits reported in small organizations, larger organizations started to adopt agile approaches in larger development undertakings as well. These large projects are often critical for companies or even nations (Dingsøyr T and Moe NB 2013).

The use of agile at scale poses several challenges as it involves a larger number of teams, which often leads to coordination issues and overhead (Dingsøyr et al. 2018. Having several teams also reduces communication effectiveness (Dingsøyr et al. 2014). Adopting agile is as much, perhaps even more, about changing the mindset in an organization rather than simply applying agile practices. Achieving such an "agile mindset" is often challenging (Klünder et al. 2018; Sekitoleko et al. 2014), and it has been argued that agile should be understood as the culture of an organization rather than a methodology or tool for change (Mucambe et al. 2019).

Several factors have influenced the success of agile methods, and the most salient success factors identified are management support, choosing and customizing the agile model, and investing in training and coaching (Dikert et al. 2016).

Fig. 1
figure 1

The Scaled Agile Framework, Version 5.0 Scaled Agile Inc (2018d)

Full size image

The adoption of agile scaling frameworks has increased rapidly in recent years (Karvonen et al. 2018). Many researchers and practitioners have reflected on the need for more research in understanding the usage and adoption of these frameworks (Mucambe et al. 2019; Dingsøyr et al. 2018). Especially, they have highlighted the need for understanding SAFe transformations (Dingsøyr et al. 2018; Moe and Dingsøyr 2017). A multi-vocal literature review (Putta et al. 2018) also reported the need for more research-based studies, as most of the studies on SAFe come from the official website of SAFe. These studies are biased towards presenting only positive elements of SAFe to promote its usage (Putta et al. 2018). Recent years have seen a growth in the number of studies published on SAFe (Beecham et al. 2020; Gustavsson 2018, 2020), but there is still a lack of studies looking in detail on the adoption of the framework. One explaining factor could be the complexity of the framework (Cram 2019; Dingsøyr et al. 2018), as this can make it very resource-consuming to study such implementations in-depth.

2.2 The Scaled Agile Framework (SAFe)

The Scaled Agile Framework (SAFe) Scaled Agile Inc (2018d) is a commercial framework for scaling agile to large contexts, with various courses and consulting services available through its owner, Scaled Agile Inc., and its partners. The framework was designed to scale agile and lean practices to the enterprise level (Uludağ et al. 2021). The framework incorporates practices from agile, lean, extreme programming (XP), and Scrum. The framework has four different configurations: essential SAFe, large solution SAFe, portfolio SAFe, and Full SAFe, see Fig. 1.

The essential SAFe contains a minimal set of roles, events, and artifacts to deliver business solutions. It consists of the team level, with agile teams working using Scrum or ScrumXP practices. The teams are capable of defining, building, testing, and deploying a particular feature in a given iteration. They are self-managing, self-organizing, and empowered. Each team has a Scrum Master, who facilitates the team’s work and ensures that the team follows the processes. The Product Owner handles the team’s backlog and is responsible for prioritization of user stories, and ensures they are well described and understood.

The teams are synchronized through agile release trains (ARTs). A release train is a virtual organization that aligns the people and the work around a common mission to deliver solutions incrementally by using time-boxed program increments (PIs). The PIs typically range between 8 and 12 weeks. One key role of this configuration is the release train engineer (RTE), who is a servant leader responsible for facilitating the key events. Other roles include the business owner, who takes responsibility for the technical and business solution developed by the release trains, product management, with responsibility for communicating the needs of the customers and for defining the program backlog. The key events at this level are PI plannings, the system demo, and Inspect and Adapt. The PI planning is an event that takes place at the beginning of every product increment, where all teams in release trains jointly plan for objectives that need to be accomplished during the next increment. The event typically lasts for two days, and the purpose is to align the teams to a common vision Scaled Agile Inc (2018c).

The large solution SAFe Scaled agile Inc (2022) is intended for organizations that aim to build large and complex solutions beyond the scope of a single release train.

The portfolio SAFe configuration aligns strategic objectives to portfolio execution by organizing the enterprise around the flow of value through one or more value streams. A value stream is defined as a series of steps taken in order to build a solution that generates continuous customer value. SAFe identifies two types of value streams: operational and development. Operational value streams consist of the people and the systems that provide business solutions to the end-user. Development value streams support operational value streams by developing new business solutions. Important related artifacts are: business epics that reflect new capabilities and enabler epics, which reflect architectural and technological initiatives. These epics are managed by portfolio Kanban.

Key roles in this configuration are the Epic Owners who help coordinate portfolio epics through the portfolio Kanban system. The Lean Portfolio Management is a group of individuals who are responsible for strategy and investment funding, agile portfolio operations, and lean governance; and an Enterprise Architect who helps towards optimizing the portfolio. The key events in this configuration are portfolio sync and participatory budgeting.

The full SAFe configuration consists of all three previously mentioned configurations.

SAFe also incorporates other practices like Communities of Practice (CoPs), shared services, and the system team. CoPs are groups of people with a common interest in a specific domain. They collaborate to share information and improve their skills on a regular cadence. The system team is a special agile team that helps in continuous delivery. Shared services are special part-time roles or services which support a release train.

SAFe provides a roadmap that helps guide the organizations by providing 12 steps for what it claims to be a successful implementation of the framework. The 12 steps are:

  1. 1.

    Reaching the Tipping Point. Identifying a need for change in the organization. There are two possible conditions for this change: (i) organization reaching a burning point, where the existing means of development fail to produce the needed end product or solution, (ii) proactive leadership, where leaders are motivated to drive an organizational change.

  2. 2.

    Train Lean-Agile Change Agents. Training change agents with SPC, SAFe program consultants course, which lasts four days and provides knowledge to guide the change.

  3. 3.

    Train Executives, Managers, and Leaders. Train leadership through participation in a leading SAFe training, a two-day course. Contains the fundamentals of SAFe, e.g., SAFe principles, and core values. This is claimed to help sponsor and support the implementation of SAFe.

  4. 4.

    Create a Lean-Agile Center of Excellence. Create an organizational entity guaranteeing explicit time and resources for guiding and supporting the transformation process.

  5. 5.

    Identify Value Streams and ARTs. Identifying the operational value streams and the people and systems that support these value streams. Identify the development value streams that contain these people and systems. Define agile release trains that realize the development value streams.

  6. 6.

    Create the Implementation Plan. Key stakeholders plan for designing the new agile release trains.

  7. 7.

    Prepare for ART Launch. Conduct activities necessary for launching the ARTs, which include setting the launch dates and cadence for the program calendar, education, scrum masters, product owners, and preparing program backlogs.

  8. 8.

    Train Teams and Launch the ARTs Train the teams are trained in SAFe, and launch the ARTs.

  9. 9.

    Coach ART Execution Coach teams on iteration planning, daily stand-ups, ART sync, PO sync, Scrum of Scrums, and PO sync

  10. 10.

    Launch More ARTs and Value Streams. Roll out additional release trains based on experiences and lessons learned from the first one. Adopt the SAFe solution level if needed.

  11. 11.

    Extend to the Portfolio. Create a lean and agile program management office responsible for lean budgeting, relentless improvement, and adopting leaner approaches in contracts and with third-party suppliers

  12. 12.

    Sustain and Improve. Relentlessly improved the process. Focus on the DevOps journey, agile architecture, and agile technical practices.

2.3 Related Work

Despite its overall popularity, there are few research studies looking at the SAFe framework.

Beecham et al. (2020), conducted a longitudinal case study in a distributed organization using SAFe and found that the framework could mitigate most of the software project risks encountered in global software development.

Razzak et al. (2017, 2018) conducted a longitudinal case study to find the role of SAFe practices at the team level by using SAFe team-self assessment. During the initial stages of SAFe adoption, the case struggled with PIs and technical health. However, over time they have a significant improvement in all the team-level practices.

Turetken et al. (2017) designed a maturity model to guide software organizations in defining a roadmap for adopting SAFe. This model was evaluated by conducting a case study.

Several studies focused on specific SAFe practices (e.g., PI plannings Foo et al. 2020) and roles (e.g., the role of product owner in SAFe Remta et al. 2020) and inter-team coordination mechanisms Gustavsson (2019, 2020, 2019). For example, Gustavsson (2019, conducted a multiple-case study to identify how companies tailored the PI planning. In Gustavsson (2020), a multiple-case study was conducted to investigate how dependencies were visualized on the program board during PI planning.

None of the above-mentioned studies described the transformation process, the reasons for selecting the framework, the achieved benefits, and or adoption challenges in detail. Studies either described the implementation of a particular practice of SAFe or investigated the effect of practices, or assessed the current SAFe practices.

Only a handful of studies have partially described the above four dimensions of SAFe adoption, the information from these studies is described in the next sections.

2.3.1 Reasons for Adopting SAFe

We could identify only a small number of studies that discussed the reasons for adopting SAFe, and none of them provided in-depth information about them. The reasons were typically mentioned only as a part of the case organization description.

The reasons for SAFe adoption mentioned in the case study of Comptel Paasivaara (2017) were: to improve collaboration between management and IT, faster reaction to changes, and to overcome the delays in spreading the features across teams. The reasons mentioned by the case study from SimCorp Pries-Heje and Krohn (2017) were: to solve the challenges related to visibility, poor morale of employees, complexity, and under-estimated dependencies.

A survey study on why organizations adopt scaling frameworks (Putta et al. 2021) identified the fact that SAFe is widely adopted, well-defined and clearly documented, and well-supported by coaching, training, and guidance as important reasons for its adoption compared to other frameworks (Putta et al. 2021). The same study also found that general goals, such as being able to scale agile to a larger number of teams and remain competitive in the market, were common generic reasons for organizations to adopt agile scaling frameworks.

Thus, the reasons for adopting SAFe can be similar to the reasons for adopting any scaling framework or even agile without any scaling framework, either in the form of goals for improving the present state of the organization or to mitigate the existing challenges. Another category of reasons is related to the particular characteristics of the framework and its ecosystem.

2.3.2 SAFe Transformation Process

To our knowledge, only two studies provided information related to the SAFe transformation process Paasivaara (2017); Pries-Heje and Krohn (2017).

In both aforementioned cases, the adoption of SAFe was a top-down approach, i.e., from product management to the teams. The transformation in Comptel proceeded by providing SAFe courses, adopting change agents and external coaches supporting the transformation, and introducing PI planning events. Both SimCorp Pries-Heje and Krohn (2017) and Comptel Paasivaara (2017) mentioned using external consultants to help with the transformation.

In-depth information on how the SAFe transformation proceeded is not reported in the literature. Interestingly, while the framework itself prescribes a 12-step adoption process, we found no studies referencing or analyzing this process.

2.3.3 Benefits of SAFe

The most commonly mentioned benefits reported by a multivocal literature (MLR) review (Putta et al. 2018) were improvements in the following: transparency, alignment, productivity, predictability, and time to market (Putta et al. 2018). Another MLR reported alignment, continuous improvement, frequent and predictable deliveries, and improved code quality as the most commonly mentioned benefits (Ciancarini et al. 2022). A recent survey on SAFe reported improved transparency, as well as collaboration and dependency management between agile teams as common benefits of SAFe (Putta et al. 2021). The benefits of SAFe are largely similar to the benefits of agile implementations in general. However, information about what practices actually help achieve those benefits would be interesting (Putta et al. 2018). Given the current reports, it is unfortunately not possible to separate the benefits of SAFe from the general benefits of implementing agile, thus leaving it difficult to assess the value of the actual framework.

2.3.4 Challenges of Adopting SAFe

The most commonly mentioned challenges of adopting SAFe reported by the SAFe MLR were: change resistance, challenges with the first PI planning session, moving away from agile, challenges with staffing the roles, and controversies with the framework (Putta et al. 2018). Another MLR mentioned the following challenges with SAFe: planning, the complexity of the framework, challenges with continuous improvement, hierarchical decision-making, resistance to change, lack of autonomy, and communication with non-agile parts of the organization (Ciancarini et al. 2022). The most often experienced challenges of SAFe, according to a survey (Putta et al. 2021), were organizational politics, difficulties in establishing an agile mindset, change resistance, and team formation challenges. A comparison of the challenges experienced by users of different frameworks (Safonova et al. 2023), showed that the challenges difficulties in establishing an agile mindset, the framework brings additional overhead, and uncertainty with respect to middle management’s role in agile were more prevalent for SAFe than for other frameworks. A recent case study on SAFe challenges also indicated similar challenges of SAFe adoption in the banking industry (Tengstrand et al. 2021; Kowalczyk et al. 2022).

Many practitioners and researchers are keen on understanding how an organization transforms to agile by using SAFe (Moe et al. 2016; Moe and Dingsøyr 2017). Grey literature has partially covered the gap by explaining the benefits, the challenges, the transformation process, and the reasons. However, these studies are inclined towards presenting benefits rather than challenges to market the framework. Extensive information on challenges is missing from both grey literature and scientific literature (Putta et al. 2018). This further emphasizes the need for objectively studying the benefits and challenges of SAFe transformation to provide scientific evidence regarding SAFe implementations across industries.

3 Research Methodology

In this section, we discuss the research methodology employed. First, we present the research goals and research questions. Next, we describe the case organization, after which we describe the data collection and analysis. Finally, we explain how we validated the results of the study.

3.1 Research Questions and Goals

The goal of this study was to understand the SAFe transformation process in the case organization. We posed the following research questions.

  1. RQ1:

    What were the reasons for adopting SAFe?

  2. RQ2:

    What activities were involved in the SAFe adoption?

  3. RQ3:

    What were the benefits of the SAFe adoption?

  4. RQ4:

    What challenges were faced when adopting SAFe?

3.2 Case Study Method

We conducted the study as an exploratory case study. According to Yin, a case study is defined as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between the phenomenon and its context are not clearly evident." Yin (2009)

We opted to conduct a case study, as it is a suitable method for investigating a contemporary phenomenon, i.e., a SAFe transformation in an organizational context (Yin 2009). A case study provides rich and detailed data that can be used to understand a contemporary phenomenon, like an ongoing SAFe transformation. The case study method allows us to examine such a complex phenomenon in a realistic setting, as in our case, the case organization, and to provide a deep understanding of the context in which a phenomenon occurs. The method also allows researchers to examine the complex interaction between different factors that may be affecting the phenomenon being studied, which can provide a more holistic understanding of the situation.

We followed the guidelines developed by Yin (2009) for exploratory case studies and used purposeful case selection (Patton 2002).

The case was purposefully selected for the following four reasons: 1. The case was within the financial sector, the industry sector in which the largest number of SAFe adoptions have been reported (Putta et al. 2018). This gave a possibility to explore the reasons, challenges, transformation process, and benefits of SAFe adoption in a domain with a high interest in the framework; 2. The SAFe implementation at the organization had reached a certain level of maturity, with, e.g., multiple agile release trains implemented and continuous improvement and expansion of the practices of SAFe in place. 3. The case organization was one of the largest traditional corporations in Denmark adopting SAFe, and 4. The organization was accessible to the researchers and allowed us to conduct the study.

3.3 Case Organization

The case organization, PFA, was established in 1917. PFA is a large financial company specializing in pension and insurance products. At the time of the data collection, it employed a total of 1300 persons, of which around 300 were in software development, organized into 32 teams. Software development was distributed between two sites: Denmark and Poland. The main part of the development took place in Denmark, with support from sub-contractors in Poland, hired as the organization could not find enough people with the right skill set in Denmark. Nearshoring from Poland was chosen due to the close geographic and cultural distances to Denmark.

Before the transformation, the organization had adopted the PRINCE 2 (AXELOS Limited Ed. 2017) process model. Using this model, it took from six months to three years to deliver a project to the internal customers. The organization was organized functionally, with different areas working in their own silos led by directors "owning" the resources under them. The organization had a hierarchical structure and used a command and control leadership style.

In 2015, the organization hired a new CEO, who introduced a modern leadership style. At that time, they had 14 different software development models, as well as a portfolio with several projects. A strategy to change the traditional mindset in the organization was developed, but people did not embrace it due to challenges such as long queues and capacity allocation. The organization started with a Kanban initiative in 2016, which introduced the concept of lean projects to optimize the work in the existing projects.

Around the same time, a group of 20 persons working on front-end development started adopting agile practices. Later on, they adopted SAFe by forming a pilot agile release train. Finally, they had four release trainsFootnote 1, namely, DCE ART digital and customer experience (50 persons, seven teams), DBI ART: data and business intelligence (40 persons, five teams), IP ART: insurance and products (120 persons, 11 teams) and DM ART: digitalization and management (80 people, nine teams), see Fig. 4.

3.4 Data Collection

We conducted twenty-four interviews with twenty-seven people during three months (Feb-Apr 2018), following the guidelines in Patton (2002). We interviewed people in several roles from different levels of the organization, e.g., the team level and the program level. The list of interviewees, along with their background and duration of the interview, is shown in Table 1. The different stages of the interview process are explained below.

Interview Guide

We prepared an interview guide consisting of a list of themes and related questions. The primary focus areas were reasons for transformation, transformation process, achieved benefits, identified success factors, and challenges in adopting SAFe. All interviews covered the same themes mentioned above, but we rephrased some questions and added a few based on experiences from the first few interviews. In the first interviews, we also asked about the overall organizational history, the background of the transformation, and the organizational structure.

Selection of the Interviewees

The first two authors participated in the eighth PI planning session of the first release train in February 2018 to identify suitable persons for the interviews. A list of potential candidates for interviews was made during this session. After the session, this list was verified by one of the core members of the transformation team, who suggested additional people who could provide information on the transformation process. Some interviewees also suggested other colleagues for interviews. Participation was voluntary; some people we contacted also refused to participate, e.g., for lack of time. We aimed to have interviewees from different roles and different levels of the organization, e.g., developers and managers, to validate our findings through data source triangulation (Jick 1979) and to understand different perspectives on the transformation. We purposefully interviewed people only from the two longest-running agile release trains, as we wanted to interview people with more experience in agile transformation. The two later release trains were only newly formed at the time of the interviews and were still in their formative stages.

Conducting the Interviews

All the interviews were conducted face-to-face at the case organization. Two interviewers were present during all the interviews. The interviews were semi-structured. One interviewer asked questions, and the other took detailed notes and asked clarifying questions.

One interview had two interviewees, as the interviewee brought another person to give more in-depth information, and another interview had three interviewees, as the developers located in Poland had only a limited free timeslot available during a PI planning day. Thus, we decided to interview all three willing to be interviewed at the same time to get more viewpoints instead of being able to interview only one of them. All other interviews had just one interviewee.

Table 1 Overview of interviewees
Full size table

Transcription and Audio Recording

All the interviews were audio recorded, with the interviewees’ consent. All the recordings were transcribed by professional transcribers. We verified and corrected the transcripts by listening to the recordings and comparing them with the transcripts to ensure the correctness of the data. Any missing words were added, and misunderstandings were corrected.

3.5 Data Analysis

All the transcripts were imported into the Nvivo 12 (QSR International 2011) tool for qualitative data analysis. The first author coded the transcriptions, following the guidelines in Corbin and Strauss (2008). We developed a preliminary set of codes based on our research questions, after which we started coding by marking interesting text segments and labeling them with open codes under the main predefined categories. These open codes were grouped into code categories based on conceptual similarities and differences observed. A snippet from the transcript is presented below to describe the coding process.

Q: What were the reasons for SAFe adoption at PFA?

  1. 1.

    “You don’t really get the full value of the developers. So I think that’s, has been reading that correctly and saying, perhaps we need to turn that around, so basically, so we empower the people who actually do the code to support more production and more quality, basically, in what we do. And also shorter iterations. I think those are some of the motivations, at least."

  2. 2.

    “But we had a problem back then that from an idea was created until you have something that you could put out, it took at least one and a half year. And that’s a very long reaction time if you want to be market-relevant. So that was one of the reasons we were trying to alter the way we were delivering things and developing things because it simply took too long."

In snippet 1, the interviewee says to have shorter iterations, improve productivity and quality as the goals of adopting SAFe. In snippet 2, the interviewee mentions to overcome the long development cycle by adopting SAFe. In both snippets, the interviewees address shorter time to market, in the first one, as a goal and in the second one to overcome the existing challenge. Therefore, both these snippets were coded as “shorter time to market" grouped under the higher code category, “reasons to adopt SAFe". Also, the first snippet was coded with the following open codes: improving quality and improving productivity.

During the coding process, there were several discussions between the authors regarding the naming of the codes and categories. Based on the discussions, some codes were renamed, and others were regrouped under other categories. After that, a list of research themes and respective codes under them, along with quotes from the interviews, was prepared. The list of codes and categories used for presenting the results in this paper are given in Table 2. Finally, the codes and categories under each research question are summarized in the results section.

Table 2 List of codes and categories
Full size table

3.6 Data Validation

After the analysis phase, we organized a feedback session at the case organization. We invited all interviewees by email, and a message was sent to all the people working in the company about the feedback session. Fifteen people attended the presentation, the majority of which were our interviewees. In the presentation, we discussed our main findings using anonymized interview quotes.

In the session, the participants agreed with our findings, and we received no correction proposals. Some participants mentioned changes that had been implemented after the interviews. Many interviewees were interested in benchmarking the SAFe transformation process to other similar organizations. In particular, they were interested in learning how to map the value streams when having tightly coupled systems, i.e., pension and insurance, as they were facing many challenges with the idea of value streams in this area.

4 Results

This section presents the results of our study, arranged according to the four research questions. Section 4.1 presents reasons for transformation, Section 4.2, presents the transformation process, Section 4.3 presents the benefits, and Section 4.4 presents challenges related to the SAFe adoption in the case organization.

In the sections below, we have ordered the reasons, benefits, and challenges in the tables in thematic categories and within each category based on the number of mentions. We acknowledge that the number of respondents mentioning a topic is not a measure of its significance or importance but merely a depiction of how widely they are acknowledged amongst our respondents. For instance, an interviewee may forget to mention a certain benefit or some benefit might not be visible or important to them directly, even if it is critical to the organization as a whole. However, while somewhat arbitrary, we have included this metric in the tables to showcase which topics were more salient in our interviews than others.

4.1 Reasons for the SAFe Transformation

In this section, we answer our first research question: "What were the reasons for adopting SAFe?"

We identified three main categories for the reasons at PFA, business reasons, organizational reasons, and framework-related reasons. The reasons with short descriptions and the number of subjects mentioning them are summarized in Table 3, and the most salient ones are discussed further below.

It is worth noting that the business and organizational reasons provide the background for an agile transformation and are basically independent of what approach is taken. The framework-related reasons help explain why the SAFe framework was selected over other possible approaches.

Table 3 Reasons for the SAFe transformation
Full size table

4.1.1 Business Reasons

The business reasons represent attributes of the business operation that typically are important to top management, such as improving profits, adding market value, and addressing changing market demands. Typically, these are related to overall organizational goals and not directly tied to any concrete action, like performing an agile transformation or selecting a specific framework.

Shorter Time to Market

PFA struggled with long delivery cycles — getting a feature from the initial idea to production often took 18 months. This created problems in the market, as the customers’ needs might have changed by the time the feature was delivered. In all our interviews, this was the most commonly mentioned reason for adopting SAFe. Many interviewees also mentioned that the smaller batches and shorter iterations of SAFe would help the organization to deliver faster.

I think mainly was the fast delivery, faster to market. That’s because we have a problem ... but we had a problem back then [before the SAFe adoption] that from an idea was created until you have something that you could put out, it took at least one and a half years. And that’s a very long reaction time if you want to be market-relevant.—Project Manager

We want smaller projects [...] the world is changing fast, and we want to be ready for the changes. Before, we had long projects [...] one and a half, two years, and then when we were finished, the world had changed, and the customers wanted something else. —Product Manager

Become a Market Leader

PFA was a 100-year-old traditional organization built on the values of stability and trust. They wanted to change their image and market position to become a market leader and influencer in the pension industry. To this end, a new way of working was thought to help to attract new talent and increase motivation and decrease attrition of the existing employees.

I think it was probably because the markets or the whole IT world was actually moving, so we wanna be on the edge instead of being in the back. We had always been in the back because this is a company that’s built on stability and trust and don’t rock the boat, because… so he was actually changing the way we were working—Project Manager

4.1.2 Organizational Reasons

The organizational reasons focus on improving organizational and customer-related processes.

Improve Collaboration

Before the SAFe adoption, IT and business were organizationally and physically separate and worked in their own silos. Changes in the market increased the need for collaboration, especially between IT and business. Improving collaboration by bringing people from IT and business together was one of the significant reasons for adopting SAFe.

There was this need [...] to work together, from the developers as well. They wanted to work closer with the customers and closer with the rest of the organization. And they were stuck in some quite strong silos, and that was a problem for them. —Lean and Agile Consultant

Improve Productivity

PFA had struggled with low productivity for a long time. An interviewee mentioned that the company had a goal of improving productivity by 50% within two years. As the SAFe framework claims to improve productivity, that affected their decision to adopt SAFe.

I think that if you look at the entire PFA IT, one could argue that the productivity has been very low. —Product Owner

4.1.3 Framework-related Reasons

PFA adopted SAFe due to what the organization considered positive elements and its popularity in the software-intensive industry. The framework was chosen over other scaling frameworks for the following reasons.

The Framework is Well-described and Comprehensive

PFA considered the comprehensiveness of SAFe to be an asset. Furthermore, the amount and quality of documentation was considered important. One of the interviewees mentioned that SAFe makes it easy to start, as it is well-described, and it is easy to educate people, as it is well-documented. The interviewee also considered that it is easier to get help from consultants and coaches for educating people on SAFe than for other scaling frameworks. The quality of the website was also mentioned as a factor in favor of SAFe.

Popularity of SAFe

Several interviews mentioned that the popularity of SAFe among many other large-scale organizations was one of the reasons for selecting SAFe over other frameworks. More specifically, several other organizations in the financial sector were known to have adopted SAFe, a factor that further motivated PFA to adopt it.

So we basically looked at SAFe being the popular way to go [that] many [others] do. And we could see the benefit of SAFe because it’s very well described, and it’s easy to start with. —Agile Coach

One of the agile coaches presented a more negative view: his opinion was that PFA did not have any genuine internal motivation for implementing SAFe but was doing it because everyone else is and because they had a lot of money allocated for the transformation. He stated that one needs to have motivated employees and a clear vision and goals for an organizational change to be successful.

...but it’s not really on to their skin, and that’s the problem with this because you need the motivation to do this, you need employees which are motivated and have a clear goal, clear vision on what and why we’re doing it, and that’s difficult for many just now —Agile coach

Easy to get Management Buy-in

The fact that SAFe prescribes a top-down approach was considered to make getting buy-in from top management easier. Our respondent also noted that the quality of the documentation made it presentable and gave top management the impression that it is easier to understand and implement than other frameworks.

4.1.4 Unaware of the Reasons

Many interviewees at the team level, e.g., team members and Scrum Masters, mentioned that they had no clue about the reasons for adopting SAFe, and some mentioned the lack of clear communication about the reasons for SAFe transformation.

Table 4 Transformation process
Full size table

4.2 Activities Involved in the SAFe Adoption

In this section, we answer our second research question: "What activities were involved in the SAFe adoption?"

We describe the activities that took place during the SAFe adoption in chronological order, as listed in Sequential Activities in Table 4. Some activities took place throughout the adoption, e.g., discussions about the role of projects. We have grouped them under the heading Continuous Activities in the table.

Fig. 2
figure 2

Transformation Activities at PFA

Full size image
Fig. 3
figure 3

Timeline of the transformation

Full size image

The transformation process at PFA was top-down, with management taking the lead in deciding to adopt agile and selecting the SAFe framework. The transformation was done step-wise, starting with a pilot, forming an initial agile release train, and rolling out by forming additional release trains. Next, we provide short descriptions of the transformation-related activities, which are summarized in Fig. 2, in chronological order from the framework selection until the end of our study period.

Figure 3, represents the transformation timeline, from 1917, when PFA was established, until 2019, when all the release trains were formed. More detailed information on release train formation is available in our earlier article (Putta et al. 2019).

4.2.1 Sequential Activities

1. Framework Selection

At the end of 2016, top-level executives discussed selecting a suitable scaling framework for PFA in several meetings. Different scaling approaches, such as the Spotify model (Kniberg and Ivarsson 2012), DAD (Ambler and Lines 2012), and LeSS The LeSS Company BV (2016), were considered. There were discussions about adopting the Spotify approach and how to work with it, e.g., the language that can be adopted at PFA and whether to use the same language or create own terms.

The idea of combining frameworks, in particular SAFe and the Spotify model, was raised in the discussions but was decided against, as it was considered easier for people to learn practices from a single framework. Adopting Spotify was considered more ambitious and harder to initiate than SAFe.

At the end of 2016, two directors attended sessions on SAFe at a large international agile conference. After the conference, they attended a two-day "Leading SAFe" course. This was followed by several discussions within PFA, leading to the decision to adopt SAFe. The full list of reasons was discussed in Section 4.1 above.

After the decision, some people attended a two-day PI planning session at the tax ministry, seeing it work in real life, adding to their inspiration to adopt SAFe.

The new CIO supported the decision to adopt SAFe. He had been part of a successful transformation at his previous employer. As a new hire, he wanted to make a footprint in the organization by leading the SAFe transformation. His previous experience led to selecting the smallest SAFe configuration, the Essential SAFe for adoption.

2. Committing and Communicating

Committing top management to the transformation was done through individual dialogues and steering group discussions with directors and executives from different departments. The new IT director discussed topics like the need for being more responsive in the business and business maturity with the directors before stating the transformation.

After deciding to adopt SAFe, the CIO conveyed the message of transition towards SAFe via a monthly newsletter to the organization. However, the detailed information, like the scope, timeline, people involved, etc., regarding the transition was only conveyed to people actually involved in the pilot. This led to a feeling of exclusion in the rest of the organization, as they were not actively informed. In reality, many others were actually included, as the pilot had several dependencies with other parts of the organization, but this was not communicated clearly.

3. Piloting

The practical implementation of SAFe started with the forming of the first "pilot" agile release train. For this, the front-end development was chosen, as the people working with it had prior experience with agile methods. This first release train consisted of four teams.

Initially, the pilot participants had a three-day workshop about the SAFe model, the agile mindset, and different approaches to implementing the framework. The workshop also prepared people for the upcoming cultural shock by repeatedly telling them about different feelings they will encounter during the journey, e.g., frustration regarding the transformation and wanting to revert to the old ways of working.

4. Coaching

PFA employed six external and three internal coaches to support the teams during the transformation. Coaches helped the teams during PI planning sessions, Scrum meetings, and Scrum-of-Scrums meetings. They assisted the teams by providing exercises for recalling agile and SAFe principles. These exercises were considered intense and helpful by the participants.

Instead of hiring a single external consulting partner, PFA handpicked individual external consultants. The motivation for this was that using a single consultancy provider would likely result in a coaching team consisting mainly of junior coaches, with very few experienced ones, making it difficult for the organization to get the desired level of support. The reason for developing internal coaches was to build competence to carry the organization forward after the external coaches leave.

But the real learning process starts the moment you start to implement it. And there have been a lot of coaches around us and a lot of exercises to brush up and recall the principles and how to use them. And that has been pretty intense and helpful. I mean, without those coaches all the time, trying to push us to stick to the framework, we would — for sure — have fallen back to our old way of working.—Scrum Master

5. Education

PFA made considerable investments in educating people on SAFe. Over 300 people at different levels received training on the framework. Initially, PFA focused on training executives, directors, and people from higher-level management. This helped spread awareness about what was happening in the organization and made them comfortable with the new SAFe setup. Later on, team members were trained and received various SAFe certifications.

An external consultancy company provided the education. Various training sessions and courses were given, including: Leading SAFe, SAFe Practice Consultant, SAFe for Teams, and SAFe Practitioner. In addition, role-specific trainings, such as SAFe Scrum Master, SAFe Product Owner Product Manager (POPM), Advanced Scrum Master and Advanced Product Manager were given. Finally, some coaches took coach certifications.

While the organization was generally satisfied with the SAFe education provided, top management felt that the Leading SAFe training did not provide any insights about leading; the course content had only one chapter on leading people, which fell short of their expectations. As a result, they designed a separate one-day course on lean and agile leadership.

Fig. 4
figure 4

Organizational structure after transitioning to SAFe

Full size image

6. Roadmap, Portfolios, and Transformation Board

The organization developed a roadmap consisting of strategic intents and goals. The roadmap was divided into seven different portfolios, as illustrated in Fig. 4. Each portfolio contained ideas for how to achieve the strategic intents. Each portfolio was owned by one of the executives of the organization.

A transformation board consisting of directors prioritizes the strategic initiatives, after which the initiatives are passed on to agile release trains as strategic tasks. Along with strategic tasks, release trains get tactical tasks from the business.

7. Building the SAFe Organization

PFA formed four prioritization councils, which were composed of directors, senior managers from business and IT, and portfolio managers.

The people working in the agile release trains, e.g., developers, were organizationally allocated to one of two newly formed software centers. Competence centers were formed consisting of the Scrum Masters, project managers, and support functions. These competence centers provide training paths for project managers and Scrum Masters to help them grow in their careers, e.g., from Scrum Master to agile coach and to enterprise agile coach.

According to the SAFe framework, there is no role for middle-level management, but several Centers of Excellence (CoEs) are needed to support the SAFe implementation. Instead of firing middle-level management, the organization placed all of them into six Centers of Excellence (CoEs): the Project and Program CoE, DevOps CoE, Lean and Agile CoE (LACE), SAP CoE, Integration and BPM CoE, and Test CoE.

8. First PI

The first PI for the first ART had some pre-PI events, e.g., refining backlog and prioritization. These events are described next.

Backlog refinement. Before the first PI planning of the first pilot agile release train, product management worked on refining the epics and features. The backlog refining process involved several meetings between the product owners and business owners.

Prioritization in the priority councils. Product managers from the release train prepared a feature list for the upcoming PI. The product manager displayed the list of features to the prioritization council, which prioritized the features based on their value, need, and urgency.

First PI planning. The first PI planning was conducted as prescribed by the SAFe framework. Three coaches supported four teams for two days. Before the PI planning session, coaching was given to product management on feature refinement for product management.

Since the organization was distributed, some developers had to fly in from Poland to attend the first PI planning session, which was held on-site. The Polish developers stayed for a week in Denmark. On Monday, they participated in retrospectives, on Tuesday, they discussed the features that were planned for the upcoming program increment, and on Wednesday and Thursday, they attended the PI planning.

9. Forming a Transformation Team

PFA formed a transformation team consisting of top management, IT, coaches, and specialists. The team was tasked with the design of the rest of the agile release trains.

The organization had many discussions around forming value streams and aligning the agile release trains with them, as the SAFe framework prescribes. However, due to the fact that such a structure was difficult to implement due to internal power politics and because some employees would need to be shared, at least initially, between two value streams, the organization decided against this. Instead, the design of the new release trains retained the old organizational structure with departmental silos.

The transformation team presented the initial release train design to the employees and made small adjustments to the initial design based on feedback received from the employees. However, the basic structure remained the same.

10–11. Forming agile Release Trains

The pilot ran successfully for six months, after which it was renamed to Digital Customer Experience (DCE), and officially became the first agile release train.

Next, the second release train was formed to align several data initiatives, for instance, data warehousing. The third train consisted of insurance and pension products, and the last one focused on future areas of the organization, e.g., digitalization and robotics.

4.2.2 Continuous Activities

Creating a CoP Culture

Management and coaches took an active role in creating the first communities of practice (CoPs). At the time of the interviews, the following CoPs had been formed: integration CoP, continuous delivery CoP, UX CoP, Scrum Master CoP, Product Owner CoP, automated testing CoP, core model CoP, testing CoP, and enterprise CoP. Initially, management insisted that team members participate in specific CoP meetings.

In addition, CoPs were given decision-making power. This mode of working, with CoPs making decisions, and management stepping back and not even influencing the decisions, was new to the organization.

Discussions on the role of Projects

At the time of our data collection, the organization had over 30 ongoing projects, running in parallel with the agile release trains. Due to the way the release train structure was built, the projects which were tasked with delivering solutions needed work from several release trains to be able to deliver. This created a lot of need for coordination and prioritization and made the work in the release trains more difficult. This spurred an ongoing discussion about the role of projects and how to organize them in relation to the release trains.

The release trains were allocated a combination of IT and business work in the form of product epics. At the time of the interviews, the release trains had allocated 40–50% of their time to the project tasks and the rest to strategic and tactical tasks.

Several discussions took place among the project managers and external consultants on aligning the projects with the SAFe world. There were also many sessions dedicated to deciding on ways of working with the project management office (PMO). Moreover, discussions on controlling the money and budget for projects were still ongoing at the time of the interviews.

...we have a discussion going on, [on] how we will put programs into the SAFe world...I have been in discussion for maybe three-quarter years now [sic].—Project Manager

Introducing Scrum Tours

The SAFe training courses mostly focus on understanding the framework rather than the agile ways of working. As most people in the last three agile release trains had no prior experience with agile, the organization saw a huge need for basic agile training. Therefore, coaches designed a set of five modules called Scrum tours: Agile principles, Guidance on scrum, Lean and agile leadership mindset, Feature refinement, and Relative estimation. Each module was short, taking only three to four hours to complete.

People in the last three release trains participated in these Scrum tours before their formal SAFe courses to be better prepared for understanding the framework. At the time of our interviews, Scrum tours were part of an "agile academy," and courses were run around the year. Employees were free to sign up for the tours at their convenience and also to retake a module if they felt a need for a refresher.

The tours were considered beneficial, as they contributed value to the organization by providing a frame of reference for various agile concepts.

And that has been super helpful. That has contributed value because now you can put it in a frame of reference. So now you know what a feature is, you know what the complications are, and so on, and so on. You didn’t know that upfront. So from that perspective, the Scrum Tours have been super-helpful.—Product Manager

4.3 Benefits of the SAFe Adoption

In this section, we answer our third research question: “What were the benefits of the SAFe adoption?" The benefits in each category have been ordered from the highest to lowest number of mentions and summarized in Table 5. They are grouped under two categories: business benefits and organizational benefits.

Table 5 Benefits of SAFe adoption
Full size table

The most significant benefits from Table 5 are elaborated below.

4.3.1 Business Benefits

Shorter Time to Market

Closer collaboration between IT and business decreased the need for handovers between the departments, leading to faster deliveries. Cross-functional teams made it possible to increase parallelism in development, which shortened the time to market.

Having the right priorities helped people focus on the most important items, which led to more content being delivered at the end of each program increment. The organizational disruption and dynamic way of producing things enabled people to react faster, leading to a shorter time to market.

Things come ...some of the things, if you have the right priority, it can actually be delivered very fast — within a PI that is ten weeks, we can actually have working code in production.—Project Manager

Clear Prioritization

All directors were assigned to the prioritization council, jointly deciding on the most important items for PFA. As a result, the directors from different departments gained an understanding of other departments and their tasks.

Further, as priorities are explicit in the PI planning, even teams got a good understanding of the priorities for all tasks within a program increment. Before the SAFe adaption, there were no clear discussions on these priorities.

[..] teams have actually been allowed to work on things that have been prioritized, and they haven’t been stopped again, saying, well, now this is not prioritized anymore. So the stalling or misuse of resources has decreased.—Agile Coach

Increase in Predictability of the Deliveries

The delivery predictability at PFA improved after SAFe. For instance, the ARTs delivered 70 percent of what they promised to deliver. Reports also mentioned that 95 percent of the estimated story points were completed. After the SAFe adoption, there was less task switching which helped teams to be more shielded. This made them estimate and commit to the tasks themselves, which helped them to have more predictable deliveries.

And when I took a look at the projects around us, there were no projects that were in [the] red because of missing deliveries from the train.—RTE

4.3.2 Organizational Benefits

Improved Collaboration

The transition resulted in getting people together from different departments and levels. For instance, directors and teams from five different areas were bought together during the PI planning sessions.

I think it’s gathering people, sitting closely together... has been a success that they can just turn their head and say, this is not working, can you have a little [look at] this, and then it’s fixed right away, instead of taking two or three weeks [...]—Agile Coach

Forming cross-functional teams including people from IT and business, also improved collaboration between the two departments.

I can see that bringing business in with IT and working with the same teams, is absolutely brilliant. Having all the knowledge in one team is fantastic because you can get quick answers, you can get quick, it just works.—Integration Manager

Improved Transparency

Interviewees mentioned that SAFe helped in increasing transparency across the organization in terms of what is being delivered and what is being worked upon. For example, people from the business could see what is happening inside the agile release trains.

The whole process of prioritization also has become transparent, i.e., now all the team members and the business people know the reason behind the order of priority.

In my point of view, the benefits are... I think there are maybe two or three major areas where the benefits are. One of them is, the top benefit is the transparency about what we’re doing. So everybody knows what all the development teams are working on. Before we did SAFe, there were a lot of questions about what are you doing, because if they had a project that they wanted to staff the project, we were doing something else.—Release Train Engineer

Table 6 Challenges of SAFe adoption
Full size table

Improved Value Delivery

One of the most significant benefits received after adopting SAFe was faster delivery of high-value features. PI planning sessions and prioritization helped the organization focus better on the most important items. Some people felt that they were delivering more value, but fewer features than before.

Improved Team Autonomy

After the SAFe transition, teams became more proactive; they did not need a manager to tell them what to do. The teams took more ownership and were self-governed. They could produce value, and implement new ideas, or new enablers based on their knowledge and experience.

Increased Employee Satisfaction

The management conducted employee surveys for the entire organization every year. The score from the people who were involved in the trains was 9 out of 10, which was extremely high when compared to the rest of the organization. They also measured the happiness index on a scale of 0 to 5 for each release after every PI. This score kept improving over time. Further, employees confined to the coaches that they were happy with the new ways of working:

Another great thing is that people are happier. I get a lot of employees coming to me and saying thank you for this, and I’ve never been so happy for working in my entire work life. I have people coming, telling me that. And also we have employee surveys and we can see that people are getting more and more happy.—Agile Coach

Fig. 5
figure 5

Most critical challenges

Full size image

4.4 Challenges of the SAFe Adoption

In this section, we answer RQ4: “What challenges were faced when adopting SAFe?" Table 6 shows the identified challenges ordered from the highest to lowest number of mentions. The challenges with the most mentions in Table 6 are described in the following text.

In addition, Fig. 5, illustrates the most critical challenges. They represent the challenges that our interviewees explicitly mentioned as the most critical, explaining that they caused the most problems and were the hardest to overcome. They were also widely discussed in the organization, which can be seen in the fact that they also have many mentions.

4.4.1 Organizational Challenges

Release Trains Unaligned with the Value Streams

The organization had significant challenges in forming the release trains. As a result, the release trains formed retained the old organizational structure instead of aligning them to value streams. This led to a situation in which there were significant dependencies between the trains.

The reason for this was mainly organizational politics. In the old organization, each unit was headed by a director. The power of each director was measured by the number of people under them. Forming release trains aligned with the value streams would require the directors to give up some of their resources, which they were unwilling to do. As a result, it was decided to form release trains that almost identically resembled the old organizational silos.

The idea was to use this knowingly sub-optimal structure to get started with the SAFe adoption without performing big changes in the organization structure and thus with less change resistance, and then later on gradually change the vertically sliced release trains towards horizontal end-to-end trains.

Another reason for this decision was that the pension and insurance systems were tightly coupled, with the same people working for both systems. Many people who worked for these systems had specialized deep competencies that were needed in both systems but lacked the broad skill set to work on tasks outside their specialty. Further, the organization was not willing to re-skill or add new resources, making it even more challenging to separate these two systems.

Role of Projects Challenging

The organization had several complex projects running during the transition and the organization decided to continue running these as projects. This created significant challenges:

Projects Depend on Several Release Trains. Due to a lack of full-stack release trains, implementing the project tasks required more than one release train. The teams could identify the dependencies between the release trains but were not capable of managing them. To manage these dependencies, the organization used the role of a project manager, which should not be part of a SAFe organization.

Dependencies with Third Parties. The organization had several external dependencies with third-party suppliers. These suppliers could not work as a part of the release trains, which created a high overhead in coordination. Further, the third parties found it difficult to understand the ways of working in SAFe, as they were not used to, or knowledgeable about, agile development and SAFe.

Prioritization of Project Tasks across Release Trains. Prioritization across release trains was lacking since the portfolio level was not implemented. This led to challenges for projects, as their tasks ended up having different priorities in different release trains. While it was the role of the project managers to coordinate between the release trains, many interviewees insisted on creating a common prioritization forum to help resolve the conflicts.

Release Cycle Mismatch. The project release cycle and the program increment cycle of the release trains were not synchronized. The release trains had a release after every program increment, but the projects had monthly releases. At the time of our study, the organization was working on synchronizing these release cycles.

Diminishing Role of Project Managers

After adopting SAFe, almost 80% of the work done by the project managers was moved to the team level. This diminished the need for managerial roles at PFA. Instead of giving project managers new responsibilities, they were organizationally moved into the Program and Project Center of Excellence. This created insecurity and uncertainty regarding their future among the project managers.

Some project managers took up other roles, e.g., Product Owners, Scrum Masters, and epic owners. However, many resisted, as they could not envision themselves in the position of a Scrum Master or a Product Owner after more than ten years of experience; it felt like a demotion. Many project managers reported feeling like they are [left] in the air. At the time of the interviews, some respondents mentioned that the project manager role could be removed, but no firm decision had been made.

Before SAFe, project managers communicated directly with the developers about the project requirements. In agile development, however, teams are supposed to get the requirements from their Product Owner and manage the project tasks themselves. Thus, the project managers had to communicate the project details via the Product Owners to the developers. As the Product Owners did not understand the technical details, they missed many details in passing on the information to the developers. This complicated communication through added overhead, and several project managers disliked this. They also thought that the quality of the end product might decrease due to such a communication lag.

Now we have all those POs, all those Scrum Masters, all those PMs and RTEs, and we have to communicate with them and not with the developers. So there is a lot of know-how going lost in all that communication.—Project manager

Confusion about Agile Roles

With the transformation, many people got new agile roles and felt confused about their roles and the related expectations: New first-time Scrum Masters and Product Owners faced insecurities with their roles, as they came from an entirely different way of working. The way they were supposed to work and collaborate was different from their previous roles. People in leadership roles had difficulties understanding the concepts of servant leadership, e.g., the dynamics of pushing down decision-making to the team level. For instance, one release train engineer mentioned that, in the new set-up, there was more responsibility and less power to tell people how to act on something in comparison to their previous role:

I think my job was much easier before, because it’s like, I have the responsibility. When my manager looks at me, he says, you have the responsibility for this train. But I have no arms or legs because I cannot tell people on the train what to do. In some parts, I can, I can say I want this. I want well here.—Release Train Engineer

Difficulties in Adopting the New Culture

Adopting the agile mindset felt challenging. For example, developers found it difficult to take the initiative to independently start things due to the history of a top-down culture where tasks were assigned. Further, people were wary of taking risks, as they were not allowed to fail in the old culture. Thus, people were afraid to do something new for which they could be held accountable if it failed. Finally, shifting to an agile way of working with planning and delivering weekly, instead of after two years felt challenging to many respondents.

Being Agile in Waterfall World

When the first ART was launched, the rest of the organization was still working in the waterfall model. The pilot ART looked like another silo, i.e., being like a silo of the agile world in the organization with other silos surrounding it. This pilot had a lot of dependencies with other parts of the organization, which were not in a position to change their way of working for the pilot, as they felt it was temporary and will shut down soon.

After forming the other new ARTs, there were several dependencies between them, so each ART had to wait until other ARTs finished their tasks, similar to handovers in the waterfall model.

People Abandoning the Agile Organization

People were not confident about how to use their competencies and skills inside the ARTs. Thus, many started to quit the ARTs and joined other IT-related parts of the organization. The organizational change had drastically reduced the need for everyday coordination responsibility for the leadership and managerial roles, which created frustration and insecurities among them, especially for project managers, and many started to leave the organization.

But when we have staffed most of our people in ARTs, Scrum Teams, there’s no need for daily coordination of task type of responsibility. So we have cut that away from the leadership role almost. I think that’s a really good (move) we have managed, this has created a lot of frustration in the leadership areas and probably also led to a few people leaving.—Integration Manager

4.4.2 Challenges in Implementing Practices and Events

Events taking Up Time

SAFe defines a variety of events, e.g., Scrum of Scrums (SoS), CoP meetings, and PO sync meetings. Our interviewees felt they spent more time sitting in meetings than working on their tasks. Some complained that this time was spent on non-value-adding things. Team members felt frustrated about not being able to complete their tasks on time due to the high number of meetings. Some interviewees also conveyed that team members did not have a clear understanding of the purpose of the meetings.

I mean, I think, it’s my feel that, to our surprise, people have felt that this… the idea was, this transformation to SAFe framework was to give us more time to actually do the work we are hired to do. Not going to meetings, not doing a lot of other exercises, reporting, documentation, whatever that could be. But most people’s experience, I think, is that we have so many things, activities going on in this train all the time, Scrum of Scrum, retrospectives, refinements, COP meetings, different things, and we still have something like unit meetings. That is outside the whole agile framework.—Scrum master and Developer

4.4.3 Human Factors Challenges

Challenges with Personality and Attitude

During the transformation, people were feeling pressured and stressed, as many things were changing, e.g., new ways of working and new roles and responsibilities. Many developers reported that highly skilled people often are introverts and are used to relying on personal managers for collaboration. Communicating more, in particular with people from other teams that they might not know, felt difficult, and to be forced into this felt like a personal challenge to them. Moreover, they told us that until they became comfortable with their new roles, they did not have much energy to collaborate and interact, as most of the energy was used to comprehend the new roles. As a result, many felt frustrated by the change and questioned elements of it. The following quote from a Product Owner illustrates this:

[...] could all Scrum Masters please go out with their teams and get two activity items for the team to improve happiness and two activity items for the train to improve happiness. OK. My take: did you totally misunderstand your reason for being here? What does it say on your paycheck? Kindergarten manager, or what does it say? Come on! There are root causes, go find the f*cking root causes, do something about them, don’t make, don’t start dot voting process here and there where are gonna get all kinds of things, like going to Poland. Actually, that was said, OK, we can improve happiness by going to Poland for a week.—Product Owner

Further, some developers thought that many issues were caused by the fact that they considered project managers to have big egos that found it difficult to accept the role of facilitation instead of dictatorship.

Challenges with Coaching

PFA employed coaches with different backgrounds. It was challenging to synchronize the messages from all these coaches. Many people across PFA felt that external coaches did not understand their reality, the organization’s context. The coaches were felt to be agile purists who did not understand the trade-offs needed in the organization.

Some interviewees reported that the presence of coaches made the teams feel like they were continuously being evaluated. They also felt that the coaches asked questions that interrupted their everyday work, so some even requested space to do their regular work. Further, there were no technical coaches, e.g., for coaching the system architect and guiding on the technical runway.

4.4.4 SAFe Framework Related Challenges

SAFe Courses Too Theoretical

The SAFe courses give an introduction to the roles, artifacts, and processes of the framework. Interviewees mentioned that the SAFe training courses were challenging to understand without prior agile experience and were lack of practical aspects. The courses focus on framework aspects such as PI plannings and release trains. They do not emphasize the agile way of working and building an agile mindset. The courses felt to be theoretical, helping people understand how the framework works in the big picture, but the link to everyday life in their context seemed to be missing.

Yeah, it was useful, and understanding the concepts and the frameworks, but when having, you know, starting to work with the practical aspects of it, it wasn’t really that useful. Finding out what did it mean to write an epic in the PFA context or what do we need to put in a feature or what should we write in the user story and so forth, all these things wasn’t anything that, we didn’t learn it as part of the course, we learned it as part of starting and working with SAFe.—Product Owner

5 Discussion and Limitations

In this section, we discuss the results and compare them to the previous literature. We also present the threats to validity and discuss the implications for practice and research.

RQ1:What were the reasons for adopting SAFe?

We found 17 reasons for the SAFe adoption in our case study. The most frequently mentioned business reason was to shorten the time to market. This is a common reason both for adopting agile at scale, in general, Version One (2019); Kettunen et al. (2020), and for adopting SAFe (Korosec and Pfarrhofer 2015; Schongot and Man 2018). The second most mentioned business reason, to use SAFe to help become a market leader is, as far as we know, not previously reported. In this case, the idea was to use the framework as a means to revamp the market image of the organization to help both with growing the market share and gain a stronger position as an employer for software developers.

Of the organizational reasons, the most salient one was to improve collaboration between IT and business. This reason for adopting SAFe has also been previously reported, e.g., in Paasivaara (2017).

We identified framework-specific reasons motivating the choice of SAFe over other frameworks, e.g., the framework being well-described, comprehensive, and popular. The last one is mentioned in a recent survey on scaling frameworks as well (Putta et al. 2021). One of the reasons for SAFe popularity is due to the aggressive marketing by the framework founders, e.g., by publishing case studies inclined towards reporting benefits (Putta et al. 2018). Similar viewpoints were also mentioned in Maximini (2018).

Getting coaches and consultants to support a SAFe adoption was easier for PFA in comparison to other frameworks. The above-mentioned reasons were also found in a practitioner survey (Putta et al. 2021).

Other reasons found in our study, e.g., to improve productivity, quality, and predictability and to enable cross-functional teams, have been previously reported (Putta et al. 2021). Thus, our results on reasons support the findings in Putta et al. (2021).

In conclusion, we can say that most reasons for adopting SAFe were similar to reasons for adopting agile in general. However, we also found a set of framework-specific reasons that motivated the choice of SAFe over other frameworks.

RQ2: What activities were involved in the SAFe adoption?

We described the SAFe adoption process at PFA using eleven sequential, and three parallel activities. The organization did not use the SAFe-prescribed roadmap for implementation, but many steps are similar, as we discuss below.

The transformation was initiated by top-level leaders and proceeded step-wise. The transformation started by communicating reasons and creating awareness among the people across the organization regarding SAFe. However, many people from the team level did not know the reasons for the transformation. Similar information on the need for communicating change was mentioned in Paasivaara (2017).

Forming the LACE team to support the transformation process was an important step. Similarly, transformation teams were formed in Media Aust post (2017); Scaled Agile Inc (2016a, 2017a, 2016b); EdgeVerve Systems Limited (2018); McMunn and Manketo (2017). Several names have been used to refer to these teams. However, LACE is the official name prescribed by the SAFe roadmap (Scaled Agile Inc 2023).

Another significant part of the transformation was forming a pilot and later the other ARTs. The SAFe roadmap of implementation also suggests starting with a single release train. However, the word pilot is not mentioned by the SAFe roadmap (Scaled Agile Inc 2023). Other cases from the literature started a single ART and then added more ARTs similar to our case Paasivaara (2017); Pries-Heje and Krohn (2017); Oren and Man (2017); McMunn and Manketo (2017).

PFA invested a lot in educating people on SAFe with different SAFe-prescribed courses, which is natural, and commonly reported, e.g., in Media Aust post (2017); Scaled Agile Inc (2017c, 2018a, 2016d, 2018b).

Apart from the usual SAFe training courses, in our case, the coaches created Scrum Tours for teaching the basic knowledge of agile, lean and agile leadership, feature refinement, and estimation, as the SAFe training courses lacked these basics of agile. Several other cases also organized this kind of basic agile training (Scaled Agile Inc 2017b, 2016c, 2017c; Rally software 2015; Scaled Agile Inc 2017d, 2018f). This emphasizes the need for incorporating basic agile and lean training, instead of using only SAFe courses, especially for organizations that are new to agile.

For coaching, our case did not rely on a single consulting partner but instead handpicked individual external consultants. This way they were able to employ very experienced coaches to guide them during the journey. In contrast, several cases in the literature were primarily dependent only on the scaled agile partners (Pries-Heje and Krohn 2017; Paasivaara 2017; McMunn and Manketo 2017).

When comparing the PFA transformation steps to the 12-step roadmap that SAFe lays out for guiding a successful implementation of the framework, we can see three notable differences:

  1. 1)

    Value streams were not identified at PFA, and release trains were thus not built based on value streams. Instead, the release trains were formed based on the existing departmental "silo" organization.

  2. 2)

    The use of projects continued, with project work split amongst several release trains, and project managers kept coordinating the work, even though projects or project managers do not exist in SAFe. The idea behind both of these differences was not to do drastic and difficult changes to the organization at once, but to gradually move towards real value streams and to finalize the ongoing projects first, and not start new ones. However, even though the reasoning behind both of these decisions makes sense, they also caused a lot of challenges and showed that in a real-world complex organization identifying value streams and forming release trains based on those or just getting rid of projects is not as simple as it might seem.

  3. 3)

    Scrum Tours to train people on the basics of lean and agile was the third difference to the SAFe roadmap as the roadmap does not specifically emphasize the aspect of educating people on agile basics. This is an important reminder also for other organizations, that basic agile knowledge is needed first.

Finally, at the end of our study period, our case organization was discussing step 11 in the SAFe implementation roadmap "Extend to Portfolio" as the next possible step to continue.

RQ3: What were the benefits of the SAFe adoption?

We identified 13 benefits of SAFe adoption. The most often mentioned benefit of the transformation was increased collaboration, especially between IT and business. The improved collaboration has been one of the commonly mentioned benefits of SAFe discussed in the previous studies (Putta et al. 2018, 2021).

The second most mentioned benefit was an increase in transparency and visibility. Some recent studies on SAFe also indicated transparency as the biggest benefit (Laanti and Kettunen 2019; Putta et al. 2018, 2021). Faster time to market is another significant benefit reported at PFA. Likewise, several cases from recent SAFe MLR also reported faster time to market as a common benefit after SAFe adoption (Putta et al. 2018).

The SAFe website prescribes metrics to measure certain quantitative benefits, e.g., predictability and employee satisfaction (Scaled Agile Inc 2018e). In our case, employee satisfaction was measured by conducting employee surveys, which are also prescribed by SAFe. Other organizations have reported the use of surveys (Paasivaara 2017) and happiness index (Neil et al. 2018) to measure employee satisfaction.

Our interviewees claimed an increase in productivity, measured in terms of velocity. They knew it was not the right metric. However, they used this to showcase the positive effects of SAFe to the directors. Productivity, according to SAFe, is supposed to be measured in terms of feature cycle time.

Other benefits that were reported by PFA, such as improved team autonomy, improved communication, waste removal, and better dependency management were also found in other recent studies on SAFe (Putta et al. 2018, 2021; Ciancarini et al. 2022; Sebola and Khoza 2022).

It would be interesting if organizations could provide evidence of metrics on the benefits received. More research on metrics is needed to establish scientific evidence. Unfortunately, we could not capture the metrics from this case, as most of the reported benefits were qualitative.

RQ4: What challenges were faced when adopting SAFe?

We identified a total of 16 challenges in adopting SAFe. One of the biggest challenges at PFA was the formation of agile release trains (ARTs) that were not formed according to the value streams, but instead aligned to the old silos, due to political struggles to get the resource buy-in from the directors, as well as for technical and knowledge reasons. Several previous cases have reported difficulties in defining the value streams and forming the ARTs (Herbai and Gusch 2018; Crudup 2018; Holdorf 2018; Levaslot et al. 2016). Unfortunately, these cases did not provide detailed information on this challenge, whereas our case provided in-depth information related to the value streams. The same challenge of retaining old silos was witnessed also by several organizations who were implementing large-scale agile in Dikert et al. (2016).

The ARTs formed were not end-to-end, which created a lot of dependencies between them and was reported as a challenge. Challenges with dependencies between ARTs were reported in the recent studies on SAFe (Putta et al. 2018; Sebola and Khoza 2022).

Interestingly, better dependency management was one of the benefits of SAFe adoption at PFA. However, this meant that teams could identify the dependencies during PIs. Unfortunately, they were not capable of managing them. Project managers were still coordinating dependencies between the four ARTs, which led to a coordination overhead.

At PFA, the project managers continued to coordinate the project tasks. Instead of giving them new agile roles and responsibilities, the company placed them in permanent Centers of Excellence (CoEs). This created insecurity and resistance among the project managers. SAFe only suggests one CoE, Lean and Agile Center of Excellence (LACE), but our case had six. Instead of permanent CoEs, an informal CoP structure could have been an alternative community format as described in Paasivaara and Lassenius (2014). Similar challenges with middle management roles and resistance from project manager roles were found in an SLR on large-scale agile transformations (Dikert et al. 2016) and in an MLR on SAFe (Putta et al. 2018). As middle manager roles are drastically reduced in an agile set-up, persons with these roles need to consider taking other agile roles or they might leave the organization, which had already started happening at PFA.

Other challenges found at PFA, e.g., struggle with automated testing and global distribution challenges, were also reported in the MLRs on SAFe (Putta et al. 2018; Ciancarini et al. 2022). Challenges related to events taking up too much time, and forming permanent centers of excellence instead of CoPs were only found in our case, which adds to the knowledge gap on the challenges of SAFe adoption.

The lack of basic information on agile in the SAFe course program was a surprising finding, as SAFe is perhaps the most comprehensive of all scaling frameworks. The case organization had to create its own training to fill this gap. The SAFe courses were also complained to be too theoretical and lacking practical implementation advice. Other scientific studies on SAFe have pointed out other shortcomings of the framework, e.g., Trienekens et al. (2018), mentioned that SAFe does not have an explicit focus regarding customer involvement and suggested incorporating a customer involvement level in the program, value stream, and in team levels. While Sreenivasan and Kothandaraman (2019), reported that SAFe lacks standardization for estimation.

The framework developers can get an insight into these challenges for improving the framework, for instance, by adding some courses on agile and lean, especially for traditional organizations without prior agile experience.

5.1 Threats to Validity

We discuss the validity and reliability of the research as proposed by Yin (2009). Construct validity: Construct validity concerns how well the constructs created in the analysis reflect reality (Yin 2009). We attempted to increase the construct validity through triangulation of data sources and investigators (Yin 2009; Jick 1979). We triangulated the data sources by interviewing multiple persons in each role in the case organization, when possible. We triangulated the investigators by having the first two authors perform the interviews together. The interviews were coded and analyzed by the first author, but the second and third authors reviewed the analysis. Collecting data through interviews may induce many kinds of bias in the collected data (Shadish et al. 2002). These can be mitigated by triangulating the interview data with data from organizational artifacts such as meeting memos, metrics data, and release logs. Unfortunately, due to confidentiality reasons, we were not able to do this in this study. However, we think that the number of interviewees and roles covered by the interviews help limit this threat. We are thus confident that our findings accurately reflect the perceptions of the members of the case organization. However, we did not collect quantitative data on the changes in the development lead time, flexibility, planning efficiency, or productivity before and after the SAFe transformation. Thus, we have no hard data on how much the transformation affected such indicators.

Internal Validity: Internal validity is concerned with the validity of any identified or claimed causal relationships, and it is only relevant to explanatory results (Yin 2009). In our findings, we describe several causal relationships claimed and explained by the interviewees or otherwise identified by us. Due to the nature of a descriptive case study, the internal validity of the findings cannot be assessed accurately. There is a large number of threats to internal validity that cannot be removed in descriptive case studies, such as maturation and temporal ambiguity (Shadish et al. 2002), and the causal relationships reported should be considered tentative. To increase the validity of our findings, we presented our analysis to the case organization in a feedback session. In the sessions, there was much discussion, and several questions were posed. However, no corrections to our findings were necessary based on the feedback session.

A specific challenge worth mentioning is that the SAFe adoption represented the first attempt at large-scale agile adoption in the organization. It is thus difficult, based on our data, to separate benefits and challenges related specifically to SAFe from those the organization could have experienced in general when adopting agile using, e.g., another framework.

External validity: This threat concerns the ability to generalize the results to other contexts, which is known as the analytical generalization of a case study (Yin 2009). The main question to address is: if another organization does a SAFe transformation, as in this case, which contextual characteristics must be similar to our case for the findings presented here to be valid? We can hypothesize that important contextual characteristics include a large, departmental organized organization, with a traditional development process and tightly coupled legacy systems. To increase the analytical generalizability of a descriptive single case study, the description must be detailed enough to allow analytical comparisons and synthesis with other studies on similar topics (Yin 2009). We have provided rich description of what we consider significant findings to allow for later comparisons and synthesis.

Reliability: Reliability is concerned with the repeatability of the research (Yin 2009). Had other researchers conducted the same study, had they arrived at the same findings? The main threat to the reliability of this research is the variability in the data collection. We used a general interview guide approach (Patton 2002), which introduces variability with respect to how the various topics are discussed in the interviews. We tried to mitigate this through a sufficiently large number of interviewees and by using several interviewers, which allowed for data source and investigator triangulation (Jick 1979; Patton 2002), which helped increase the reliability of the findings.

5.2 Implications for Practice

Based on the findings from the case we have some takeaways and lessons learned for the practitioners.

  • Reflecting on if and why SAFe is the right choice. Many organizations have been adopting SAFe due to its popularity in general and among particular industries such as the financial sector. We believe it is important for organizations to reflect on whether and why they need an agile scaling framework and this particular framework. Did they reach a tipping point where the existing methods cannot be used to develop a product? Are the individuals motivated to change for a better future state of the organization? The popularity of a framework should not be the sole reason for adopting it, something reported also in Putta et al. (2021).

  • Understanding agile principles. The practices cannot be implemented well if the underlying principles are not understood. Organizations should focus on agile principles, mindset, and culture, something not addressed properly in the SAFe courses. An organization must cultivate an agile mindset, especially in the case of traditional organizations, with little or no agile experience. For instance, in our study, PFA introduced Scrum Tours to disseminate the concepts in lean and agile.

  • Courses are not enough. Just training people in SAFe-specific courses does not make you successful in implementing the framework. Courses simply give you a glimpse of SAFe, and it is the people inside the organization who need to learn to apply them to their own context by understanding the underlying principles. This stresses the important role of investing enough in proper coaching.

  • Getting value streams right. A significant step towards SAFe is to identify the value streams and form release trains that are aligned with them. Our case and several others reflected struggles in building the right value streams (Putta et al. 2018). Building a working agile organization can need profound changes to the existing structures in a company, creating various challenges, e.g., organizational politics, re-skilling, and motivational issues inside the organization. Organizations should carefully evaluate these challenges and try to overcome them as early as possible.

  • Working with projects. Many large-scale organizations have projects similar to our case. Combining projects with agile release trains can create dependencies and clashes with the delivery cycles. Organizations should carefully consider the role of projects in an agile world, and decide whether to continue them or not. In case projects are retained, proper rules for project tasks need to be established in advance. Avoiding projects altogether may be a viable option.

  • Role of project managers. SAFe has no place for project managers. In our case, project managers were still needed, as projects coexisted with the agile organization. The project managers were placed in CoEs and were tasked with coordinating between the trains. Many of them reflected uncertainty about their role and the difficulty to perform their job properly. Organizations need to properly plan how to smoothly transition project managers to other suitable roles, e.g., through reskilling, before losing them.

  • Focus on metrics. Previous studies and our study did not have much information on what metrics were used to measure the benefits of SAFe implementation. Establishing clear metrics to track the progress and success of the SAFe adoption is difficult, but valuable. Such data can further be used to gain management buy-in and also to make data-driven decisions.

  • SAFe is not a silver bullet. Many companies might have the impression that adopting SAFe could solve all the problems related to scaling agile. However, companies still face a lot of challenges related to scaling. SAFe, or any other framework, is not a silver bullet for scaling agile, but any framework or method needs to be adapted to the circumstances of each company. We think a solid understanding of agile principles and practices and an agile mindset are necessary, and potentially more important than any framework.

5.3 Implications for Research

Our findings also have several implications for research:

  • How to form agile release trains? This study showed that forming agile release trains based on first identifying the value streams was not straightforward, both due to technical issues caused by the structure of the old systems, as well as due to organizational politics. As other cases have had similar problems, this would be an interesting area to study further to be able to help companies struggling with this.

  • What to do with projects? Continuing with the projects caused challenges in our case. As many other large organizations moving to agile have projects, it would be useful to study different ways of handling these projects while transitioning and using agile at scale.

  • Studies on the benefits of scaling frameworks. Studies should explore the benefits of scaling frameworks. Such studies could look both at the high-level benefits achieved from the framework adoption as a whole, as well as at the practice level — are there framework-specific practices that are crucial from the point of view of providing those benefits?

  • Studies on the challenges of scaling frameworks. More studies are needed on the challenges: both framework-specific challenges and challenges related to the change to large-scale agile in general. It seems that many of the challenges are common to large-scale agile adoption in general, but some are specific to some practices related to a specific framework. Going into details of the challenges in the specific context might be helpful to other companies.

6 Conclusions and Future Work

In recent years, the adoption of agile scaling frameworks in general, and the Scaled Agile Framework in particular, has increased significantly. However, there is a lack of scientific studies on framework adoption. This study aimed to help bridge the gap by analyzing the reasons, transformation process, benefits, and challenges of SAFe adoption in a large financial corporation.

The most significant reasons for SAFe adoption in this case study were the expectation of a shorter time to market and improved collaboration between business and IT. This specific framework was selected as it was well-described, comprehensive, and popular.

The transformation process in the case resembled the SAFe implementation roadmap, starting with a pilot release train, organizing SAFe training courses, and launching new agile release trains. The biggest differences compared to the roadmap were: not forming the release trains based on value streams, but largely based on the previous siloed organization structure and not terminating the big projects, but continuing them and splitting the work into different release trains coordinated by project managers. More in-depth studies are needed on the transformation process, especially on how complex organizations are identifying the value streams, forming the ARTs, and dealing with the projects.

The reported benefits were aligned with the reasons for adopting SAFe, i.e., shorter time to market and improved collaboration between IT and business. Practices like PI planning, shorter increments, and having clear priorities helped in collaboration, improved transparency, and enabled faster deliveries and better predictability.

The study identified some metrics for benefits, such as employee satisfaction and predictability. However, there is a wide range of potential competing explanations for achieving this, in addition to implementing the SAFe framework. It is indeed difficult both to separate the contribution to the benefits due to introducing SAFe, as opposed to other changes done and even more so to separate the role the framework played in achieving the benefits, compared to implementing agile without any framework. Designing studies to understand this will likely be difficult and probably best done at the practice level. However, such studies would contribute much to our understanding of the costs and benefits of agile in general and scaling frameworks in particular.

The study identified a large number of challenges, several of them not found in previous studies on SAFe. One of the significant challenges identified in the case study was the difficulty in identifying value streams and forming ARTs around silos instead of value streams. The spread of projects over the four ARTs caused coordination overhead, and the lack of end-to-end or full-stack trains resulted in many dependencies between the trains.

Additionally, the number of challenges and the number of challenge mentions in the interviews were higher than the corresponding numbers regarding benefits. In contrast to this, the previous literature has been more inclined toward presenting the benefits of the framework. This indicates publication bias, especially in the grey literature on SAFe, e.g., the case studies linked on the SAFe website present several benefits, but only a few challenges.

Scientific studies exploring the usage of scaling frameworks, reasons, transformation process, and challenges from different organizations are needed to understand the state of practice of scaling frameworks and to compare between different agile scaling frameworks and contexts.

In conclusion, more research is needed to understand the challenges and benefits of SAFe adoption and to develop metrics that can be trusted to assess the benefits of the framework. Multiple case studies on large organizations with more than three years of experience in implementing SAFe are necessary to create solution-centered research and advise organizations on mitigating encountered challenges.