Keywords

1 Introduction and Background

Scaled Agile Framework (SAFe) was launched in the 2011 Agile conference [1]. Since then it has become the dominant agile scaling model in large enterprises [2]. According to the 12th State of Agile Report conducted by Version One and Collabnet, SAFe is now the most widely adopted large-scale agile framework with the usage rate of 29% [3]. Scaled Agile Inc. has announced that there are 300,000 SAFe-certified practitioners in 110 countries, and that 70% of Fortune 100 companies employ SAFe-certified professionals [4].

Both benefits and challenges of SAFe adoptions have been reported. However, more empirical research is needed to understand how to adopt established frameworks like SAFe, how are they used and tailored, and what their benefits and challenges are under different circumstances [5, 6]. This paper examines SAFe usage, benefits, and obstacles in organizations in Finland to contribute to those research gaps.

In general industrial experiences report a 10–50% improvement in employee motivation, 30–75% faster time-to-market, 20–50% increase in productivity, and 25–75% reduction in defects [4] – but empirical research validating these results are missing. In 2017 the Large-Scale Agile Workshop called for more research on scaling agile along with inter-team coordination, knowledge sharing and knowledge networks, agile transformation and business agility [5]. This raises numerous questions, like is SAFe used alone, or together with other agile methods? What kind of results do the companies get with it?

The existing research on SAFe cover for example SAFe and Portfolio Management [7,8,9], SAFe and testing [10], SAFe in distributed settings [11], maturity model for adoptions [12,13,14], principles of large-scale agile [15], case studies of adoption [16, 17], comparison of different scaling frameworks [18], and reported benefits and challenges [6]. However, there is lack of empirical research on SAFe adoptions and usage in companies. With this paper we are aiming to fill in the gap with a survey research into SAFe adoption across multiple industries and companies.

2 Survey Research Design

The purpose of our survey research was to examine the current state of Agile in Finland. Agile has previously been surveyed in many different investigations, but currently in particular the rapid advances of digitalization make it especially topical for different organizations – not limited to software companies.

Our questionnaire was composed starting from our selected main themes of interests including SAFe usage and transformations. The specific questions were then compiled by referring to prior surveys for comparison purposes (e.g., [20]) and by deriving from our industrial experiences. The target audience was intentionally not limited to software companies since we were also interested in non-software companies currently facing digitalization and becoming more software-intensive. The draft questionnaire was first piloted both in our industrial and academic organizations. The final version consisted of 50 questions with some variable parts depending on the selector questions. Especially the questions “What have been the three most significant benefits of the SAFe adoption?” and “What have been the three biggest obstacles to the SAFe transformation (adoption)?” were only presented to those respondents who stated they are using SAFe.

The survey was implemented as a web-based online questionnaire. The questionnaire was distributed social media and a proprietary industrial mailing list mass postings to over 600 people, and it was open for responding for four weeks during November and December in 2018.

3 Results

We received 136 responses to our survey, out of which 111 respondents (80%) were using SAFe. In this paper “SAFe users” refer to all respondents that replied to the multi choice question “What agile methods and models are there in use in Your company?” with the choice ‘Scaled Agile Framework (SAFe)’. Part of this group also used Scrum and Kanban and other methods. “Non-SAFe users” refer to the respondents stating they were not using SAFe. This is because the aim was to study differences between these two groups. Also there was no significant difference detected what comes to the length of agile methods adoption between SAFe and non-SAFe users.

The majority of respondents were using agile methods (“How widely does Your company use agile methods in software development?”) either for their entire organization (20%) or for the entire business unit (59%); see Fig. 1. The result is in line with SAFe targeted to be the operational model for the entire company. Yet the further analysis of results reveals that there is no significant difference between SAFe users and non-SAFe users in this respect. In fact, it seems that many respondents use multiple methods mixed together. 90% of all SAFe users used also Scrum, and 83% Kanban, 12% XP, 12% own model, 9% LeSS and 2% DAD. Only 5% of all SAFe users (6 respondents) responded that SAFe is the only agile method they use.

Fig. 1.
figure 1

Most respondents use agile methods in entire company or in parts of the organization. Legend shows number of respondents (multiple choices allowed).

Full size image

Figure 2 presents the SAFe practices the respondents use. Not all respondents responded to this free text question.

Fig. 2.
figure 2

Number of SAFe practices used by those respondents who use SAFe.

Full size image

With the scale from 0–100 most of the respondents (96 replies) thought that SAFe as useful, with the average of 65,78; see Table 1.

Table 1. Statistical analysis of the responses to the question about the usefulness of SAFe with the scale from 0–100.
Full size table

47 respondents out of 111 SAFe users responded to the question about the most significant benefits of SAFe usage. The question was an open text question asking for the three biggest benefits. The three biggest benefits reported were (1) transparency (24) (2) co-operation (10) and (3) common cadence or rhythm (9). The coded (original answering data partially in Finnish) results are presented in Fig. 4.

51 respondents out of 111 SAFe users responded to the open text question to name three biggest problems on SAFe adoption. The biggest problems reported were (1) old mindset and culture (14 answers), (2) the model has not correctly fitted to own organization (8 answers) (3) missing fluency when using the model (8 answers) and (4) leadership (7 answers). Figure 3 represents a coding summary of these open answers.

Fig. 3.
figure 3

Result of analysis of open questions of SAFe adoption benefits.

Full size image

4 Discussion and Conclusion

The purpose of our research was to study the state of SAFe adoption in companies in Finland. Yet, because of the anonymous nature of the survey we do not know exactly how many companies were represented by these answers. The respondents represented widely different industries though, and not just software or IT industries.

The results indicate that although SAFe is widely in use (80% respondents), it has not in most cases replaced other methods and old practices, but is used together with those methods and practices. Only 5% of respondents stated that they are using SAFe as the only method. Most likely these could be e.g. subcontractors that work only with SAFe. The most commonly received benefit received with SAFe is transparency. The conclusion is, that although methodologist discuss hectically of the pros and cons of different models, most organizations use a combination of these methods.

Transparency is usually just the first benefit of SAFe transformation that is received when organizations have implemented backlogs. SAFe is also being used without using all the SAFe practices; e.g. only 20% replied that they use DevOps with SAFe. We though received feedback that the term Scaled DevOps used specifically as the choice in this question caused confusion amongst the respondents – thus the real number of DevOps users could be higher than indicated by responses.

The conclusion is that although many respondents stated they were using SAFe, it seems that most organizations are still at the beginning of their agile transformation. The benefits that SAFe lists got also mentioned but less often (see Fig. 4): execution speed was 4th mentioned benefit (5 notes) Value and employee satisfaction were both mentioned only 2 times, but improved quality was not listed as a benefit.

Fig. 4.
figure 4

Answers to three biggest problems in SAFe adoption analyzed.

Full size image

We hope to conduct a longitudinal study in this area and see if the responses change over time. We also hope to get more results on what are the reasons behind successful and non-successful agile transformations. We also hope that we can repeat a similar study in Sweden and compare those results to these in Finland reported here.