Keywords

1 Introduction

Agile Software Development (ASD) arose in the early 2000s [4] and several studies [11, 13, 14] have aimed to understand challenges faced and mechanisms adopted by teams in the transformation to agile, which encompasses mainly changes in team culture, people skills, and mindset. Among the studies investigating agile adoption, opinion-based surveys contribute to bringing up a big picture of how practitioners have embraced these agile practices.

Industrial surveys to investigate ASD are common [35], and some of them have been conducted year after year for years now, as Version One’s [30]. In academia, we also find surveys that describe agile adoption, but they are usually less frequent. The last countrywide survey in Brazil took place in 2011 [26].

The goal of this study is thus to describe current agile usage in Brazil. We conducted a countrywide survey in 2018–2019, asking ASD practitioners which are the practices, principles, and methods they apply. We also investigated the perception of project success, reasons for agile adoption, challenges faced, and the impacts they perceive. Furthermore, we collected data to identify which principles influence improvement in different aspects of the software process. Data were analyzed using descriptive statistics and machine learning techniques.

The remainder of this paper is organized as follows: Sect. 2 briefly presents related work, from the perspective of academic surveys. Section 3 presents our research approach. Section 4 shows the results and Sect. 5 discusses our results by comparing them with other ASD surveys. Section 6 concludes the paper.

2 Related Work

Industrial surveys have long been performed in the context of ASD [30]. They have been serving as a benchmark for practitioners to understand other companies’ characteristics and outcomes [33]. Besides methodological limitations pointed out by Stavru [35], these surveys are mainly represented by practitioners in North America and Europe. Academic studies, on the other hand, have a less frequent application but present more methodological rigor. They usually focus on specific contexts but are still important since they characterize the studied community and allow for future comparisons between contexts.

For example, Livermore [25] studied the Extreme Programming (XP) adoption among 112 practitioners associated with the Software Engineering Institute’s Software Process Improvement Network (SPIN). In the European context, Salo and Abrahamsson [34] report on Extreme Programming and Scrum adoption in 13 industrial organizations and Kuhrmann et al. [23] showed how 69 European practitioners combined agile development with traditional approaches. In Finland, Rodriguez et al. [33] investigated the adoption of Lean Principles while also studying ASD adoption. Bustard et al. [9] studied how agile is adopted in 37 software companies from the Northern Ireland. There are also surveys characterizing the adoption of agile in India [28] and in North America [32].

From South America, Melo et al. [26] present the results that serve as a reference to our study. They conducted a large-scale survey in Brazil in 2011. They had 471 participants. Still focusing on Brazil, the study by Diel et al. [12] was conducted to describe the understanding that Brazilian practitioners have about agile methods. This study collected data from about 200 professionals mainly located in Brazil’ South and Southeast regions. Two years later, Bolatti et al. [5] conducted a smaller survey in Argentina (79 participants) given the lack of studies focusing in the Argentinian market.

The above shows that academic agile surveys on how ASD has been adopted in different contexts and countries indeed have been conducted for several years. But it also demonstrates how infrequent they are. Our study aims to update the current state-of-practice in Brazil taking Melo et al.’s work [26] as reference.

3 Research Approach

The goal of this research is to describe agile usage in Brazil. The GQM model – Goal-Question-Metric [3] – that guided our study design, is shown in Table 1. Using GQM approach is a recommended practice in surveys to limit scope and stick analysis to the research objective [27]. We chose the opinion-based survey (or survey only from now on) as our research method. A survey is a “comprehensive system for collecting information to describe, compare, or explain knowledge, attitudes, and behavior” [31, p. 16]. Surveys must have a target population - the group of individuals to whom the survey applies [20]. In this survey, the population is information technology practitioners that work with agile methods in Brazil. We chose a non-probabilistic sampling, denoted convenience sampling, proper for when respondents are easily accessible [20]. Our sample is practitioners that attend agile industrial conferences in Brazil. We chose this approach to locate those who work with agile methods more likely. We collected data in 6 editions of 4 distinct agile software development industrial conferences during 2018 and 2019, namely: Agile Trends (2018 and 2019 editions), Agile Trends Gov (2018), Agile Brazil (2018 and 2019), and Agilidade Recife (2019)Footnote 1.

Table 1. Research goal, questions and metrics
Full size table

Kitchenham and Pfleeger [18] advocate that survey instruments are conceived in four steps, namely: 1) search the relevant literature; 2) construct the instrument; 3) evaluate the instrument, and 4) document the instrument. We based our instrument on existing studies for our research’s first step. We started (Step 1) by searching for the relevant literature, carefully analyzing the questions published in the following reports: The 11th annual state of the agile report - Version One [29], Azizyan et al. [2], Rodriguez et al. [33], Melo et al. [26], Bustard et al. [9], Diel et al. [12], Bollati et al. [5], and Kuhrmann et al. [23]; and proposed our questionnaire from them (Step 2)Footnote 2. Next, we evaluated our instrument (Step 3) [19] with six full-professors and researchers for readability and completion time. We also applied it with an experienced practitioner for content validation. In the fourth step, we documented the process in our research protocol.

We chose to make the questionnaire available for the first conference (Agile Trends Teams - São Paulo/SP, in 2018) using the Qualtrics tool, but the strategy was not effective, as we got only 16 responses out of 854 attendees. We changed then the approach to apply the questionnaire personally, in a printed form; this way we could approach people face-to-face and ask them for their attention [19].

Our new data collection strategy included personally approaching conference attendees during check-in and coffee breaks. Three or four people (depending on the conference size) were hired to aid data collection. In Agile Trends Gov, Brasília/DF (2018), there were 192 filled questionnaires out of 550 attendees; in Agile Brazil (Campinas/SP), 2018, we got 225 responses (we do not have the number of attendees); in Agile Trends Teams (São Paulo/SP) in 2019 we got 226 responses (from 898 attendees); in Agile Brazil (Belo Horizonte/MG), 2019, there were 161 responses out of 771 participants in the conference; and, finally, in Agilidade Recife (Recife/PE), in 2019, there were 77 responses from a group of 350 attendees. We got 551 full-responses from 897 answered questionnaires.

We chose to consider also the partially responded questionnaires for data analysis given that questions can be individually analyzed. Kitchenham and Pfleeger [21] recommend doing so when questions are independent of one another. All of our questions data were analyzed with descriptive statistics. Cronbach’s Alpha was used to measure consistency when applicable.

To complement our analysis, we used machine learning (ML) to predict improvements in the software process’s different aspects based on principles application. The application of ML techniques instead of statistical ones – such as Linear Discriminant Analysis (LDA) or Logistic Regression (LR) – has been shown to perform better in several application domains [1, 10, 17]. Furthermore, the application of statistical procedures require assumptions about data or about relationships among them, such as homoscedasticity, which are not necessary when using ML [7].

Three different techniques were applied: Artificial Neural Network – ANN [16], Support Vector Machine - SVM [36], and Random Forest – RF [6]. The first – ANN – was used to identify which improvements were predicted by applying specific sets of principles. Then, we used SVM and RF to determine which principles were more relevant to get to improvements. We trained the ML models in two rounds as follows.

First Round. We trained thirty different ANNs (using Weka software) for each of the evaluated impacts (each data set). The values we tested for the hidden layers parameter were 30, 40, 60, and 70, for the learning rate parameter were 0.2, 0.5, and 0.7, and for the momentum, parameters were 0.3, 0.5, and 0.7. A specific value for the predictor variable is called “class”, and our interest was in class “Improved”. For the class “Improved”, the classification precision was used to identify which impacts could be predicted by applying certain principles. We chose the impacts that had accuracy for “Improved” class higher than 97%;

Second Round. After identifying which impacts were predicted by applying certain principles, we trained the model with SVM and RF algorithms (using R programming language) to determine precisely which principles mostly affected the perceived impact by extracting the most important attributes when training the models. Both models resulted in all prediction statistics and showed each attribute’s relevance in the prediction. We executed 3328 SVMs with a radial base function kernel for each data set in the SVM execution. The C parameter was adjusted from 0.01 to 100. The sigma kernel parameter was tested from \(10\wedge -15\) to \(10\wedge 3\). The models’ quality was measured using holdout with 80% for the training data set and 20% for the test data set. In the Random Forest execution, the parameter that sets the number of trees was tested from 100 to 1000. The parameter that sets the number of attributes used was tested from 1 to the total number of attributes in each data set. As a result, we considered the 75% more relevant attributes for each evaluated aspect. Using this information, we identified which attributes (principles) were more critical in predicting the resulting variable (improvement in each aspect).

3.1 Threats to Validity

Reliability and validity are relevant concerns in survey research [8]. We consider we addressed reliability by using questions already asked and analyzed in different contexts by other researchers, and using Cronbach’s alpha test for our questions, which ranged from 0.51 to 0.92. Regarding validity, we addressed it by using the accuracy and precision metrics – outcomes from ML techniques, and using the measures already applied in numerous research in other contexts and other moments in time and comparing them. On respect to generalizability, ours was a convenience non-probabilistic sampling. Our data is representative only to the context where we have collected them, considering that conference attendees might represent a subset of agile practitioners profile. However, as for the reference studies we used here, serve as a benchmark for comparison to other surveys with the same objective. Thus, there is a practical relevance in replacing the statistical relevance [20].

4 Results

As previously mentioned, this survey aims to describe agile software development usage in Brazil. We collected 897 responses (men = 69.8%, women = 30.2%). Over 40% of the participants (41.6%, n = 897) have between 36 and 45 years old and the remaining are distributed as follows: 6.7% are under 25 years, 45% have between 26 and 35 years, and 6.7% are above 45. Our results are described next as per the research questions and metrics established in our GQM model.

Which Is the Profile of Companies that Use Agile Methods? Regarding the company size where the respondents’ work (n = 897), more than half of the respondents (61.1%) reported to work in companies with 1000 people or more and a bit over one quarter of them (26.8%) in companies with 100 to 999 people. Other results include: 1.7% - less than 9 employees; 5.7% - 10 to 49 people; and 4.8% - 50 to 99 people. These companies are mainly distributed in the following Brazil’s regions: 55.7% in Southeast, 15.5% in Midwest, and 11.8% in Northeast.

Regarding team sizes, of all valid responses (n = 876), 13.6% work in a team with less than 6 people; 22.8% in teams with 6 to 10 people; 20.8% with 11 to 20 people; 15.0% work in teams with 21 to 50 people; and 27.9% work with more than 50 people. When asked about the teams physical distribution (n = 888), 60.0% said the teams are not distributed; 28.0% said they are distributed within Brazil; 9.0% is globally distributed; and 2.9% in located in South America only.

The top-5 industries that the participants’ companies belong to are: software (34.8%), financial services (27.5%), government (26.7%), education (9.9%), and internet services (9.3%). Due to the expressive amount of respondents related to the Brazilian government and public services, an excerpt from this specific 2018 dataset is reported in [15]. Moreover, regarding the length of time that the respondents’ companies use agile methods (n = 880), 14.7% said it is less than a year; 28.1% have 1 to 2 years of agile usage; 34.3% have 3 to 5 years of use; 15.9% have 6 to 10 years; and 7.0% have more than ten years.

Which Is the Profile of Practitioners that Use Agile Methods? When asked about their software development experience, 6.8% stated to have less than a year of experience; 5.9% have 1 to 2 years; 12.2% have 2 to 5 years; 25.0% state to have 5 to 10 years of experience. An expressive amount, 39.9%, have 10 to 20 years of experience; and 10.2% have more than 20 years (n = 844). Concerning their experience with agile methods, 11.4% have very little knowledge, 61.5% are moderately experienced, 20.7% are very experienced, and 6.4% declared to be extremely experienced in agile methods (n = 886).

What Are the Characteristics of Agile Software Development Usage? Our questionnaire asked practitioners about the methods they use. We also asked whether they usually combine agile methods with more traditional ones, i.e., hybrid methods [23]. We found out that 79.3% use Scrum, 67.3% use Kanban and 21.6% point out to combine them with Scrumban. As relevant results we also see that 20.6% report to apply a hybrid customized method, 15.6% use Scrum/XP hybrid and 11.6% report to use Lean Development. Finally, 11% report to use XP (Cronbach’s alpha = 0.51, n = 893). Respondents could choose more than one option.

When asked whether they combine these agile methods with traditional ones, 56.7% combine them, 35.6% do not combine, and 7.6% stated not to know (n = 894). Regarding the range of adoption, 3.3% of the respondents use agile methods in none of the teams, probably teams that are starting agile methods usage; 42.0% use in less than a half of the teams; 33.3% use in more than a half, and 21.4% of the respondents use agile methods in all teams (n = 886).

What Is the Perception of Success in Agile Projects? We asked about their general perception of success in projects that use ASD. They could answer yes, no, sometimes, or that they did not know. As a result, we got 41.9% of respondents saying that yes, projects are successful; 6.7% said they are not successful. The majority – 48.1% – said that they are successful sometimes, and 3.3% stated that they do not know about projects’ success.

What Is the Extent to Which Agile Methods Principles Are Applied? Based on results presented by [33], we investigate agile principles together with lean principles. Respondents could point out the intensity of application for the principles that applied to them (this is why n differs for each principle). Table 2 shows that the most frequently applied principles are working together with business people (62.4%), valuing continuous improvement (60.8%), and valuing working software more than comprehensive documentation (58.1%). The least applied principles are limiting work in progress (22.1%), inspecting team members’ work (31.9%), and measuring progress with working software (38.4%).

Table 2. Intensity of the application of agile principles in practitioners’ companies (percentage of respondents). Cronbach’s alpha = 0,91
Full size table
Table 3. Reasons for agile development adoption. Cronbach’s alpha = 0,67
Full size table

When contrasting the agile principles for the success perceptions, we could see that, depending on project success perception, the intensity of agile principles’ application differs. We clustered respondents into three groups: one that reported successful projects, one that reported sometimes-successful projects, and those that reported unsuccessful projects. We then calculated the mean percentage of principles application. Successful projects apply principles more frequently: when practitioners reported that their projects were successful, 58.8% reported applying principles frequently, 29.9% reported to rarely apply, and 20% reported to never apply them. Conversely, when projects are not successful, 3.2% reported that they frequently apply principles, 9.8% that rarely apply, and 18.7% that never apply them. Figure 1 shows the mean percentage for each intensity of applying principles for each project success perception group.

Fig. 1.
figure 1

Mean percentage of principles’ application by projects’ success perception

Full size image

Which Are the Reasons for Adopting Agile Methods? Table 3 shows the reasons for agile adoption. The main reported reasons are accelerating software delivery (70.4%), increasing productivity (62.5%), and enhancing the ability to manage changing priorities (41.8%). The less reported reasons are improving engineering discipline (15.1%), improving team morale (16.7%), and increasing software maintainability (16.9%). Respondents could chose multiple reasons.

Which Are the Practices Applied? When asked about the respondents’ practices in their daily routine, the most used practices are daily standup meetings (78.4%), kanban boards (76.7%), and retrospectives (67.4%). Among the least-used practices are emergent design (7.1%), agile portfolio planning (14.1%), and behavior-driven development (16.4%).

Which Are the Challenges Faced? Table 4 shows the challenges that the practitioners perceive in the use of agile methods. The most cited challenges are cultural change (62.8%), resistance to change (53.6%), and agile practices customizing (48.9%). The least-mentioned were the need for special skills (5.9%), the inadequacy of existing technologies and tools (7.8%), and loss of management control (10.6%). Respondents could also select all options that apply.

Table 4. Challenges faced when using agile methods. Cronbach’s alpha = 0.54
Full size table

Which Are the Impacts Felt with Agile Methods Adoption? We asked respondents to rate the perceived impact of listed aspects between Improved, No effect, Got worse, and Do not know. Table 5 shows that team collaboration (87.9%) was the aspect perceived as improved by most of the respondents, along with team communication (83.6%), and learning and creating knowledge (82.2%). The aspects least perceived as improved are project cost reduction (37.3%), engineering discipline (37.4%), and managing distributed teams (37.9%). Table 5 also shows that the aspect most mentioned as getting worse due to agile adoption is project predictability, indicated by 6.0% of the respondents.

Table 5. Percentage of respondents that report each impact level for different aspects of agile adoption. Cronbach’s alpha = 0.92
Full size table
Table 6. Improvements predicted by the application of agile principles
Full size table

Which Principles Affect the Perception of Improvement in the Software Process? We applied two rounds of machine learning algorithms to verify whether principles adoption could predict improvements. In the first round, the Artificial Neural Network (ANN), we were interested in models that are good at predicting improvements. We identified the impacts that presented best measurements in precision values. The impacts which have best precision (>97%) for the class “Improved” are: learning and creating knowledge, business/IT alignment, team collaboration, team communication, self-management skills, time to market, ability to adapt to changes, and ability to manage changing priorities. It means that different combinations of applying principles might define improvements in these aspects. In our second round of analysis, we ran Support Vector Machine (SVM) and Random Forest (RF) algorithms to identify which specific principles positively affect the perception of improvement in these aspects. Using the confusion matrix results for the execution of SVM and RF techniques for each evaluated aspect – considering the 75% more critical attributes – we identified the True Positive resulting values, as they reflect the percentage of prediction in which the models correctly predicted improvement in the evaluated aspects.

The execution of these models resulted is a list of principles that are more relevant for the predictions. Table 6 shows the principles that contribute to improvements in agile software development. For instance, when the principle “Attention to technical excellence” was applied, machine learning models could predict improvements in “Time to market” and “Ability to adapt to changes”. The same interpretation applies to the other principles in Table 6.

5 Discussion

The goal of this study was to describe current agile usage in Brazil. We conducted a survey in 6 editions of 4 industry-based agile conferences in 2018–2019, resulting in 897 responses. Descriptive statistics and machine learning models were used to analyze data. We learned that most Brazilian practitioners that participated in our research work in teams with up to 20 people and that most of these teams are not geographically distributed. Most of the respondents were from the software, financial services, and government industries. The majority have been using agile between 3 to 5 years, although there is also a significant percentage of companies that are young in agile usage (1 to 2 years of adoption).

Scrum and Kanban are the most used methods, albeit we could see that more than half of practitioners state to mix agile methods with traditional ones. This combination of traditional with agile methods seems to be an established trend, as [24] also observed. In their research, a purely agile or traditional application was seldom evident. In our results, we also saw that about 20% of companies use agile methods in all teams, and 41.9% say that agile projects are indeed successful.

Practitioners showed us that the most frequently applied agile principles are working together with business people, valuing continuous improvement, and valuing working software more than comprehensive documentation. When relating the application of agile methods principles to the perception of project success, we could show that, when respondents pointed out that agile projects were mostly successful, the intensity of the application of ASD principles was frequent for most of them. The main reasons for adopting ASD are accelerating software delivery, increasing productivity, and enhancing the ability to manage changing priorities.

Table 7. Comparison of our results with Melo et al. (2013)’s.
Full size table

Regarding agile practices, we see that the most applied are daily standup meetings, kanban boards, and retrospectives. Practices are a important part of application of agile methods, as they have been related to an increase in the degree of agility [24]. Moreover, the study by [22] identified a relation of practices with team satisfaction, as they enable team cohesion and support tracking of the progress.

The main challenges teams face are related to personal issues, such as cultural change and resistance to change (also presented as hindrances in [22]), and process issues as practices customizing. Moreover, our data shows that improvements could be perceived mainly on team collaboration, team communication, and learning and creating knowledge. We also uncovered that improvements in the areas of learning and creating knowledge, business/IT alignment, team collaboration, team communication, self-management skills, time to market, ability to adapt to changes, and ability to manage changing priorities could be predicted by the application of certain principles (uncovered by machine learning models).

Part of our results can be directly compared with other studies. We did so with the Brazilian study by Melo et al. (2013) [26] and with the international commercial survey by Version One (2019) [30]. Chi-square tests were used to identify differences in frequency distributions, in which p values lower than 0.05 mean statistically significant difference. Not all items that we asked in our study were available to compare to the others. When contrasting our study results to Version One’s (2019)’s [30], we could apply comparisons to the length of time using agile, reasons for adopting agile, benefits and agile methods. We see that companies in Brazil seem to be younger on the use of agile methods. Regarding the reasons for adopting ASD, a similar number of Brazilian practitioners state reasons for accelerating software delivery, reducing project risk, and better managing distributed teams. We see a similar perception of benefit in team morale, project risk reduction, and better managed distributed teams. Concerning the adopted agile methods, we see that our results differ from Version One’s (2019) [30]; that is, the percentage of practitioners who adopt each method is different in Brazil, mainly expressed by significantly larger Kanban adoption in Brazil.

By comparing our results to Melo et al. (2013)’s [26], it is possible to identify the evolution of Brazilian community (see Table 7). Regarding the time using agile, it is interesting to notice how, in our study, seven years later, the aging of the teams appeared. We have more teams that use agile for more than five years, although we still have young companies with regards agile adoption. Reasons for adopting agile has also evolved over the years. The only reason that remains with the same distribution is accelerating time to market.

Last but not least, we also compared the perceived benefits upon agile adoption. The perception of benefit remains similar to the Melo et al.’s study for team morale, software maintainability, risk reduction, quality, and cost reduction. Our dataset shows that more people perceives benefits on time to market, project visibility, productivity, manage distributed teams, manage changing priorities, and alignment between IT and business.

6 Conclusions

This study aimed to report how ASD has been applied in Brazil. Based on responses from 897 practitioners, we showed the profile of companies and teams, characteristics of agile usage, perception of success, principles and practices applied, reasons, challenges, and impacts of ASD adoption. We also explored the relevance of principles in practitioners’ improvements.

Although results are limited to a non-probabilistic sample, the information we presented here might help practitioners understand the state-of-the-practice of ASD adoption in the country and compare their own practices and maturity in contrast to a previous portrait. Although there are no ground-breaking insights, our results should motivate people to improve and seek for better alternatives to software development in their own ecosystem. Results should also shed some light to researchers with themes that might be of attention for further investigation.