1 Introduction

The two most common methodologies of software development to date are agile and waterfall. According to surveys, 71% of businesses prefer agile, whereas 51% of businesses utilize waterfall (Serebryantseva 2022). When developing a software project, specific practices and standards are described as being used in a software development methodology. There are many different methodologies available to developers, each with its own set of quirks. But there isn't a standard methodology that fits all projects, so it's crucial to pick the method that will benefit the project the most. The overall development process is more effective and seamless when a set methodology is followed since everyone on the team is aware of their roles and due dates. Additionally, it makes it easier for the team leader to move through the project and divide the work efficiently to avoid disruption and delays. A poor software development methodology, however, might seriously impede the project's progress or even cause it to fail. The list of potential issues includes anything from missed deadlines and subpar goods to high-risk misunderstandings with the customers and within the team (Mishra et al. 2021).

Understanding the flow of several development methodologies is crucial, especially agile and waterfall as the two most used methodologies. Due to its organized and structured phases, the waterfall methodology is still commonly employed. When the requirements are fixed, functioning software is one of the advantages of waterfall (Kuusinen et al. 2016). Waterfall, however, suffers from several drawbacks, including an inability to react to requirements changes and a lack of client collaboration. Agile methodology, on the other hand, is exceptional since they place a strong emphasis on clients’ interaction and treat them like team members. Also, in agile development, the development process may be made faster by timely addressing the customer’s feedback (Almeida and Simões 2019).

The software firms realized that certain project aspects, for instance, purpose, requirements, resources, architecture, and scale will determine which methodology best suits them—either agile or waterfall or even a hybrid of the two (Bishop and Deokar 2014). Anecdotal evidence and instances of overcoming hardship by well-known specialists over the past few years show that using suitable methodology is sometimes more effective and sensible (Chandran and Das Aundhe 2022; Fagarasan et al. 2021). Important issues are evolving, and so are people and their perspectives (Bishop and Deokar 2014). Although waterfall development is still necessary, utilizing the agile methodology often leads to higher progress. Accordingly, in some circumstances, like a large project with scattered teams, combining both methodologies may be the best choice (Alam et al. 2022).

Consequently, employing a combination structure is the best course of action. This highlights the reality that the issue is frequently not just the variety of methodologies that are now available but also the proper use of the existing methodology (Thesing et al. 2021). In light of this, the novelty of this study lies in its examination of this viewpoint to present an insightful perspective on when agile is more suitable than waterfall, and vice versa. There has been a scant comparative study published despite an ongoing argument about whether the method is more successful (Włodarski et al. 2020). It also evaluates whether software development companies should adopt a hybrid approach that incorporates both agile and waterfall methodologies. This study will particularly answer the following questions:

When is the agile methodology preferable over the waterfall methodology, and vice versa?

When is it best to combine the waterfall and agile methodologies?

By addressing the aforementioned research questions and completing a comprehensive analysis of both waterfall and agile approaches, the findings of this study contribute to the corpus of research as follows.

  • In light of this, this paper discusses a broad overview of the traits of both waterfall and agile methodologies. Data were gathered from currently available literature (e.g., conferences, journals, white papers, blogs, and professional and industrial websites like “Standish group report” (The-Standish-Group 2020), “Pixel Plex” (Serebryantseva 2022), and “Vitality Chicago” (Mersino 2022). This was essential to investigate the two methodologies as well as the different forms of combination frameworks applied in real-world contexts.

  • This paper suggested a decision tree that incorporates several scenarios of whether using agile or waterfall approaches is the most effective.

  • The paper detail situations in which businesses would need to use hybrid methodologies—combining agile and waterfall—for certain projects. If a hybrid methodology is taken into consideration, the model will offer trustworthy insight into which components of the different strategies should be blended or which model is the best option.

The remaining sections of the paper are organized as follows. Section 2 discusses the research background and related work. The properties of agile and waterfall are covered in Sect. 3. Section 4 compares agile and waterfall methodologies. The possibilities of a hybrid methodology of agile and waterfall are covered in Sect. 5. Section 6 discusses the findings and limitations of this paper. Section 7 concludes this paper and indicates future research directions.

2 Background and related work

Software development methodologies have been through several modifications; agile and waterfall methodologies are continually improving to fit new patterns and, most importantly, maintain firms' competitiveness. Today's software firms are looking for methods that succeed, which has led to the development of several combination frameworks. the literature that compares agile and waterfall methodologies or advocates merging the two methodologies in the development life cycle is scanned in this section. Despite the enormous number of studies that have been reported for this aim, this section reviews a few of them. Because there have been many studies published on this topic, we examined the literature at a high level in this section. Even though other studies have compared the agile and waterfall approaches, this article stands out because it presents the most recent research on the subject and offers a decision tree that software project organizations may use to determine when and how the two methodologies may be merged. Moreover, for businesses to consistently execute successful projects, this article makes the point and provides evidence for the need for some form of a hybrid methodology that combines agile and waterfall methodologies. According to our investigation of the literature, three main categories were identified in the context of agile and waterfall methodologies: comparison between the two methodologies, the transition from waterfall to agile, and the hybrid combination between the two methodologies. This section briefly investigates the literature about each category.

2.1 Comparisons between agile and waterfall methodologies

Agile methodology and communication that improve software development have been the subject of several studies (e.g., Alzoubi et al. 2016; Friess 2019; Ammad et al. 2019; Alzoubi and Gill 2021, 2020; Shameem et al. 2020; Hummel et al. 2015). Numerous studies (e.g., Kuusinen et al. 2016; Friess 2019; Al-Saqqa et al. 2020; Dingsøyr et al. 2018; Alzoubi et al. 2015) have also examined other elements of agile development, such as customer interaction, validation, verification, cost reduction, incentives, and increased performance. All of these studies found that adopting agile concepts, such as customer involvement, rapid iteration development, and informal communication, boosted project success. Many previous studies have compared waterfall and agile methodologies and asserted that agile increased the likelihood that the project would be successful, monitored all processes during each iteration to provide the client with a complete understanding of the software product, and could rapidly repair defects. Table 1 summarizes a few of these studies.

Table 1 Summary of the recent waterfall versus agile literature comparison
Full size table

According to Kumar and Bhatia (2014), agile methodology is responsive to changes, has less paperwork, and has rapid development, which may lower the cost and time of development. However, the waterfall is a reusable component that may be used to shorten the project's duration and provide it with greater order (Kumar and Bhatia 2014). Furthermore, Yadav et al. (2015) stated that using agile methodology may boost a project's success rate since customers are more involved and issues may be fixed more quickly. The waterfall, however, lengthens the development process (Yadav et al. 2015). According to the results of a study done by Khoza and Marnewick (2020), agile methodologies are 60% more effective in process development than waterfall, however, there is no distinction between projects that use waterfall or agile methodologies in terms of the success of the business. The findings showed that success in project management is higher for projects that employ a waterfall methodology (Khoza and Marnewick 2020). Regarding the deliverable's success, there is a distinct difference. Although this was not true for all initiatives, projects that used agile methodologies were more successful than waterfall projects (Khoza and Marnewick 2020).

Based on an analysis of three projects that used the waterfall methodology and three projects that used the agile methodology, Thummadi and Lyytinen (2020) looked at how much the two methodologies cause diversity in software design practices. The analysis revealed that, in both situations, a method-induced variance using the waterfall and agile methodologies accounts for around 40% of all activities, with the other 60% being justified by an individual designer's habits, the project's circumstances, and ambient noise. The impact of the methodology on software design activities is typically less significant than anticipated, and the impact of the designers and project's circumstances on process and outcome must therefore not be minimized (Thummadi and Lyytinen 2020).

More recently, Alam et al. (2022) examined the relationship between the size of the project and the best methodology (i.e., agile or waterfall) since they believed that the methodology chosen is mostly responsible for software quality. The six-pointed star model's factors were used to conduct the comparison. The findings indicated that for small projects, practically all methodological criteria are in favor of agile methodologies. Both methodologies are largely similar for medium-sized projects (Chandran and Das Aundhe 2022). The waterfall is significantly more suitable for all factors of the six-pointed star model, according to the results for large projects. Since agile is a paradigm that prioritizes the needs of the client, the respondents favor it (Alam et al. 2022; Dursun and Goker 2022). Thesing et al. (2021) conducted a review of the literature to determine when agile or waterfall methodologies were better suited for various circumstances. They then verified their responses through interviews with subject-matter specialists. Their findings indicated that which approach is best depends on several criteria, including time, cost, scope, organizational structure, and team qualities.

In order to evaluate the effects of a certain development strategy on the achievement of student computer endeavors, Wodarski et al. (2020) described an experiment that was conducted at two engineering institutions in Europe. In order to identify trends related to the use of waterfall, agile, and no-process methods on the aspects of quality of product and teamwork and team productivity, a systematic, metric-based evaluation scheme was used. The results showed that, in comparison to the waterfall team, an agile strategy resulted in stronger team cohesiveness and superior production. However, there was a decline in functional sufficiency. Similar to this, agile has lower quality metrics scores than waterfall. It appears that agile teams prioritize delivering a demonstrable piece of software on schedule while somehow ignoring its quality (Włodarski et al. 2020).

Sinha and Das (2021) investigated the implementation of the software quality assurance or testing phase and how it will function with the agile approaches. The results showed that while the waterfall technique is rigid and ineffective at communicating, it offers greater structure and quality assurance. The agile approach, on the other hand, is particularly effective if the sprints are completely dedicated to bringing about new functional enhancements as opposed to resolving quality problems early in the SDLC process. Additionally, using agile methodologies, serious performance flaws aren't found until a late stage of project development (Sinha and Das 2021).

Raza (2023) outlined the differences between the agile and waterfall methodologies from an industry perspective. Waterfall projects have a sequential structure, a high emphasis on documentation, and little interaction between engineers and clients. Agile, on the other side, emphasizes better collaboration between developers and consumers through reduced paperwork. Although agile methodologies are popular among software companies, they lack a consistent system architecture, an agile mentality isn't completely adopted throughout the business, and they are of poor quality (Hamilton 2023).

2.2 Transition from waterfall to agile

Many previous studies have reported the transition from waterfall to agile. Table 2 summarizes a few of these studies. Kuusinen et al. (2016) developed a mixed approach centered on performing decomposition tasks or functions in a waterfall while adhering to agile concepts. For agile to fit a traditional waterfall company, Richter et al. (2016) explored applying agile as a semi-structured framework that applies bottom-up change. Dima and Maassen (2018) used the Delphi technique with surveys and interviews to identify the most common software development methods in the IT industry and their effects on management. The findings showed a trend in favor of the agile software development methodology as it quickly meets changing consumer demands (Alzoubi and Gill 2022). According to this study, software development methods vary as technology dynamics do at the business level (Dima and Maassen 2018). Even though the agile methodology has several advantages, including regular customer contact, teamwork, and greater assessments of the advancement achieved toward goals, a major problem is the pressure on team members to handle growing pressure from the outside (Dima and Maassen 2018; Mishra et al. 2018).

Table 2 Summary of the recent literature on transition from waterfall to agile
Full size table

In their paper, Kasauli et al. (2020) summarized the findings of an exploratory case study that concentrated on two divisions of a sizable systems engineering firm that was adopting agile. They described the difficulties that teams encountered while transitioning to agile, and they suggested potential solutions to the problems. The problems were due to the lack of developers' awareness of the changes of requirements, managing these requirements, and the lack of basic agile knowledge like user stories (Kasauli et al. 2020). Fagarasan et al. (2021) and Almeida and Simões (2019) assessed the agile methodology capabilities inside waterfall software. This assessment took into account variables including predictability, practicability, and intricacy. The findings demonstrated that Scrum is the most approved agile methodology since it can handle complicated issues and quickly adjust to changing market conditions (Fagarasan et al. 2021).

The total effect of agile methods on the software development, developers, and management of project dimensions was examined by Wafa et al. (2022). To investigate the effect of agile methodologies on Pakistan's IT business, they polled several software engineering experts from various backgrounds. The same data were then quantitatively analyzed. The results imply that applying Agile methods and efficient software development are strongly correlated. The investigation demonstrates that the achievement of projects for software development in terms of quality, client satisfaction, cost, and time depends critically on agile methods (Wafa et al. 2022).

To respond to the inquiry, "What effect does increasing aspects of quality have?", (Saarikallio and Tyrväinen 2022) studied this inquiry by putting into practice treatments based on earlier research and information obtained from semi-structured interviews, and the concerns were addressed. The setting was an established company with a complicated structure of revenue streams, so the combination of the project, service, and product-based work made the adoption of agile methods problematic. As evaluated by reported flaws, interventions significantly improved quality. In order to move the software industry forward, it is therefore argued that agile methodologies are insufficient on their own without a quality-focused environment (Saarikallio and Tyrväinen 2022).

Agile transformations, according to KnowledgeHut (2023) company, may contribute to all agile advantages when done well including responsiveness and flexibility into an organization's basic foundation, and its effects are seen at every level of the company. However, as the numbers demonstrate, firms frequently embark on an aggressive full-scale change only to realize midway through that it is not succeeding (ProductPlan 2023). Agile transformation failures can occur for several reasons, including the improper application of prominent agile techniques, a lack of management support, a lack of awareness of the approach that is best for your firm, and hurrying the transformation process (KnowledgeHut 2023).

2.3 Hybrid methodology combining agile and waterfall methodologies

The third category of the literature identified focused on the hybrid combination development of agile and waterfall in order to enhance the development process of the traditional methodologies. Table 3 summarizes a few of these studies. Käpyaho and Kauppinen (2015) used a mixed approach and presented a framework to tackle waterfall's prototype documentation concerns and improve communication by deploying agile concepts in the development process. According to Soundararajan and Arthur (2009), Kumar and Bhatia (2014), and Stoica et al. (2013), combining agile and waterfall might be advantageous while concentrating on the needs of each methodology.

Table 3 Summary of the recent literature on combining waterfall and agile
Full size table

Richter et al. (2016) discussed a framework that supports a dual application model (combining the finest elements of traditional and agile methodologies). Following the agile methodology, the combination framework is more helpful since customer participation is increased (Nerur et al. 2005). The fundamental goals of utilizing agile methodologies and how they may be effective throughout SDLC were discussed in Soundararajan and Arthur (2009), Stoica et al. (2013), Srivastava et al. (2020). They addressed how agile methodologies improve software quality, self-management, and efficient communication between team members and stakeholders (Soundararajan and Arthur 2009; Stoica et al. 2013).

Signoretti et al. (2020) conducted a case study involving two software development organizations from a major business that are migrating to the hybrid approach in order to determine the failure and success aspects of a hybrid approach deployment. Focus groups and semi-structured interviews were employed to gather information. The results revealed a number of success elements, including team autonomy, communication, trust, and team involvement. The failure reasons, on the other hand, include a business policy, budget allocation, and team autonomy at stake. The authors also discussed potential remedies for the issue of "product focus rather than project focus," which teams perceived as being of the highest significance for the change (Signoretti et al. 2020).

While companies are currently under continual pressure to create software more quickly owing to new demands arising around the world, this ongoing endeavour results in unique development methods that are ostensibly motivated by practice (Kirpitsas and Pachidis 2022). Modern approaches are neither fully linear nor fully agile, so choosing the right mix of strategies to achieve objectives and ensure value generation for enterprises presents a problem. Technology companies still need to ensure value generation and conformity to industry norms, directives, and laws. Similarly, it is anticipated that IT auditing will continue to evolve in line with changes in company strategies, sociocultural trends, and the requirement for IT governance in the coming years (Kirpitsas and Pachidis 2022).

In recent years, the usage of hybrid models, which include aspects of both waterfall and agile methods, has grown in favour since it enables businesses to benefit from the advantages of both methods (DevFacto 2023). The hybrid method is largely supported by the industry as a way to enhance their development practices. The hybrid model combines the structure and consistency of the waterfall technique with increased versatility and adaptability to change. The hybrid approach does raise certain issues, though, such as the necessity for cautious project management of deadlines and schedules, as well as the possibility for confusion and disagreement amongst members of the team who are accustomed to working in various methods (ProjectManager 2023).

3 Agile and waterfall developments mechanisms

This section discusses an overall view of the characteristics of the waterfall and agile methodologies. Different agile framework concepts are introduced here including Extreme Programming (XP), Scrum, and Kanban to underline the characteristics of agile frameworks.

3.1 Agile development

When changes to the software project are needed, the waterfall methodology's drawbacks and rigidity become apparent. Several software development experts convened and decided to introduce a new methodology that can handle these changes. After the agile alliance group gathered in 2001, the term "agile" first appeared (Poppendieck and Cusumano 2012). According to Kuusinen et al. (2016), agile methodologies place a greater emphasis on people, working software, collaboration, customer cooperation, and change than they do on waterfall sequential, procedures, contracts, extensive documentation, and plans. Since then, agile methodology has grown in popularity throughout the business (Käpyaho and Kauppinen 2015). Agile development frameworks and phases are discussed in the following sections.

3.1.1 Agile development frameworks

Several frameworks are available for the agile methodology, including Scrum, Kanban, XP, Behavior-driven development, Feature-driven software development, Dynamic Systems Delivery, and Crystal, among others (Gustavsson et al. 2022). We won't go into detail about all of the agile frameworks in this paper; instead, we'll concentrate on the three most popular ones: Scrum, Kanban, and XP. The two pillars of scrum development are management and speed. Approximately two weeks pass during a scrum sprint, and a client-tested version of the finished product is produced. In turn, XP seeks to quickly supply the solution (Ahmed et al. 2023). Test-driven development, in which automated unit tests are written before the actual code, is the main tenet of XP. Kanban, on the other hand, makes extensive use of visualization and a unique tool called a Kanban board. There are various columns in this board, which might be digital or physical, including backlog, in process, testing, and done. The jobs shift across columns, making it obvious how the project is progressing.

3.1.2 Agile development phases

Agile methodology emerged in reaction to the increasing need for a rapid and more flexible method of development (Halani and Jhajharia 2022). Sprints of one to four weeks are used in the agile workflow (e.g., the Scrum method), and there are weekly sprint reviews when members of the team report their progress and set plans for the upcoming sprint. In order to be prepared to make any appropriate changes and guarantee that everything satisfies the customers’ requirements, the team also processes feedback from customers. Agile places a greater emphasis on software quality than documentation, which makes documentation less crucial. The Scrum Master, the Product Owner, testers, developers, and data engineers make up a typical agile team. A special position that acts as a liaison between the customer and the team is that of the Product Owner. The project's success is the responsibility of the Product Owner. To do this, the Product Owner must ensure that there is effective communication with customers and that everyone knows what is required (Kavlakoglu 2022). Typically, as shown in Fig. 1, the following phases dictate how software is developed using the agile methodology (Haag and Cummings 2009).

  • Initiation: The major project requirements, including functionality and intended outcomes, are delivered to the team by the product owner. At this phase, the timelines and costs are also projected.

  • Iteration 1: Development starts after the team has all the information. The user interface and the software architecture are first designed as a prototype.

  • Testing and review: In this phase, the customer evaluates the feature that has been created and makes changes or additions. Additionally, testers look for any problems.

  • Iteration 2: Bug fixes and changes are handled during this stage. The following product components are also produced. Testing and review processes are repeated numerous times until the solution matures into a finished product and the customer is happy with the results.

  • Release: The product is ready for release once the quality assurance team certifies that the solution functions without a hitch and poses no security risks.

  • Maintenance: The team offers post-release assistance to make sure the solution operates perfectly. They further go through iterations to improve the current product.

Fig. 1
figure 1

Agile methodology phases

Full size image

3.2 Waterfall development

The software development process, in the waterfall, starts with extensive planning and analysis. Each subsequent step can start only after the preceding one is finished (Yu and Mishra 2010). This methodology is quite rigid and unresponsive to changes. Each component of the final product is considered and documented before the process starts, which is one of the waterfall's main pillars. Progress can be easily monitored and quantified because the entire scope of the project is outlined in advance (Yu and Mishra 2010). A waterfall team consists of a project manager, a business analyst, testers, and developers, among other positions. Customers and other stakeholders are only really involved in the initial planning phase and final evaluations. The typical waterfall software development lifecycle phases are depicted in Fig. 2 (Haag and Cummings 2009).

  • Planning: The design, functions, problems, and goal of the project are first defined in this phase.

  • Analysis: At this phase, all business requirements, costs, and schedules are documented.

  • Design: The project's architecture, models, and design principles are put into production once all requirements have been reviewed and decided.

  • Development: All project is broken down into components during the development phase and assembled in a linked sequence until the entire final product is assembled.

  • Testing: The quality crew enters the game, in this phase, once the developers have done developing their code. Their responsibility is to look for errors, security holes, or bugs. When testers discover an issue, developers are notified to fix the code.

  • Implementation: The finished version is ready for public distribution after development and testing are completed.

  • Maintenance: After installation, the system can be adjusted and changed by the customer's specifications and feedback, or if the system or component does not function as planned.

Fig. 2
figure 2

Waterfall software development lifecycle (Source Haag and Cummings 2009)

Full size image

4 Comparison between agile and waterfall developments

For more than 20 years, software project management has been dominated by waterfall and agile methodologies (Sinha and Das 2021). Each has advantages and disadvantages. In order to construct the decision tree in the following section, this section discusses the pros and cons of each of the two methodologies (Table 4). This table is independent of above citation.

Table 4 Comparison between agile and waterfall methodologies
Full size table

4.1 Agile development pros and cons

If agile didn't have a lot of benefits, it wouldn't have emerged as one of the most successful methodologies (Govil and Sharma 2022). Let's examine the advantages agile methodology offers. Agile is a communication- and customer-oriented methodology that improves customer requirements understanding, feedback, and sharing among team members through informal communication and face-to-face interaction (Kodmelwar et al. 2022). Moreover, agile methodology is fast in detecting bugs and delivering the product (Saarikallio and Tyrväinen 2022). Also, it is flexible with changing the focus of the project based on the customer’s feedback (Hauck and Vieira 2021). On the other hand, agile has some drawbacks, such as unclear outcomes, time and cost constraints (due to this, it may be challenging to guarantee that everyone is on the same page or work may be lost or miscommunicated between team members, especially when people leave and join the group in the middle of projects) (Aitken 2014). Furthermore, if the relationship between management and the technical team weakens, especially when new members join the team and old members depart, which happens frequently in agile development, it could result in time and money waste (Mishra et al. 2021).

4.2 Waterfall development pros and cons

The waterfall is a well-established development methodology that has been approved over many years. Additionally, it comes with thorough instructions and user manuals. Also, it is built on clearly identified requirements, which make the timeframe and cost frame predicted and expected as well as the performance aspects (Kodmelwar et al. 2022). The waterfall methodology also gives the team members more and a progress flow structure so that even if some team members quit or join, the plan won't be significantly affected. However, there are drawbacks to everything. The waterfall methodology lacks the adaptability to accept changes in customer requirements, the rate of development, and the ability to patch defects or security flaws until that phase is complete (Wafa et al. 2022). In addition, the waterfall methodology mandates that customers first outline the project from beginning to end. This may become a problem if the customer is divided about its goals and doesn't become aware of this until the project's execution or a later stage. Moreover, testing occurs at the completion of a waterfall project, the final quality check step requires a lot of time, and there is no review throughout the project lifespan, which could lead to a failure to meet the requirements of the customer.

4.3 Agile and waterfall comparison

Agile adoption is primarily driven by quick development cycles, transparency, productivity, and efficiency (Singer 2022). Based on the Standish group report (The-Standish-Group 2020), Fig. 3 demonstrates that in 2020, less than 15% of waterfall projects and more than 40% of agile initiatives were successfully completed. Moreover, compared to 10% of agile projects, about 30% of waterfall programs failed. There are many causes of the failure of the project, but they can all be traced back to poor project management. Project failures can be caused by a lack of time, misunderstood requirements, or inadequate communication. Software development project failure statistics support the claim that 55% of projects fail due to a lack of time. Furthermore, compared to about 50% of agile initiatives, almost 60% of waterfall projects experienced difficulties throughout the project lifecycle. These challenges were caused by limited time and changing customer requirements (Mersino 2022).

Fig. 3
figure 3

The successful rate among agile and waterfall methodologies (source The-Standish-Group 2020)

Full size image

The recent studies confirm part of the Standish group report, while conflict with others. A 2019 TrustRadius.com study revealed that 42% of respondents believed that an agile project would succeed, compared to 26% for a waterfall project (PremierAgile 2023). According to a study by Khoza and Marnewick (2020), waterfall initiatives only have an average success rate of 47% while agile projects have an average success rate of 88.2%. The success percentage for agile initiatives is greater than 50%, according to 78% of the respondents (Wafa et al. 2022). Agile methodologies have a success ratio that is two times greater than waterfall-based solutions (Leong et al. 2023).

Agile software development projects take 22.4% longer than anticipated compared to 13.65% longer than the estimated time for the waterfall method (Khoza and Marnewick 2020). However, according to a more recent study, agile-based solutions generate products 37% more quickly than waterfall programs (Leong et al. 2023). It is noteworthy that agile software development initiatives cost 42.62% more than the first estimate. Contrarily, waterfall projects come in at 10.16% less than planned (Khoza and Marnewick 2020). Wafa et al. (2022) reported that 68% of the participants in the survey believed agile practices have a greater impact on their productivity. Compared to 53% who found working with the waterfall challenging, 21% felt working with agile was challenging (PremierAgile 2023). According to 68% of respondents, agile has a greater impact on improving product quality than waterfall, 16% believe it has no impact, and 15% believe it has a smaller impact (Wafa et al. 2022).

Based on the pros and cons of waterfall methodology and agile methodologies, discussed in Sects. 3, 4.1, and 4.2, and summarized in Table 1 (Mersino 2022; Ana 2022; Boersma 2022). In summary, the agile approach involves breaking down projects into smaller tasks, allowing for more feedback and adjustments throughout the process. It is flexible in that it allows for changes to be made based on the project's requirements and priorities. Collaboration is also a key aspect of agile, emphasizing teamwork and interaction between team members and clients. However, the agile methodology can present challenges when dealing with large and complex projects and may require extensive stakeholder involvement that can be difficult to maintain over time.

In contrast, the waterfall method follows a sequential process where each project stage must be completed before moving on to the next one. It is a systematic approach that relies on a well-defined project strategy and scope. Documentation is also a key component of the waterfall methodology and is required for each project stage. However, there are several drawbacks to this approach, including its inflexibility compared to agile, making it challenging to adapt to changes in scope or requirements. Managing large and complex projects can also be problematic. Additionally, the waterfall methodology places less emphasis on collaboration and teamwork, which can result in communication breakdowns and project failures.

4.4 Decision tree

The waterfall project management is more established compared to an agile project. Waterfall is a method of project management where all of the elements of the product are laid out in advance. The project moves along on a predetermined schedule. On the other hand, agile uses an iterative process rather than outlining the full project from the beginning. Customers are given immediate access to small components of the product by developers, who then use their comments to refine and create the following component. The project may work better with the waterfall or agile methodologies, depending on its unique characteristics. Figure 4 provides a summary of some considerations that can help to choose the suitable methodology. These recommendations were extracted from several research and professional websites such as (Mersino 2022; Halani and Jhajharia 2022; Kavlakoglu 2022; Boersma 2022; AdobeTeam 2022; Hoory and Bottorff 2022).

  • Project complexity For complicated projects, the decision between waterfall and agile methodologies depends on several variables, including the nature of the project, the amount of flexibility necessary, and any legal needs. The ideal strategy, in the end, is thoroughly analyzing the project's unique requirements and selecting the technique that best suits them. In general, the waterfall methodology works well for simple projects and for making improvements to current products, while agile is a better option for complicated projects. Agile emphasizes communication and collaboration, which may be especially beneficial when executing large initiatives which call for input from several stakeholders. Conversely, a waterfall could be more appropriate for tasks that are well-defined and have a set route to completion. When engaged in initiatives that have stringent regulation standards, this technique can be quite beneficial.

  • Clarity of requirements The waterfall methodology is an excellent method for managing projects when the customer is aware of every requirement from the outset. If the project's requirements are not known from the beginning, the agile methodology works well. The tight regulations and deliverables for each phase ensure that they are met, making waterfall projects more suitable for projects with restrictions or criteria. As an illustration, since safety is a concern, the Department of Defense and the aerospace sector are two industries that would probably choose to employ waterfall over agile.

  • Time constraints The waterfall is all about set timelines, but if they're not too strict, agile would be a good choice. Agile has advantages in that it produces software faster, but it suffers to deliver predictable completion dates and costs for projects.

  • Budget constraints Similar to the timetable, the waterfall is best suited when money is limited, while agile works better when money is infinite or large.

  • Customer involvement level Agile works best when the customer demonstrates true engagement in the development process. Contrarily, waterfall involves little customer engagement in the development process.

  • The team collaboration level The degree of group cooperation agile places a strong emphasis on the value of teamwork and communication. Developers can work more autonomously while using the waterfall methodology because they are each in charge of their components of the project.

  • No methodology is a "magic solution" for all types of projects. Each is best suited for specific project categories that meet predetermined criteria. The strategy is likely to fail if these project conditions are not met, or are only partially achieved.

Fig. 4
figure 4

Decision tree to select if agile or waterfall methodology is more suitable

Full size image

5 The hybrid software development approach

It is evident from the discussion in Sects. 2, 3, and 4, particularly the decision tree section, that neither agile nor waterfall can be used for all kinds of software projects or situations. As a result, a plethora of authors and industry professionals have recently advocated using the hybrid software development methodology (Signoretti et al. 2020). To put it another way, depending on the project, either agile or waterfall should be used, and in certain situations, both methodologies may be used at various stages of the project (Włodarski et al. 2020). The adoption of both agile and waterfall methodologies in the same software development firm is explored in this section using various suggestions and recommendations.

According to Fagarasan et al. (2021), a typical hybrid model observed in the information technology business is that when the starting and planning stages are carried out using the waterfall methodology. Nevertheless, the agile methodology, which emphasizes iterative processes, is used for implementation operations. In order to use and embrace hybrid methodology within a software firm, there are two strategies to enhance the delivery processes (Fagarasan et al. 2021).

  • In the case of young firms that are fresh to software development, it may be easier to utilize the agile methodology at the organizational level to create the delivery model and adhere to the methodology's precepts.

  • A more hybrid methodology is advised for established firms that already have a delivery methodology to combine the conventional methods that have made the firm successful up to this point with the agile methodology.

According to Küpper et al. (2018), the evolution of hybrid software development methods involves three common levels. The first level consists of procedures, which refer to the range of tasks carried out by the development team during the software development life cycle (SDLC) and the guidelines that govern how these tasks should be executed. The second level is frameworks, which encompass one or more techniques and approaches that define the management of the SDLC that the team must adhere to. Finally, the third level is context, which pertains to the overall success criteria and is derived from the project goals.

Leong et al. (2023) suggested that in order to improve the chances of success in agile development, it is necessary to integrate three essential components: people, procedures, and organizational elements, into the project management paradigm. This has led to the development of a new project management methodology called the hybrid methodology. Kirpitsas and Pachidis (2022) stated that in the hybrid method, the creation of a prioritized product backlog of user stories follows the design phase to support the agile process. The team then engages in iterative development, primarily following agile principles and gradually expanding on those requirements. During the sprints, the development team conducts unit testing, while systems integration testing and user acceptance testing are carried out once the iterative development is complete. The tested and developed code can then be delivered to customers or put into production, along with the necessary documentation.

In most businesses, there is no clear winner when it comes to methodologies; therefore, agile and waterfall must coexist because both have a lot to contribute. Some projects require both the responsiveness of an agile methodology and the discipline of a waterfall methodology. Taking the best of the two could be preferable, but it is unclear how to do so, particularly when it comes to handling criteria in a large project (Kasauli et al. 2020). Positive results are possible when combining the two thanks to expectations and interaction (AdobeTeam 2022). For instance, unlike a waterfall, a hybrid methodology does not require engineers to wait for one phase to end before moving on to the next (Leong et al. 2023). Instead, as quickly as feasible, the subsequent phase starts (Khan and Akbar 2022). As a result, the project's components and modules are developed independently (one of the key agile features). The documentation may still be kept up to date and deadlines can still be reached because the distinctive waterfall characteristics are also present (Kassab et al. 2018). Hybrid methodology benefits from lower project costs, faster development, and well-managed documentation (Morgan 2018).

Management of projects is changing. Teams that were formerly compartmentalized are now expected to collaborate to reduce reporting hassles and improve visibility across all task kinds. To increase team productivity, complete faster production, provide precise metrics to all project stakeholders, and foster a climate that is conducive to all forms of work, 44% of project directors embrace a hybrid methodology, according to AdobeTeam (2022). Adapting the principles to match the team is one of the basic principles of agile project management. Agile teams that rigidly follow the rules they've established lose the objective of the sense of agility altogether (Kirpitsas and Pachidis 2022). This demonstrates that hybrid methodology is feasible and appropriate for many projects. A hybrid methodology might combine the waterfall's organized up-front project plan with an agile emphasis on delivering small software components (Boersma 2022). This hybrid methodology accomplishes some of the favorite parts of waterfall methodology while completing tiny software components faster. Customers prefer receiving software more rapidly, while management loves the more disciplined approach to software development (Dima and Maassen 2018). Many developers value being free from the numerous sessions that agile requires of their members (Hoory and Bottorff 2022).

Some software development experts blend a structured, waterfall methodology with elements of the agile mindset that succeed for them. The team members may find genuine benefit in conducting frequent retrospectives to pinpoint opportunities for improvement. Whatever the cause, the best way to ensure success is to address the plan for project management by selecting a methodology that works for team members or the specific project (Boersma 2022). By employing a solution that supports mixed methodologies, the hybrid methodology can help all teams and the company as a whole by enhancing productivity and visibility (Prenner et al. 2022). The hybrid methodology may enable total insight into what has been done, what is being done, and what to do in the future (Gill et al. 2018). This will not be possible without increased visibility and adequate planning, training, and benchmarks. While project management may select waterfall to ensure dependencies and deadlines, developers may select agile to improve code productivity. Providing teams with the freedom to pick their work manner boosts productivity (Özkan and Mishra 2019).

6 Discussion

This review investigated the perennial debate over whether agile or waterfall software management methodologies are better. This study addressed the following two research questions in order to resolve this conflict: When is the waterfall methodology preferred over agile, and vice versa? (RQ1) And when should waterfall and agile approaches be combined? (RQ2). Many firms are still using the waterfall methodology since it simply works and has a successful track record, even though it has begun to be replaced by agile methodology as the industry norm for software project delivery. The managers who use a waterfall are at ease because everything is thoroughly recorded, and a centralized decision-making procedure minimizes human mistakes.

Waterfall and agile are two distinct management methodologies that work best for particular project types. Waterfall could be the greatest option when the project goals are known from the beginning. When a project must adhere to rigorous standards, the waterfall methodology is preferable because it calls for deliverables for each step before moving on to the next (Pai et al. 2019). Agile, on the other hand, is especially suitable for teams that intend to move quickly and don't know exactly how the project will turn out before they begin. Agile demands a dynamic team that is cooperative and self-organized, as well as regular progress updates from customers and enterprise owners. Agile philosophy has impacted software development practices overall. Despite this, there is growing concern regarding software design.

Achieving high quality may be challenged with the adaptability of any improvement strategy that neglects to adequately take the design issue into account (Arcos-Medina and Mauricio 2019; Schmalz et al. 2014). The failure rate was identified, in the Standish Group report, based on project size. Smaller projects are more likely to be successful than larger ones since they include shorter feedback cycles and lower risk (The-Standish-Group 2020). Small projects have a success rate that is three times higher than large ones for agile projects, and for waterfall projects, small initiatives have a 6X higher success rate than large ones (Mersino 2022). Additionally, the feasibility of combining the waterfall and agile methodologies in software development management is investigated in this paper. To take the finest elements of each methodology and use them in various project contexts, many experts and authors suggested creating a hybrid combination of the two methodologies (Alam et al. 2022).

7 Conclusions and future directions

Agile software development has been advocated for more than 20 years as a solution to the rigidity issue with structured/traditional methodologies such as the waterfall. However, current evidence indicates that many software firms continue to use the waterfall methodology. Although there are many alternative methodologies available for software development, the waterfall and agile methodologies are generally regarded as the most efficient. The flexibility of agile and the stability of the waterfall are the two most substantial benefits. There are ongoing discussions regarding whether Agile Methods or structured methodology is better suited for software firms, which never seem to end. Therefore, it is important to thoroughly grasp the project scope before selecting a development methodology to ensure that the project is on the right track.

The goal of this study was to investigate this claim by comparing the waterfall and agile methodologies in order to determine why both approaches are so popular. The article also included a decision tree that could be used to decide whether the agile or waterfall methodology is better suited for a certain project setting. The study came to the conclusion that no single silver belt methodology is appropriate for all kinds of software development projects. The software firms may therefore need to use a hybrid framework that combines agile and waterfall. Utilizing the framework's possible beneficial usage of both agile and waterfall methodology's features is its core goal. Because it's popular and hip, some people strive to incorporate agile into every project they work on. However, the waterfall methodology is a far superior choice in other sectors, such as marketing and construction. It is crucial to remember that agile is a very flexible methodology that can be tailored to the specific requirements of the firm. While the waterfall methodology is quite rigid and it is very formal, it too may be altered as needed. Therefore, while it is advised to adhere to the fundamental tenets of each methodology to ensure that it functions as intended, firms could also adapt their own methodology and adjust it as necessary.

Future research might concentrate on examining an actual company that uses a hybrid software development methodology. This is essential to learn how and when they employ agile or waterfall, and what kinds of projects they successfully accomplished utilizing the hybrid methodology. All of that will assist in validating the findings of this paper. To sum up, successful firms must be able to deliver software on a massive scale. It is advised that firms utilizing the current waterfall software delivery paradigm implement agile concepts as the digital world is already at a pivotal time.