European Journal of Information Systems

, Volume 18, Issue 4, pp 281–284

‘Lots done, more to do’: the current state of agile systems development research

  • Pekka Abrahamsson
  • Kieran Conboy
  • Xiaofeng Wang

DOI: 10.1057/ejis.2009.27

Cite this article as:
Abrahamsson, P., Conboy, K. & Wang, X. Eur J Inf Syst (2009) 18: 281. doi:10.1057/ejis.2009.27

Agile systems development methods emerged as a response to the inability of previous plan-driven approaches to handle rapidly changing environments (Highsmith, 2002). Originating from the so-called ‘light-weight’ methods and promoted through the publication of the Agile Manifesto (2001), the agile method family have become highly prevalent in recent years. Meantime, agile system development research has gained momentum, as is evident from the increasing number of dedicated journal special issues, conferences, conference tracks and workshops. However, practitioners and consultants have largely driven the creation and dissemination of these methods. Agile research has lagged behind practice, as is often the case with new and emerging phenomena in Information Systems Development (ISD).

Current shortcomings in agile systems development research

Despite the fact that existing agile system development research should be lauded in that it is very applied and relevant to industry, the current body of knowledge has several shortcomings. Conboy (2009) and Dybå & Dingsøyr, 2008 conducted two studies in particular which surface issues including clarity regarding what constitutes agility, more research required into the adaptability and extension of agile methods, a deeper understanding of how agile methods are deployed in practice, and an overall necessity to improve the level of rigour in agile system development research. Before introducing the papers in this special issue, it is worth summarising these issues, so one can then get an appreciation for the contribution the special issue papers add to the current body of knowledge.

The need for a better understanding of what constitutes ‘agility’: Any good concept needs a strong underlying logic and rationale that serves as a ‘theoretical glue’ and binds all of the factors together (Whetten, 1989). However, the current body of agile method knowledge suffers from a lack of clarity as to what constitutes agility. It seems almost every piece of research adopts a unique interpretation of agility. This is to be expected to some degree; Lyytinen & Rose (2006) argue that, in the context of ISD, agility as a concept needs to be multifaceted and contextual, and that agility is achieved through various different means depending on the project environment. Based on this argument, every organisation needs to adopt an appropriate interpretation of what agility means to them, and in some cases, this may be unique as opposed to expropriating a single commercial version, such as XP or Scrum. While a universal interpretation is not promising, in order to significantly advance this area of research, it is important to have some solid platform on which to build a cohesive body of knowledge. Work is needed that at least frames acceptable agility concepts in known general contexts.

The need to extend the applicability of agile methods: Agile methods are sometimes seen as largely restricted to small, co-located development teams, non-critical system development, with on demand access to developers, and other such constraints (e.g. Stephens & Rosenberg, 2003). While this view has been contradicted by some, Conboy (2009) suggests that this may not be the case, pointing to the significant volume of research attempting to rebuild and tailor these methods so they can operate in other environments (e.g. Bowers et al., 2002; Crispin & House, 2003; Stotts et al., 2003; Cao et al., 2004; Kahkonen, 2004; Lindvall et al., 2004). It reflects a persistent demand on extending the applicability of agile methods in broader contexts.

The need for a better understanding of agile methods beyond the adoption stage: One of the main focuses of agile method research has been the introduction and adoption of agile methods (Dybå & Dingsøyr, 2008). In contrast, the studies of issues associated with post-adoption use of agile methods are much less in number, even though there is increasing need to have a better understanding of agile methods in use as many organisations have completed adoption stage and agile methods start to become well-established processes of these organisations. Specific needs of organisations and human nature inevitably lead to diverse interpretations and implementations of a method, which in turn lead to different, sometimes surprising, effects and consequences of use of agile methods and associated practices.

The need for more rigorous studies: Dybå & Dingsøyr's, 2008 systematic review of 33 primary empirical studies of agile software development (up to and including 2005) reveals a need for more rigorous studies in agile research. They have discovered that the research methods used in these studies were not described well in general; bias, validity, and reliability issues were not always addressed; and data collection and analysis processes were often not explained well. Consequently, the contribution of the existing agile studies regarding the benefits and limitations of agile methods and for decisions related to their adoption is ‘very low’ and ‘very uncertain’ (Dybå & Dingsøyr, 2008, p. 851).

Advancing agile software development: contribution of the special issue

The seven papers of this special issue on agile software development have addressed the abovementioned issues to various extents and from different angles.

A better understanding of what constitutes ‘agility’

One paper in this special issue, ‘Assessing the Relative Contribution of the Facets of Agility to Distributed ISD Success: an Analytic Hierarchy Process Approach’, helps to gain a better understanding of agility concepts in a distributed systems development context. Based on the observation that previous research has not attempted to assess the relative importance of the various types of agility with respect to different systems development success measures, in this paper, Sarker, Munson, Sarker, and Chakraborty identify which agility facets facilitate success of distributed systems development projects from the perspective of managers and technical members, as well as from an integrated view of both stakeholder groups. Their work enriches our understanding of agility concept through empirical evidence of the specific effect of agility elements on software development effectiveness.

Extending the applicability of agile methods

Across the papers in this special issue, we can see three diverse approaches to implementing agile development in environments to which they would not normally be suited. First, Karlsson and Agerfalk illustrate a formal and methodological approach where the method is constructed ‘from the ground up’. Preserving and emphasising agile goals and values in a method can help avoid that method being ‘reduced to a series of steps executed by rote’ (Highsmith, 2002, p. 14). Drawing on the Method for Method Configuration, a method engineering approach to tailoring software development methods, their paper ‘Exploring Agile Values in Method Configuration’ provides insights and examples of tailoring an agile method while maintaining and promoting agile goals and values of the base method.

Combining agile methods with other existing approaches is also a promising avenue to extend the applicability of agile methods. In their paper, ‘Simulating Mixed Agile and Plan-based Requirements Prioritization Strategies: Proof-of-Concept and Practical Implications’, Port and Bui discuss the efficacy of mixing two strategies for requirement prioritisation to incorporate the benefits of agile development methods and the plan-based approaches. They argue that a mixed strategy for requirement prioritisation outperform either agile or plan-based strategies, surprisingly, even within their respective home grounds. They also provided two instances of the mixed strategies, and outline a framework for analysing and assessing their effectiveness.

The third paper, ‘A Framework for Adapting Agile Development Methodologies’, authored by Cao, Mohan, Xu, and Ramesh, examines how the structure of agile methods, projects, and organisations affect the adaptation of agile methods. While a substantial amount of existing research has examined sole case studies describing how a particular method was forcefully tailored to suit an environment, this paper examines how the structure of agile methods, projects, and organisations affect the adaptation of agile methodologies and the factors that should be considered in such an environment.

A better understanding of agile methods beyond adoption stage

From a perspective of innovation diffusion, there are three papers of the special issue going beyond agile adoption and adaptation stages by examining the use of agile methods in various real-world settings. In ‘Acceptance of Software Process Innovations – The Case of Extreme Programming’, Mangalaraj, Mahapatra, and Nerur focus on the acceptance of agile practices in terms of the innovation diffusion cycle (Kwon and Zmud, 1987), and aim at providing insights into individual, team, technological, task, and environmental factors that expedite or impede the acceptance of various practices of eXtreme Programming (XP). They showed that how the same set of XP practices are used differently by two teams in the same company.

In contrast, ‘Role of Collective Ownership and Coding Standards in Coordinating Expertise in Software Project Teams’ is concentrated on two XP practices only: collective ownership and coding standards. Based on the understanding that the ability of expertise coordination is critical in an increasingly dynamic environment, Maruping, Zhang and Venkatesh examine the relationship between collective ownership, coding standards and expertise coordination, and software project technical quality. They show the positive role played by the two practices in improving the technical quality of software projects and different moderating effects of the two on the relationship between the quality and expertise coordination.

Unlike other papers in this special issue that argue for the effectiveness of agile methods and practices, John McAvoy and Tom Butler unveil the ineffectiveness of agile methods in terms of decision making. In their paper ‘The Role of Project Management in Ineffective Decision Making within Agile software Development Projects’, they discover several potential factors in agile projects that can negatively affect the efficacy of decision making by agile teams. For example, the high level of empowerment of a cohesive agile team can exhibit groupthink or Abilene paradox. Therefore, rather than acting as a facilitator, the role of project manager in agile teams may well be re-oriented as devil's advocate.

Increasing research rigour

The seven papers in the special issue have been selected not only because of their contribution to a better understanding of various phenomena in agile system development. They are included in this volume also because of the rigour with which the studies reported in these papers have been conducted. The rigour is shown firstly through the well-crafted research methods that have been applied in these studies. Across the seven papers, the deployed research methods include simulation (Port and Bui), multi-case study (Cao et al., Karlssona and Ågerfalk, and Mangalaraj et al.), single longitudinal case study (McAvoy and Butler), and survey method (Sarker et al. and Maruping et al.). The applications of the research methods are all argued with adequacy and presented with clarity in the papers. Secondly, the rigour of the studies is shown through the sound theoretical base upon which the studies have been conceived and the findings drawn. The theoretical base has been constructed with the same merits as the research methods in these papers. Adaptive Structuration Theory (Cao et al.), Innovation Diffusion (Mangalaraj et al.), Method for Method Configuration (Karlssona and Ågerfalk), etc. have inspired and provided the studies with sound theoretical grounding.

Emerging research directions: beyond the special issue

Agile system development is a highly dynamic domain with constantly changing and emerging phenomena that are worth of research efforts. The papers in the special issue are addressing the research topics that have high relevance currently, as shown in the previous section, meanwhile indicating the future directions to explore. The empirical studies of agile system development published up to and including 2005, reviewed in Dybå & Dingsøyr (2008), covered four main groups of research topics: introduction and adoption, human and social factors, customer and developer perceptions, and comparative studies. Most of the papers included in this special issue have addressed one or more than one topics in these groups. In this sense, the special issue can be seen as a continuation of the research efforts of agile research community.

Meanwhile, the special issue also provides insights on the aspects of the body of knowledge of agile research, which could be improved by future studies and a number of potentially new avenues for research.

Early in this discussion, we stressed the need to understand what constitutes agility. If a better understanding of agility concept can be achieved, we need this be taken forward into something measurable. The need is ensued for researchers to identify rigorous ways in which agility can be measured and assessed. Obvious benefits of measurability include the ability to compare agility across projects, and identify improvements or decline in agility over time. A further step would then link measured agility to measured outcomes, such as project or product success. For example, Sarker et al.'s work can be taken as an initial step to bridge the understanding of agility to constructs that can be measured (even though they have not done so in their study), then link them to system development success. Another type of link for which supporting evidence is much needed is between the true ability of agile methods and practices and innovation. While many of the proprietary agile methods texts do explicitly state that facilitating innovation is a key motivation underpinning the emergence and use of agile approaches, rigorous research evaluating innovation in an agile context is desired and could be highly valuable.

Another future direction is implied by the increasing number of studies on agile methods beyond the adoption stage, including the papers in this special issue, which look into the actual use of agile practices and issues, and challenges around it. With agile methods being routinised and infused in the adopting organisations, one of the most pressing issues is the need to develop a better understanding of the implementation of agile at the organisational level. Mangalaraj et al. suggest that there are myriad issues and challenges that an organisation needs to overcome to sustain agile methods. Agile adoption usually takes place from the ‘bottom up’, within small development teams and championed by a small number of highly effective people. Despite initial success at the team level, some of these teams are then finding it difficult, if not impossible, to implement agile beyond their own boundaries. As a result, they are constrained by many of the functions they are dependent on to get work done. Future research could examine the typical dependencies faced by agile ISD teams, current best practices regarding synchronisation of agile and non-agile functions, and strategies for organisational level implementation of agility in ISD environments. Specific functions can include finance and contracting legal and human resources.

Our aim in developing this special issue was to deliver a compilation of research papers that represent the most recent research and advanced practices related to theory, practical applications, and implications of agile methods in information systems. By providing this venue, we expect to advance both theory and practice in information systems agility. We are delighted to present the results in the pages that follow in this issue: a better understanding of what constitutes ‘agility’ and agile methods, demonstrated approaches to extending the applicability of agile concepts into new arenas, and a collection of fine exemplars of rigorous research in agility.


We thank all the reviewers who have contributed to this special issue through their constructive, thorough and critical reviews, which helped the authors improve their papers. Special thanks also go to Professors Richard Baskerville and Brian Fitzgerald who provided us with valuable guidance and suggestions in the process of editing the special issue.

Copyright information

© Palgrave Macmillan 2009

Authors and Affiliations

  • Pekka Abrahamsson
    • 1
  • Kieran Conboy
    • 2
  • Xiaofeng Wang
    • 3
  1. 1.VTTFinland
  2. 2.NUI GalwayIreland
  3. 3.LeroIreland

Personalised recommendations