A measurement framework for assessing the maturity of requirements engineering process

Abstract

Because requirements engineering (RE) problems are widely acknowledged as having a major impact on the effectiveness of the software development process, Sommerville et al. have developed a requirements maturity model. However, research has shown that the measurement process within Sommerville’s model is ambiguous, and implementation of his requirements maturity model leads to confusion. Hence, the objective of our research is to propose a new RE maturity measurement framework (REMMF) based on Sommerville’s model and to provide initial validation of REMMF. The main purpose of proposing REMMF is to allow us to more effectively measure the maturity of the RE processes being used within organisations and to assist practitioners in measuring the maturity of their RE processes. In order to evaluate REMMF, two organisations implemented the measurement framework within their IT divisions, provided us with an assessment of their requirements process and gave feedback on the REMMF measurement process. The results show that our measurement framework is clear, easy to use and provides an entry point through which the practitioners can effectively judge the strengths and weakness of their RE processes. When an organisation knows where it is, it can more effectively plan for improvement.

Appendices

Appendix A: An example of requirements category assessment

The following example shows how REMMF measures the capability of the ‘describing requirements’ category. The practices listed in Table 3 define the describing requirements category. Practice “DR1” is highlighted as an example.

Table 3 Measuring capability example

Three elements of each RE practice are measured: the approach, the deployment and the results. The objective is to assess the strength of an individual RE practice as well the RE process category.

The first of the three measurement elements is based on the participant’s understanding of the organisation’s approach to the RE practices, i.e. the organisation’s commitment and management support for the practice as well as the organisation’s ability to implement the practice. Table 4 gives an example of how a participant might respond. The RE practice is as follows.

Table 4 Approach

1.1 DR1: Define standard templates for describing requirements

The respondent should tick one of the options in the ‘Score’ column. Using their expert understanding and by collecting relevant information from different sources, imagine the respondent selecting: Weak (2) (i.e. Management begins to recognize need).

The second element assesses how a practice is deployed in the organisation, i.e. the breadth and consistency of practice implementation across project areas. Table 5 gives an example of how a participant might respond. The RE practice is as follows.

Table 5 Deployment

1.2 DR1: Define standard templates for describing requirements

The respondent needs to tick one of the options in the ‘Score’ column. Using their expert understanding and by collecting relevant information from different sources, imagine the respondent selects Fair (4) (i.e. Less fragmented use).

The last element assesses the breadth and consistency of positive results over time and across project areas (using that particular practice). Table 6 gives an example of how a participant might respond. The RE practice is as follows.

Table 6 Results

1.3 DR1: Define standard templates for describing requirements

The respondent should tick one of the options in the ‘Score’ column. Using his understanding and by collecting relevant information from different sources, imagine the respondent selects Marginally qualified (6) (i.e. Positive measurable results in most parts of the organisation).

Now the score of three elements is: 2 + 4 + 6/3 = 4. So we can say that the DR1 practice is not strong (i.e. <7) and can be considered at FAIR.

The above three elements are performed for all the RE practices in any particular requirements category. This procedure is repeated for each practice. The score for each practice is summed and an average is used to gain an overall score for each ‘requirements process category’.

Appendix B: Assessment summaries of all RE categories

ID	Type	Practice	Organisation A	Organisation B
The 3-dimensional scores of requirements documents practices
RD1	Basic	Define a standard document structure	5	8
RD2	Basic	Explain how to use the document	5	8
RD3	Basic	Include a summary of the requirements	0	9
RD4	Basic	Make a business case for the system	3	9
RD5	Basic	Define specialized terms	5	7
RD6	Basic	Make document layout readable	5	9
RD7	Basic	Help readers find information	5	8
RD8	Basic	Make the document easy to change	3	6
The 3-dimensional overall score of requirements document category			3.8	8
The 3-dimensional scores of requirements elicitation practices
RE1	Basic	Assess system feasibility	7	6
RE2	Basic	Be sensitive to organisational and political consideration	7	7
RE3	Basic	Identify and consult system stakeholders	7	7
RE4	Basic	Record requirements sources	5	5
RE5	Basic	Define the system’s operating environment	6	8
RE6	Basic	Use business concerns to drive requirements elicitation	6	8
RE7	Intermediate	Look for domain constraints	6	8
RE8	Intermediate	Record requirements rationale	0	7
RE9	Intermediate	Collect requirements from multiple viewpoints	0	6
RE10	Intermediate	Prototype poorly understood requirements	0	7
RE11	Intermediate	Use scenarios to elicit requirements	0	8
RE12	Intermediate	Define operational processes	4	6
RE13	Advanced	Reuse requirements	0	8
The 3-dimensional overall score of requirements elicitation category			3.6	7
The 3-dimensional scores of requirements analysis and negotiation practices
RA1	Basic	Define system boundaries	1	8
RA2	Basic	Use checklists for requirements analysis	0	6
RA3	Basic	Provide software to support negotiations	0	6
RA4	Basic	Plan for conflicts and conflict resolution	0	8
RA5	Basic	Prioritise requirements	0	9
RA6	Intermediate	Classify requirements using a multi-dimensional approach	0	7
RA7	Intermediate	Use interaction matrices to find conflicts and overlaps	0	6
RE8	Advanced	Assess requirements risks	0	7
The 3-dimensional overall score of requirements negotiation category			0	7
The 3-dimensional scores of describing requirements practices
DR1	Basic	Define standard templates for describing requirements	1	9
DR2	Basic	Use languages simply and concisely	3	9
DR3	Basic	Use diagrams appropriately	5	7
DR4	Basic	Supplement natural language with other description of requirement	5	9
DR5	Intermediate	Specify requirements quantitatively	0	7
The 3-dimensional overall score of describing requirements category			2.6	8
The 3-dimensional scores: systems modelling practices
SM1	Basic	Develop complementary system models	6	6
SM2	Basic	Model the system’s environment	6	7
SM3	Basic	Model the system architecture	6	7
SM4	Intermediate	Use structured methods for system modelling	2	6
SM5	Intermediate	Use a data dictionary	0	8
SM6	Intermediate	Document the links between stakeholders requirement and system models	0	7
The 3-dimensional overall score of systems modelling category			3.3	6.8
The 3-dimensional scores: requirements validation practices
RV1	Basic	Check that the requirements document meets your standards	1	5
RV2	Basic	Organise formal requirements inspections	0	5
RV3	Basic	Use multi-disciplinary teams to review requirements	0	5
RV4	Basic	Define validation checklists	0	6
RV5	Intermediate	Use prototyping to animate requirements	0	5
RV6	Intermediate	Write a draft user manual	8	7
RV7	Intermediate	Propose requirements test cases	0	7
RV8	Advanced	Paraphrase system models	1	6
The 3-dimensional overall score of requirements validation category			1.2	5.7
The 3-dimensional scores: requirements management practices
RM1	Basic	Uniquely identify each requirement	0	8
RM2	Basic	Define policies for requirements management	0	7
RM3	Basic	Define traceability policies	0	6
RM4	Basic	Maintain a traceability manual	0	6
RM5	Intermediate	Use a database to manage requirements	0	7
RM6	Intermediate	Define change management policies	0	8
RM7	Intermediate	Identify global system requirements	0	7
RM8	Advanced	Identify volatile requirements	0	7
RM9	Advanced	Record rejected requirements	0	6
The 3-dimensional overall score of requirements management category			0	6.7