Original Article

Requirements Engineering

, Volume 9, Issue 4, pp 248-260

A framework for empirical evaluation of conceptual modeling techniques

  • Andrew GeminoAffiliated withFaculty of Business Administration, Simon Fraser University Email author 
  • , Yair WandAffiliated withSauder School of Business, University of British Columbia

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access


The paper presents a framework for the empirical evaluation of conceptual modeling techniques used in requirements engineering. The framework is based on the notion that modeling techniques should be compared via their underlying grammars. The framework identifies two types of dimensions in empirical comparisons—affecting and affected dimensions. The affecting dimensions provide guidance for task definition, independent variables and controls, while the affected dimensions define the possible mediating variables and dependent variables. In particular, the framework addresses the dependence between the modeling task—model creation and model interpretation—and the performance measures of the modeling grammar. The utility of the framework is demonstrated by using it to categorize existing work on evaluating modeling techniques. The paper also discusses theoretical foundations that can guide hypothesis generation and measurement of variables. Finally, the paper addresses possible levels for categorical variables and ways to measure interval variables, especially the grammar performance measures.


System analysis and design Information systems development Conceptual modeling Empirical comparison Requirements engineering