Requirements Engineering

, Volume 9, Issue 4, pp 248–260

A framework for empirical evaluation of conceptual modeling techniques

Original Article

DOI: 10.1007/s00766-004-0204-6

Cite this article as:
Gemino, A. & Wand, Y. Requirements Eng (2004) 9: 248. doi:10.1007/s00766-004-0204-6


The paper presents a framework for the empirical evaluation of conceptual modeling techniques used in requirements engineering. The framework is based on the notion that modeling techniques should be compared via their underlying grammars. The framework identifies two types of dimensions in empirical comparisons—affecting and affected dimensions. The affecting dimensions provide guidance for task definition, independent variables and controls, while the affected dimensions define the possible mediating variables and dependent variables. In particular, the framework addresses the dependence between the modeling task—model creation and model interpretation—and the performance measures of the modeling grammar. The utility of the framework is demonstrated by using it to categorize existing work on evaluating modeling techniques. The paper also discusses theoretical foundations that can guide hypothesis generation and measurement of variables. Finally, the paper addresses possible levels for categorical variables and ways to measure interval variables, especially the grammar performance measures.


System analysis and design Information systems development Conceptual modeling Empirical comparison Requirements engineering 

Copyright information

© Springer-Verlag London Limited 2004

Authors and Affiliations

  1. 1.Faculty of Business AdministrationSimon Fraser UniversityBurnabyCanada
  2. 2.Sauder School of BusinessUniversity of British ColumbiaVancouverCanada

Personalised recommendations