Assessing Collaborative Modeling Quality Based on Modeling Artifacts

  • Denis Ssebuggwawo
  • Stijn Hoppenbrouwers
  • Erik Proper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 68)


Collaborative modeling uses and produces modeling artifacts whose quality can help us gauge the effectiveness and efficiency of the modeling process. Such artifacts include the modeling language, the modeling procedure, the products and the support tool or medium. To effectively assess the quality of any collaborative modeling process, the (inter-) dependencies of these artifacts and their effect on modeling process quality need to be analyzed. Although a number of research studies have assessed and measured the quality of collaborative processes, no formal (causal) model has been developed to assess the quality of the collaborative modeling process through a combination of modeling artifacts. This paper develops a Collaborative Modeling Process Quality (CMPQ) construct for assessing the quality of collaborative modeling. A modeling session involving 107 students was used to validate and measure the quality constructs in the model.


Collaborative Modeling Modeling Process Quality Modeling Artifacts Instrument Validation Structural Equation Modeling 


  1. 1.
    Barjis, J., Kolfschoten, G.L., Verbraeck, A.: Collaborative Enterprise Modeling. In: Proper, E., Harmsen, F., Dietz, J.L.G. (eds.) PRET 2009, LNBIP, pp. 50–62. Springer, Heidelberg (2009)Google Scholar
  2. 2.
    Stirna, J., Persson, A.: Ten Years Plus with EKD: Reflections from Using an Enterprise Modeling Method in Practice. In: Proper, H.A., Halpin, T.A., Krogstie, J. (eds.) Proceedings of the EMMSAD 2007, held in conjunctiun with CAiSE 2007, pp. 97–106. Tapir Academic Press, Trondheim (2007)Google Scholar
  3. 3.
    Gjersvik, R., Krogstie, J., Følstad, A.: Participatory Development of Enterprise Process Models. In: Krogstie, J., Halpin, T., Siau, K. (eds.) Information Modeling Methods and Methodologies, pp. 195–215. IGI Global (2005)Google Scholar
  4. 4.
    Rittgen, P.: Collaborative Modeling of Business Processes-A Comparative Case Study. In: Proceedings of the 24th Annual ACM Symposium on Applied Computing, Waikiki Beach, Honolulu, Hawaii, pp. 225–230. ACM, New York (2009)Google Scholar
  5. 5.
    Vennix, J.A.M.: Group Model Building: Facilitating Team Learning Using System Dynamics. Wiley, Chichester (1996)Google Scholar
  6. 6.
    Vreede, G.J., de Briggs, R.O.: Collaboration Engineering: Designing Repeatable Processes for High-Value Collaborative Task. In: Proceedings of the 38th HICSS Conference, IEEE computer Society Press, Los Alamitos (2005)Google Scholar
  7. 7.
    Sedera, W., Rosemann, M., Doebeli, G.: A Process Modelling Success Model: Insights From A Case Study. In: 11th ECIS Conference, Naples, Italy, pp. 1–11 (2003)Google Scholar
  8. 8.
    Davison, R.: An Instrument for Measuring Meeting Success: Revalidation and Modification. Information and Management 36, 321–328 (1999)CrossRefGoogle Scholar
  9. 9.
    Briggs, R.O., Reinig, B.A., de Vreede, G.J.: Meeting Satisfaction for Technology Supported Groups: An Empirical Validation of a Goal-Attainment Model. Small Group Research 37 (2006)Google Scholar
  10. 10.
    Davis, F.D.: Perceived Usefulness, Perceived Ease of Use and User Acceptance of Information Technology. MIS Quarterly 13(3), 319–340 (1989)CrossRefGoogle Scholar
  11. 11.
    Ssebuggwawo, D., Stijn Hoppenbrouwers, S.J.B.A., Proper, H.A.: Evaluating Modeling Sessions Using the Analytic Hierarchy Process. In: Persson, A., Stirna, J. (eds.) PoEM 2009, LNBIP, vol. 39, pp. 69–83. Springer, Heidelberg (2009)Google Scholar
  12. 12.
    Moody, D., Sindre, G., Brasethvik, T., Sølvberg, A.: Evaluating the Quality of Process Models: Empirical Analysis of a Quality Framework. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds.) ER 2002. LNCS, vol. 2503, p. 380. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  13. 13.
    Nysetvold, A.G., Krogstie, J.: Assessing Business Process Modeling Languages Using a Generic Quality Framework. In: Halpin, T., Krogstie, J., Siau, K. (eds.) Proceedings of the EMMSAD 2005 held in conjuction with CAiSE 2005, Porto, Lisboa, pp. 545–556. Idea Group, USA (2005)Google Scholar
  14. 14.
    Locke, E.A., Latham, G.P.: A Theory of Goal Setting and Task Performance. Prentice Hall, Eaglewood Cliffs (1990)Google Scholar
  15. 15.
    Briggs, R.O., de Vreede, G.J., Reinig, B.A.: A Theory and Measurement of Meeting Satisfaction. In: Proccedings of the 26th HICCS Conference (HICCS 2003), p. 25c . IEEE Computer Society, Los Alamitos (2003)Google Scholar
  16. 16.
    Duivenvoorde, G.P.J., Kolfschoten, G.L., Briggs, R.O., de Vreede, G.J.: Towards an Instrument to Measure the Successfulness of Collaborative Effort from the Participant Perspective. In: Proceedings of the 42nd HICCS Conference (HICCS 2009), pp. 1–9. IEEE Computer Society, Los Alamitos (2009)Google Scholar
  17. 17.
    Paul, S., Seetharaman, P., Ramamurthy, K.: User Satisfaction with System, Decision Process and Outcome in GDSS-Based Meeting: An Experimental Investigation. In: Proceedings of the 37th HICSS Conference (HICSS 2004), pp. 37–46. IEEE Computer Society, Washington (2004)Google Scholar
  18. 18.
    Dean, D.L., Orwig, R.E., Vogel, D.R.: Facilitation Methods for Collaborative Modeling Tools. Group Decision and Negotiation 9, 109–127 (2000)CrossRefGoogle Scholar
  19. 19.
    Reinig, B.A: Towards Understanding of Satisfaction with the Process and Outcomes of Teamwork. Journal of MIS 19, 65–83 (2003)Google Scholar
  20. 20.
    Krogstie, J.: A Semiotic Approach to Quality in Requirements Specifications. In: Proceedings of the IFIP TC8 /WG8.1 Working Conference on Organizational Semiotics: Evolving a Science of Information Systems, Montreal, Canada, pp. 231–249. Kluwer B.V., The Netherlands (2001)Google Scholar
  21. 21.
    Dean, D.L., Orwig, R.E., Vogel, D.R.: Technological Support for Group Process Modeling. Journal of MIS 11(3), 43–64 (1994)Google Scholar
  22. 22.
    de Vreede, G.J.: Collaborative Business Engineering with Animated Electronic Meetings. Journal of MIS 14(3), 141–164 (1997)Google Scholar
  23. 23.
    Fjermestad, J., Hiltz, S.R.: A Decsriptive Evaluation of Group Support Systems and Case Field Studies. Journal of MIS 17, 115–159 (2001)Google Scholar
  24. 24.
    Fjermestad, J., Hiltz, S.R.: An Assessment of Group Support Systems. Experimental Research: Methodology and Results. Journal of MIS 15, 7–149 (1999)Google Scholar
  25. 25.
    Rittgen, P.: Collaborative Modelling Architecture (COMA), (accessed on: 09/08/2010)
  26. 26.
    den Hengst, M., Dean, D.L., Kolfschoten, G.L., Chakrapani, A.: Assessing the Quality of Collaborative Processes. In: Proceedings of the 39th Annual HICCS Conference (HICCS 2006), vol. 1, p. 16b . IEEE Computer Society, Los Alamitos (2006)Google Scholar
  27. 27.
    Green, S.G., Taber, T.D.: The Effect of Three Social Decision Schemes on Decision Group Processes. Organizational Behaviour and Human Performance 25(1), 97–106 (1980)CrossRefGoogle Scholar
  28. 28.
    Straub, D.W.: Validating Instruments in MIS Research. MIS Quarterly 13, 47–169 (1989)CrossRefGoogle Scholar
  29. 29.
    Straub, D., Boudreau, M.-C., Gefen, D.: Validation Guidelines for IS Positivist Research. Communication of the AIS 13, 380–427 (2004)Google Scholar
  30. 30.
    Lynn, M.: Determination and Quantification of Content Validity. Nursing Research 35(6), 382–385 (1986)CrossRefGoogle Scholar
  31. 31.
    Joliffe, I.T.: Principal Component Analysis, 2nd edn. Springer Series in Statistics. Springer, Heidelberg (2002)Google Scholar
  32. 32.
    Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics, 4th edn. Harper Collins, New York (2001)Google Scholar
  33. 33.
    Gefen, D., Straub, D., Boudreau, M.-C.: Structural Equation Modeling and Regression Guidelines for Research Practice. Communication of the AIS 4(7), 1–78 (2000)Google Scholar
  34. 34.
    Cortina, J.M.: What is Coefficient Alpha? An Examination of Theory and Application. Journal of Applied Psychology 78, 98–104 (1993)CrossRefGoogle Scholar
  35. 35.
    Nunnally, J.C.: Psychometric Theory 2nd edn, McGraw-Hill, New York (1978)Google Scholar
  36. 36.
    Hair, J.F., Tatham, R.L., Anderson, R.E., Black, W.: Multivariate Data Analysis, 5th edn. Prentice-Hall, Upper Saddle River (1998)Google Scholar
  37. 37.
    Brown, T.A.: Confirmatory Factor Analysis for Applied Research. Guilford Press, New York (2006)Google Scholar
  38. 38.
    Albright, J.J., Hun, M.P.: Confirmatory Factor Analysis Using Amos, LISREL, Mplus, and SAS/STAT CALIS. Technical Working Paper. The University Information Technology Services (UITS) Center for Statistical and Mathematical Computing, Indiana University, (accessed on: 17/08/2010)
  39. 39.
    Raykov, T., Marcoulides, G.A.: A First Course in Structural Equation Modeling, 2nd edn. Psychology Group Taylor & Francis Group, New York (2006)Google Scholar
  40. 40.
    Arbuckle, J.L.: AMOS 18: User’s Guide. SPSS Inc., Chicago (2009)Google Scholar
  41. 41.
    Tojib, D.R., Sugiant, L., Sendjay, F.: User Satisfaction with Business-to-Employee Portals: Conceptualization and Scale Development. EJIS 17, 649–667 (2008)CrossRefGoogle Scholar

Copyright information

© IFIP International Federation for Information Processing 2010

Authors and Affiliations

  • Denis Ssebuggwawo
    • 1
  • Stijn Hoppenbrouwers
    • 1
  • Erik Proper
    • 1
    • 2
  1. 1.Institute of Computing and Information SciencesRadboud University NijmegenNijmegenThe Netherlands, EU
  2. 2.Public Research CentreHenri TudorLuxembourg, EU

Personalised recommendations