Evaluation Measures for Similarity Search Results in Process Model Repositories

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7532)


With the increasing uptake of business process management efforts in companies, similarity search in large process model repositories has gained significance, as it forms a cornerstone of effective process model management and reuse. Similarity search uses a process model as query and retrieves all models, which resemble the query, in a ranked order. So far, the quality of the ranking has not been investigated.

In this paper, we propose quality measures for similarity search results in order to address this problem, providing information on how good and how differentiated the results are. Our measures assess result statistics, which are derived from the similarity to the query model, and the agreement of different rankings, produced by diverse similarity measures. We apply our findings to a reference process model collection and comprehensively evaluate their prediction towards human assessment of process similarity.


Similarity search evaluation measures search result quality model repository business process management 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akkiraju, R., Ivan, A.: Discovering Business Process Similarities: An Empirical Study with SAP Best Practice Business Processes. In: Maglio, P.P., Weske, M., Yang, J., Fantinato, M. (eds.) ICSOC 2010. LNCS, vol. 6470, pp. 515–526. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  2. 2.
    Awad, A., Sakr, S., Kunze, M., Weske, M.: Design by Selection: A Reuse-Based Approach for Business Process Modeling. In: Jeusfeld, M., Delcambre, L., Ling, T.-W. (eds.) ER 2011. LNCS, vol. 6998, pp. 332–345. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Becker, M., Laue, R.: Analysing Differences between Business Process Similarity Measures. In: Daniel, F., Barkaoui, K., Dustdar, S. (eds.) BPM Workshops 2011, Part II. LNBIP, vol. 100, pp. 39–49. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Shaft, U., Ramakrishnan, R.: When Is Nearest Neighbors Indexable? In: Eiter, T., Libkin, L. (eds.) ICDT 2005. LNCS, vol. 3363, pp. 158–172. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Bunke, H., Allermann, G.: Inexact Graph Matching for Structural Pattern Recognition. Pattern Recognition Letters 1(4), 245–253 (1983)zbMATHCrossRefGoogle Scholar
  6. 6.
    Bunke, H.: A Graph Distance Metric Based on the Maximal Common Subgraph. Pattern Recognition Letters 19(3-4), 255–259 (1998)zbMATHCrossRefGoogle Scholar
  7. 7.
    Chávez, E., Navarro, G., Baeza-Yates, R., Marroquín, J.: Searching in Metric Spaces. ACM Comput. Surv. 33(3), 273–321 (2001)CrossRefGoogle Scholar
  8. 8.
    Conover, W.J.: Practical Non-Parametric Statistics, 2nd edn. John Wiley and Sons, New York (1980)Google Scholar
  9. 9.
    Croft, W.B., Metzler, D., Strohman, T.: Search Engines: Information Retrieval in Practice. Addison-Wesley (2010)Google Scholar
  10. 10.
    Curran, T., Keller, G., Ladd, A.: SAP R/3 Business Blueprint: Understanding the Business Process Reference Model. Prentice-Hall (1997)Google Scholar
  11. 11.
    Decker, G., Mendling, J.: Process Instantiation. Data Knowl. Eng. 68, 777–792 (2009)CrossRefGoogle Scholar
  12. 12.
    Dijkman, R., Dumas, M., García-Bañuelos, L.: Graph Matching Algorithms for Business Process Model Similarity Search. In: Dayal, U., Eder, J., Koehler, J., Reijers, H.A. (eds.) BPM 2009. LNCS, vol. 5701, pp. 48–63. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  13. 13.
    Dijkman, R., Dumas, M., van Dongen, B., Käärik, R., Mendling, J.: Similarity of Business Process Models: Metrics and Evaluation. Inf.Sys. 36(2), 498–516 (2011)CrossRefGoogle Scholar
  14. 14.
    Dumas, M., García-Bañuelos, L., Dijkman, R.: Similarity Search of Business Process Models. IEEE Data Eng. Bull. 32(3), 23–28 (2009)Google Scholar
  15. 15.
    Hjaltason, G.R., Samet, H.: Index-driven Similarity Search in Metric Spaces. ACM Trans. Database Syst. 28(4), 517–580 (2003)CrossRefGoogle Scholar
  16. 16.
    Kunze, M., Weidlich, M., Weske, M.: Behavioral Similarity – A Proper Metric. In: Rinderle-Ma, S., Toumani, F., Wolf, K. (eds.) BPM 2011. LNCS, vol. 6896, pp. 166–181. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  17. 17.
    Mendling, J.: Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for Correctness. Springer (2008)Google Scholar
  18. 18.
    Nüttgens, M., Rump, F.J.: Syntax und Semantik Ereignisgesteuerter Prozessketten (EPC). In: Promise, pp. 64–77 (2002)Google Scholar
  19. 19.
    Rosemann, M.: Potential Pitfalls of Process Modeling: Part B. Business Process Management Journal 12(3), 377–384 (2006)CrossRefGoogle Scholar
  20. 20.
    Vanderfeesten, I., Cardoso, J., Reijers, H., Van Der Aalst, W.: Quality Metrics for Business Process Models. In: BPM and Workflow Handbook, pp. 1–12 (2006)Google Scholar
  21. 21.
    Wang, R.Y., Strong, D.M.: Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems 12(4), 5–33 (1996)zbMATHGoogle Scholar
  22. 22.
    Weber, B., Reichert, M.: Refactoring Process Models in Large Process Repositories. In: Bellahsène, Z., Léonard, M. (eds.) CAiSE 2008. LNCS, vol. 5074, pp. 124–139. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  23. 23.
    Weidlich, M., Dijkman, R., Mendling, J.: The ICoP Framework: Identification of Correspondences between Process Models. In: Pernici, B. (ed.) CAiSE 2010. LNCS, vol. 6051, pp. 483–498. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity Search: The Metric Space Approach. Springer (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Hasso Plattner Institute at the University of PotsdamPotsdamGermany

Personalised recommendations