# An extensible approach to implicit incremental model analyses

## Abstract

As systems evolve, analysis results based on models of the system must be updated, in many cases as fast as possible. Since usually only small parts of the model change, large parts of the analysis’ intermediate results could be reused in an incremental fashion. Manually invalidating these intermediate results at the right places in the analysis is a non-trivial and error-prone task that conceals the codes intention. A possible solution for this problem is implicit incrementality, i.e., an incremental algorithm is derived from the batch specification, aiming for an increased performance without the cost of degraded maintainability. Current approaches are either specialized to a subset of analyses or require explicit state management. In this paper, we propose an approach to implicit incremental model analysis capable of integrating custom dynamic algorithms. For this, we formalize incremental derivation using category theory, gaining type-safety and correctness properties. We implement an extensible implicit incremental computation system and validate its applicability by integrating incremental queries. We evaluate the performance using a micro-benchmark and a community benchmark where the integration of explicit query incrementalization was multiple orders of magnitude faster than rerunning the analysis after every change.

## Keywords

Incremental computation Model-driven Monads## Notes

### Acknowledgements

We would like to thank the anonymous reviewers of the Software & Systems Modeling journal that have helped us to shape the paper into its current form with their thoughtful reviews. This work was partially supported by the MWK (Ministry of Science, Research and the Arts Baden-Württemberg) in the funding line Research Seed Capital (RiSC).

## Supplementary material

## References

- 1.Choi, K., Hwang, S.Y., Blank, T.: Incremental-in-time algorithm for digital simulation. In: Proceedings of the 25th ACM/IEEE Design Automation Conference, pp. 501–505. IEEE Computer Society Press (1988)Google Scholar
- 2.Salz, A., Horowitz, M.: IRSIM: an incremental MOS switch-level simulator. In: 26th Conference on Design Automation, 1989, pp. 173–178. IEEE (1989)Google Scholar
- 3.De Lemos, R., Giese, H., Müller, H.A., Shaw, M., Andersson, J., Litoiu, M., Schmerl, B., Tamura, G., Villegas, N.M., Vogel, T., et al.: Software engineering for self-adaptive systems: a second research roadmap. In: Software Engineering for Self-Adaptive Systems II, pp. 1–32. Springer (2013)Google Scholar
- 4.Szvetits, M., Zdun, U.: Systematic literature review of the objectives, techniques, kinds, and architectures of models at runtime. Softw. Syst. Model.
**15**, 1–39 (2013)Google Scholar - 5.Gossman, J.: Introduction to Model/View/ViewModel pattern for building WPF apps (2005). http://blogs.msdn.com/b/johngossman/archive/2005/10/08/478683.aspx
- 6.Smith, J.: PATTERNS-WPF apps with the model-view-viewmodel design pattern. MSDN Mag.
**24**(2), 72 (2009)Google Scholar - 7.Ben-Menachem, M., Marliss, G.S.: Software Quality: Producing Practical, Consistent Software. International Thomson Computer Press, New York (1997)Google Scholar
- 8.Sutherland, J.: Business objects in corporate information systems. ACM Comput. Surv. CSUR
**27**(2), 274–276 (1995)CrossRefGoogle Scholar - 9.Chen, Y., Dunfield, J., Hammer, M.A., Acar, U.A.: Implicit self-adjusting computation for purely functional programs. J. Funct. Program.
**24**(01), 56–112 (2014)MathSciNetCrossRefzbMATHGoogle Scholar - 10.Willis, D., Pearce, D.J., Noble, J.: Caching and incrementalisation in the java query language. ACM SIGPLAN Not.
**43**(10), 1–18 (2008)CrossRefGoogle Scholar - 11.Bergmann, G., Horváth, Á., Ráth, I.,Varró, D., Balogh, A., Balogh, Z., Ökrös, A.: Incremental evaluation of model queries over EMF models. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) Model Driven Engineering Languages and Systems, pp. 76–90. Springer (2010)Google Scholar
- 12.Bergmann, G., Ökrös, A., Ráth, I., Varró, D., Varró, G.: Incremental pattern matching in the VIATRA model transformation system. In: Proceedings of the Third International Workshop on Graph and Model Transformations, pp. 25–32. ACM (2008)Google Scholar
- 13.Giese, H., Wagner, R.: Incremental model synchronization with triple graph grammars. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) Model Driven Engineering Languages and Systems, pp. 543–557. Springer (2006)Google Scholar
- 14.Giese, H., Wagner, R.: From model transformation to incremental bidirectional model synchronization. Softw. Syst. Model.
**8**(1), 21–43 (2009)CrossRefGoogle Scholar - 15.Hammer, M.A., Phang, K.Y., Hicks, M., Foster, J.S.: Adapton: Composable, demand-driven incremental computation. ACM SIGPLAN Not.
**49**, 156–166 (2014)CrossRefGoogle Scholar - 16.Hammer, M.A., Dunfield, J., Headley, K., Labich, N., Foster, J.S., Hicks, M., Van Horn, D.: Incremental computation with names. In: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications, pp. 748–766. ACM (2015)Google Scholar
- 17.Hinkel, G.: NMF: A Modeling Framework for the .NET Platform, Karlsruhe Institute of Technology, Technical Report (2016)Google Scholar
- 18.Hinkel, G.: NMF: a multi-platform modeling framework. In: Proceedings of the Theory and Practice of Model Transformations: 11th International Conference, ICMT 2018, Held as Part of STAF 2018, Toulouse, France, June 25–29, 2018. Springer (2018)
**(accepted, to appear)**Google Scholar - 19.Szárnyas, G., Semeráth, O., Ráth, I., Varró, D.: The TTC 2015 train benchmark case for incremental model validation. In: Proceedings of the 8th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2015) Federation of Conferences, L’Aquila, Italy, July 24, 2015, pp. 129–141 (2015)Google Scholar
- 20.Staron, M.: Adopting model driven software development in industry—a case study at two companies. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) Model Driven Engineering Languages and Systems, pp. 57–72. Springer (2006)Google Scholar
- 21.Mohagheghi, P., Gilani, W., Stefanescu, A., Fernandez, M.A.: An empirical study of the state of the practice and acceptance of model-driven engineering in four industrial cases. Empir. Softw. Eng.
**18**(1), 89–116 (2013)CrossRefGoogle Scholar - 22.Meyerovich, L.A., Rabkin, A.S.: Empirical analysis of programming language adoption. In: Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, pp. 1–18. ACM (2013)Google Scholar
- 23.Carlsson, M.: Monads for incremental computing. SIGPLAN Not.
**37**(9), 26–35 (2002)CrossRefzbMATHGoogle Scholar - 24.Hinkel, G.: Change propagation in an internal model transformation language. In: Proceedings of the Theory and Practice of Model Transformations: 8th International Conference, ICMT 2015, Held as Part of STAF 2015, L’Aquila, Italy, July 20–21, 2015, pp. 3–17. Springer (2015)Google Scholar
- 25.Hinkel, G., Burger, E.: Change propagation and bidirectionality in internal transformation DSLs. Softw. Syst, Model (2017)CrossRefGoogle Scholar
- 26.Hinkel, G.: An NMF solution to the Smart Grid Case at the TTC 2017. In: Proceedings of the 10th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2017) Federation of Conferences, series CEUR Workshop Proceedings, CEUR-WS.org (2017)Google Scholar
- 27.Hinkel, G.: An NMF solution to the Families to Persons case at the TTC 2017. In: Proceedings of the 10th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2017) Federation of Conferences, series CEUR Workshop Proceedings, CEUR-WS.org (2017)Google Scholar
- 28.Wert, A., Happe, J., Happe, L.: Supporting swift reaction: Automatically uncovering performance problems by systematic experiments. In: Proceedings of the 2013 International Conference on Software Engineering, series ICSE ’13, pp. 552–561. IEEE Press (2013)Google Scholar
- 29.Tarjan, R.E.: A class of algorithms which require nonlinear time to maintain disjoint sets. J. Comput. Syst. Sci.
**18**(2), 110–127 (1979)MathSciNetCrossRefzbMATHGoogle Scholar - 30.Tarjan, R.E.: Efficiency of a good but not linear set union algorithm. J. ACM JACM
**22**(2), 215–225 (1975)MathSciNetCrossRefzbMATHGoogle Scholar - 31.Holm, J., De Lichtenberg, K., Thorup, M.: Poly-logarithmic deterministic fully-dynamic algorithms for connectivity, minimum spanning tree, 2-edge, and biconnectivity. J. ACM JACM
**48**(4), 723–760 (2001)MathSciNetCrossRefzbMATHGoogle Scholar - 32.Cattaneo, G., Faruolo, P., Petrillo, U.F., Italiano, G.: Maintaining dynamic minimum spanning trees: an experimental study. Discrete Appl. Math.
**158**(5), 404–425 (2010)MathSciNetCrossRefzbMATHGoogle Scholar - 33.Acar, U.A., Blelloch, G., Ley-Wild, R., Tangwongsan, K., Turkoglu, D.: Traceable data types for self-adjusting computation. ACM SIGPLAN Not.
**45**, 483–496 (2010)CrossRefGoogle Scholar - 34.Fraiteur, G.: User-friendly aspects with compile-time imperative semantics in .net: an overview of postsharp. In: Seventh International Conference on Aspect-Oriented Software Development (AOSD) (2008)Google Scholar
- 35.Fowler, M., Parsons, R.: Domain Specific Languages, 1st edn. Addison-Wesley, Reading (2010)Google Scholar
- 36.Acar, U.A.: Self-adjusting computation. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, USA (2005)Google Scholar
- 37.Cai, Y., Giarrusso, P.G., Rendel, T., Ostermann, K.: A theory of changes for higher-order languages. ACM SIGPLAN Not.
**49**, 145–155 (2014)CrossRefGoogle Scholar - 38.Grust, T.: Monad Comprehensions: A Versatile Representation for Queries. Springer, New York (2004)Google Scholar
- 39.Hinkel, G.: Implicit Incremental Model Analyses and Transformations. Ph.D. thesis, Karlsruhe Institute of Technology (2017)
**(to appear)**Google Scholar - 40.Hinkel, G.: An NMF solution to the TTC 2018 Social Media Case. In: Proceedings of the 11th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2018) Federation of Conferences, series CEUR Workshop Proceedings, CEUR-WS.org (2018)Google Scholar
- 41.Hinkel, G.: The TTC 2018 Social Media Case. In: Proceedings of the 11th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2018) Federation of Conferences, Series CEUR Workshop Proceedings, CEUR-WS.org (2018)Google Scholar
- 42.Hinkel, G., Happe, L.: An NMF solution to the TTC train benchmark case. In: Proceedings of the 8th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2015) Federation of Conferences, Series CEUR Workshop Proceedings, vol. 1524, CEUR-WS.org, pp. 142–146 (2015)Google Scholar
- 43.Szárnyas, G., Izsó, B., Ráth, I., Varró, D.: The train benchmark: cross-technology performance evaluation of continuous model queries. Softw. Syst. Model.
**17**(4), 1365–1393 (2017)CrossRefGoogle Scholar - 44.Horn, T.: Solving the TTC train benchmark case with funnyqt. In: Proceedings of the 8th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2015) Federation of Conferences, L’Aquila, Italy, July 24, 2015, pp. 147–151 (2015)Google Scholar
- 45.Wagelaar, D.: The ATL/EMFTVM solution to the train benchmark case. In: Proceedings of the 8th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2015) Federation of Conferences, L’Aquila, Italy, July 24, 2015, pp. 152–156 (2015)Google Scholar
- 46.Krikava, F.: Solving the ttc’15 train benchmark case study with SIGMA. In: Proceedings of the 8th Transformation Tool Contest, A Part of the Software Technologies: Applications and Foundations (STAF 2015) Federation of Conferences, L’Aquila, Italy, July 24, 2015, pp. 167–175 (2015)Google Scholar
- 47.Szárnyas, G., Izsó, B., Ráth, I., Harmath, D., Bergmann, G., Varró, D.: Incquery-d: a distributed incremental model query framework in the cloud. In: Proceedings of the Model-Driven Engineering Languages and Systems—17th International Conference, MODELS 2014, Valencia, Spain, September 28–October 3, 2014, pp. 653–669 (2014)Google Scholar
- 48.Pugh, W., Teitelbaum, T.: Incremental computation via function caching. In: Proceedings of the 16th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 315–328. ACM (1989)Google Scholar
- 49.Acar, U.A.: Self-adjusting computation: (an overview). In: Proceedings of the 2009 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation, pp. 1–6. ACM (2009)Google Scholar
- 50.Hammer, M.A., Acar, U.A., Chen, Y.: CEAL: a C-based Language for self-adjusting computation. ACM SIGPLAN Not.
**44**, 25–37 (2009)CrossRefGoogle Scholar - 51.Burckhardt, S., Leijen, D., Sadowski, C., Yi, J., Ball, T.: Two for the price of one: a model for parallel and incremental computation. ACM SIGPLAN Not.
**46**, 427–444 (2011)CrossRefGoogle Scholar - 52.Harkes, D., Groenewegen, D.M., Visser, E.: Icedust: Incremental and eventual computation of derived values in persistent object graphs. In: 30th European Conference on Object-Oriented Programming, ECOOP 2016, July 18–22, 2016, Rome, Italy, pp. 11:1–11:26 (2016)Google Scholar
- 53.Harkes, D., Visser, E.: Icedust 2: derived bidirectional relations and calculation strategy composition. In: 31st European Conference on Object-Oriented Programming, ECOOP 2017, June 19–23, 2017, Barcelona, Spain, pp. 14:1–14:29 (2017)Google Scholar
- 54.Bainomugisha, E., Carreton, A.L., Cutsem, T.V., Mostinckx, S., Meuter, W.D.: A survey on reactive programming. ACM Comput. Surv.
**45**(4), 52:1–52:34 (2013)CrossRefGoogle Scholar - 55.Cooper, G.H., Krishnamurthi, S.: Embedding dynamic dataflow in a call-by-value language. ESOP
**3924**, 294–308 (2006)zbMATHGoogle Scholar - 56.Meyerovich, L.A., Guha, A., Baskin, J., Cooper, G.H., Greenberg, M., Bromfield, A., Krishnamurthi, S.: Flapjax: a programming language for Ajax applications. ACM SIGPLAN Not.
**44**, 1–20 (2009)CrossRefGoogle Scholar - 57.Meijer, E.: Reactive extensions (RX): curing your asynchronous programming blues. In: ACM SIGPLAN Commercial Users of Functional Programming, Series CUFP ’10, ACM, p. 11:1 (2010)Google Scholar
- 58.Ramalingam, G., Reps, T.: A categorized bibliography on incremental computation. In: Proceedings of the 20th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 502–510. ACM (1993)Google Scholar
- 59.Reiss, S.P.: An approach to incremental compilation. In: Proceedings of the 1984 SIGPLAN Symposium on Compiler Construction, Series, SIGPLAN ’84, pp. 144–156. ACM (1984)Google Scholar
- 60.Bergmann, G., Dávid, I., Hegedüs, Á., Horváth, Á., Ráth, I., Ujhelyi, Z., Varró, D.: Viatra 3: a reactive model transformation platform. In: Theory and Practice of Model Transformations. Springer, pp. 101–110 (2015)Google Scholar
- 61.Reps, T.: Optimal-time incremental semantic analysis for syntax-directed editors. In: Proceedings of the 9th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Series, POPL ’82, pp. 169–176. ACM (1982)Google Scholar
- 62.Hoffman, K.: Continuous linq. Dr. Dobbs J.
**33**(2), 55–58 (2008)Google Scholar - 63.Blakeley, J.A., Larson, P.-A., Tompa, F.W.: Efficiently updating materialized views. SIGMOD Rec.
**15**(2), 61–71 (1986)CrossRefGoogle Scholar - 64.Gupta, A., Mumick, I.S., et al.: Maintenance of materialized views: problems, techniques, and applications. IEEE Data Eng. Bull.
**18**(2), 3–18 (1995)Google Scholar - 65.Kuno, H.A., Rundensteiner, E.A.: Using object-oriented principles to optimize update propagation to materialized views. In: Proceedings of the Twelfth International Conference on Data Engineering, 1996, pp. 310–317. IEEE (1996)Google Scholar
- 66.Giarrusso, P.G., Ostermann, K., Eichberg, M., Mitschke, R., Rendel, T., Kästner, C.: Reify your collection queries for modularity and speed! In: Proceedings of the 12th Annual International Conference on Aspect-Oriented Software Development, pp. 1–12. ACM (2013)Google Scholar
- 67.Forgy, C.L.: Rete: a fast algorithm for the many pattern/many object pattern match problem. Artif. Intell.
**19**(1), 17–37 (1982)CrossRefGoogle Scholar - 68.Ujhelyi, Z., Bergmann, G., Hegedús, Ábel, Horváth, Ákos, Izsó, B., Ráth, I., Szatmári, Z., Varró, D.: EMF-IncQuery: an integrated development environment for live model queries, Part 1. Sci. Comput. Program.
**98**, 80–99 (2015)Google Scholar - 69.Ráth, I., Hegedüs, Á., Varró, D.: Derived features for EMF by integrating advanced model queries. In: Vallecillo, A., Tolvanen, J.P., Kindler, E., Störrle, H., Kolovos, D. (eds.) Modelling foundations and applications, pp. 102–117. Springer (2012)Google Scholar
- 70.Bergmann, G.: Translating OCL to graph patterns. In: Dingel, J., Schulte, W., Ramos, I., Abrahão, S., Insfran, E. (eds.) Model-Driven Engineering Languages and Systems, pp. 670–686. Springer (2014)Google Scholar
- 71.Cabot, J., Teniente, E.: Incremental integrity checking of UML/OCL conceptual schemas. J. Syst. Softw.
**82**(9), 1459–1478 (2009)CrossRefGoogle Scholar - 72.Reder, A., Egyed, A.: Incremental consistency checking for complex design rules and larger model changes. In: France, R.B., Kazmeier, J., Breu, R., Atkinson, C. (eds.) Model Driven Engineering Languages and Systems, pp. 202–218. Springer (2012)Google Scholar
- 73.Uhl, A., Goldschmidt, T., Holzleitner, M.: Using an OCL impact analysis algorithm for view-based textual modelling. Electron. Commun. EASST
**44**, 1–20 (2011)Google Scholar - 74.King, V., Sagert, G.: A fully dynamic algorithm for maintaining the transitive closure. In: Proceedings of the Thirty-First Annual ACM Symposium on Theory of Computing, pp. 492–498. ACM (1999)Google Scholar
- 75.Lawvere, F.W., Rosebrugh, R.: Sets for Mathematics. Cambridge University Press, Cambridge (2003)CrossRefzbMATHGoogle Scholar
- 76.Crole, R.L.: Categories for Types. Cambridge University Press, Cambridge (1993)zbMATHGoogle Scholar
- 77.Stachowiak, H.: Allgemeine Modelltheorie. Springer, New York (1973)CrossRefGoogle Scholar
- 78.Hinkel, G.: Using structural decomposition and refinements for deep modeling of software architectures. Softw. Syst. Model. (2018). https://doi.org/10.1007/s10270-018-0701-6