Skip to main content
SpringerLink
Log in

ReFlO: an interactive tool for pipe-and-filter domain specification and program generation

  • Regular Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

ReFlO is a framework and interactive tool to record and systematize domain knowledge used by experts to derive complex pipe-and-filter (PnF) applications. Domain knowledge is encoded as transformations that alter PnF graphs by refinement (adding more details), flattening (removing modular boundaries), and optimization (substituting inefficient PnF graphs with more efficient ones). All three kinds of transformations arise in reverse-engineering legacy PnF applications. We present the conceptual foundation and tool capabilities of ReFlO, illustrate how parallel PnF applications are designed and generated, and how domain-specific libraries of transformations are developed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Notes

  1. Readers may notice that algorithms and do not necessarily produce the same result. However, both implement the semantics specified by , and the result of is one of the possible results of , i.e. removes non-determinism.

    Fig. 4
    figure 4

    Two implementations of the interface

    Full size image

  2. Although called optimizations, they do not necessarily improve performance, but combinations of them typically do.

  3. We refer to the interfaces (boxes) contained inside an algorithm as internal interfaces (boxes), and to the algorithm as the parent box of those interfaces.

  4. At design time, the variable only allow us to determine whether to elements are replicated the same number of times. These variables can be instantiated when generating code.

  5. In Sect.  we provide additional details about how ReFlO verify these constraints.

  6. Replication parameters of an interface are used to set the replication parameter(s) of an implementation. If an implementation has replication parameters that are not present in the interface, the user is asked to provide a value for the parameter.

  7. The values of replication parameters of the pattern are used to define the replication parameters of the interface. The same is done to define the values of the additional parameters of the new interface.

  8. This map is similar to except that values are of type instead of .

  9. ReFlO ignores the specification of explicit post-conditions for algorithms or patterns. This prevents post-conditions from being specified that are stronger than those computed from its internal boxes.

  10. The RDM used in this derivation is available at http://cs.utexas.edu/users/schwartz/DxT/case-studies/gamma/models/databases.html.

  11. For simplicity, the derivation presented does not use replication. A derivation using replication is available at http://cs.utexas.edu/users/schwartz/DxT/case-studies/gamma/architectures/cascadejoin-rep/.

  12. There are many ways in which and can be realized. The simplest is this: is a bitmap. The join key of an tuple is hashed twice: once to determine the row of , the second to determine the column within the selected row. Thus, all tuples of substream hash to row of . combines into by boolean disjunction. For each , extracts row from and zeros out the rest of .

  13. An interface cost is set to that of its most general primitive implementation.

  14. We prevent algorithm from being chosen using preconditions.

References

  1. Garlan, D., Shaw, M.: An Introduction to Software Architecture. Carnegie Mellon University, Technical Report. CMU-CS-94-166 (1994)

  2. Crnkovic, I.: Building Reliable Component-Based Software Systems. M. Larsson, Ed. Artech House Inc., Norwood, USA (2002)

  3. The LabVIEW Environment, http://www.ni.com/labview/

  4. Gorlick, M.M., Razouk, R.R.: Using weaves for software construction and analysis. In: ICSE ’91: Proceedings of the 13th International Conference on Software Engineering, pp. 23–34 (1991)

  5. Simulink–Simulation and Model-Based Design, http://www.mathworks.com/products/simulink/

  6. Bruneton, E., Coupaye, T., Stefani, J.: The Fractal Component Model, http://fractal.ow2.org (2004)

  7. Frankel, D.S.: Model Driven Architecture Applying MDA to Enterprise Computing. Wiley, New York (2003)

    Google Scholar 

  8. Green, C., Luckham, D., Balzer, R., Cheatham, T., Rich, C.: Report on a Knowledge-Based Software Assistant. Kestrel Institute, Technical Report (1983)

  9. Marker, B., Poulson, J., Batory, D., van de Geijn, R.: Designing linear algebra algorithms by transformation: mechanizing the expert developer. In: iWAPT ’12: International Workshop on Automatic Performance Tuning (2012)

  10. Das, D.: Making Database Optimizers More Extensible. Ph.D. dissertation. The University of Texas at Austin (1995)

  11. Wirth, N.: Program development by stepwise refinement. Commun. ACM 14(4), 221–227 (1971)

    Article  MATH  Google Scholar 

  12. Liskov, B.H., Wing, J.M.: A behavioral notion of subtyping. ACM Trans. Program. Lang. Syst. 16(6), 1811–1841 (1994)

    Article  Google Scholar 

  13. Medvidovic, N., Rosenblum, D.S., Taylor, R.N.: A language and environment for architecture-based software development and evolution. In: ICSE ’99: Proceedings of the 21st International Conference on Software engineering, pp. 44–53 (1999)

  14. Wikipedia, “Component-based software engineering”, http://en.wikipedia.org/wiki/Component-based_software_engineering (2013)

  15. Batory, D., O’Malley, S.W.: The design and implementation of hierarchical software systems with reusable components. ACM Trans. Softw. Eng. Methodol. (TOSEM) 1(4), 355–398 (1992)

    Article  Google Scholar 

  16. Perry, D.E.: Version control in the inscape environment. In: ICSE ’87: Proceedings of the 9th International Conference on Software Engineering, pp. 142–149 (1987)

  17. Paige, R.F., Kolovos, D.S., Polack, F.A.C.: Refinement via consistency checking in MDA. Electron. Notes Theor. Comput. Sci. 137(2), 151–161 (2005)

    Article  MATH  Google Scholar 

  18. Wimmer, M., Kappel, G., Kusel, A., Retschitzegger, W., Schönböck, J., Schwinger, W., Kolovos, D., Paige, R., Lauder, M., Schürr, A., Wagelaar, D.: Surveying rule inheritance in model to model transformation languages. J. Object Technol. 11(2), 3:1–3:46 (2012)

    Google Scholar 

  19. Liskov, B., Wing, J.M.: A new definition of the subtype relation. In: ECOOP ’93: Proceedings of the 7th European Conference on Object-Oriented Programming, pp. 118–141 (1993)

  20. Riché, T.L., Gonçalves, R.C., Marker, B., Batory, D.: Pushouts in software architecture design. In: GPCE ’12: Proceedings of the 11th ACM International Conference on Generative Programming and Component Engineering, pp. 84–92 (2012)

  21. Gonçalves, R.C.: Parallel Programming by Transformation. Ph.D. dissertation, Departamento de Informática, Universidade do Minho (to appear)

  22. Tisi, M., Jouault, F., Fraternali, P., Ceri, S., Bézivin, J.: On the use of higher-order model transformations. In: ECMDA-FA ’09: Proceedings of the 5th European Conference on Model Driven Architecture-Foundations and Applications, pp. 18–33 (2009)

  23. Marker, B., Batory, D., Shepherd, C.: DxTer: A Program Synthesizer for Dense Linear Algebra. The University of Texas at Austin, Department of Computer Science, Technical Report (2012)

  24. Marker, B., Terrel, A., Poulson, J., Batory, D., van de Geijn, R.: Mechanizing the expert dense linear algebra developer. In: PPoPP ’12: Proceedings of the 17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 289–290 (2012)

  25. Batory, D., Marker, B.: Correctness Proofs of the Gamma Database Machine Architecture. University of Texas Department of Computer Science TR-11-17 (2011)

  26. DeWitt, D.J., Ghandeharizadeh, S., Schneider, D.A., Bricker, A., Hsiao, H.I., Rasmussen, R.: The gamma database machine project. IEEE Trans. Knowl. Data Eng. 2(1), 44–62 (1990)

    Article  Google Scholar 

  27. Baru, C.K., Fecteau, G., Goyal, A., Hsiao, H., Jhingran, A., Padmanabhan, S., Copeland, G.P., Wilson, W.G.: DB2 parallel edition. IBM Sys. J. 34(2) , 292–322 (1995)

    Google Scholar 

  28. Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Commun. ACM 13(7), 422–426 (1970)

    Article  MATH  Google Scholar 

  29. Neighbors, J.M.: Software Construction Using Components. Ph.D. dissertation, Department of Information and Computer Science, University of California, Irvine (1980)

  30. Thies, W., Karczmarek, M., Amarasinghe, S.P.: StreamIt: a language for streaming applications. In: CC ’02: Proceedings of the 11th International Conference on Compiler Construction, pp. 179–196 (2002)

  31. Dataflow application areas, http://www.ni.com/labview/applications/ (2013)

  32. TreeMap (Java Platform SE 7), http://docs.oracle.com/javase/7/docs/api/java/util/TreeMap.html (2013)

  33. ATL—a model transformation technology, http://www.eclipse.org/atl/

  34. Epsilon, http://www.eclipse.org/gmt/epsilon/

  35. Varró, D.: Model transformation by example. In: MODELS ’06: Proceedings of the 11th International Conference on Model Driven Engineering Languages and Systems, pp. 410–424 (2006)

  36. Baar, T., Whittle, J.: On the usage of concrete syntax in model transformation rules. In: PSI ’06: Proceedings of the 6th International Andrei Ershov Memorial Conference on Perspectives of Systems Informatics, pp. 84–97 (2006)

  37. Wimmer, M., Strommer, M., Kargl, H., Kramler, G.: Towards model transformation generation by-example. In: HICSS ’07: Proceedings of the 40th Annual Hawaii International Conference on System Sciences (2007)

  38. Sun, Y., White, J., Gray, J.: Model transformation by demonstration. In: MODELS ’09: Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems, pp. 712–726 (2009)

  39. Saada, H., Dolquesa, X., Huchard, M., Nebut, C., Sahraoui, H.: Generation of operational transformation rules from examples of model transformations. In: MODELS ’12: Proceedings of the 15th International Conference on Model Driven Engineering Languages and Systems, pp. 546–561 (2012)

  40. Varró D., Balogh, Z.: Automating model transformation by example using inductive logic programming. In: SAC ’07: Proceedings of the 2007 ACM Symposium on Applied Computing, pp. 978–984 (2007)

  41. Egyed, A., Mehta, N.R., Medvidovic, N.: Software connectors and refinement in family architectures. In: IW-SAPF-3: Proceedings of the International Workshop on Software Architectures for Product Families, pp. 96–106 (2000)

  42. Heckel, R., Thöne, S.: Behavior-preserving refinement relations between dynamic software architectures. In: WADT’ 04: Proceedings of the 17th International Workshop on Algebraic Development, Techniques, pp. 1–27 (2004)

  43. Mens, T., VanGorp, P.: A taxonomy of model transformation. Electron. Notes Theor. Comput. Sci. 152, 125–142 (2006)

    Article  Google Scholar 

  44. Sun, Y., Gray, J., White, J.: MT-scribe: an end-user approach to automate software model evolution. In: ICSE ’11: Proceedings of the 33rd International Conference on Software Engineering, pp. 980–982 (2011)

  45. Rozenberg, G.: Handbook of Graph Grammars and Computing by Graph Transformation Vol I Foundations. World Scientific, Singapore (1997)

  46. Habel, A.: Hyperedge Replacement: Grammars and Languages. Springer, New York (1992)

    MATH  Google Scholar 

  47. Taentzer, G.: AGG: a graph transformation environment for modeling and validation of software. In: Applications of Graph Transformations with Industrial Relevance, vol. 3062, pp. 446–453. Springer, Berlin (2004)

  48. Lohman, G.M.: Grammar-like functional rules for representing query optimization alternatives. In: SIGMOD ’88: Proceedings of the 1988 ACM SIGMOD International Conference on Management of Data, pp. 18–27 (1988)

  49. Selinger, P.G., Astrahan, M.M., Chamberlin, D.D., Lorie, R.A., Price, T.G.: Access path selection in a relational database management system. In: SIGMOD ’79: Proceedings of the 1979 ACM SIGMOD International Conference on Management of Data, pp. 23–34 (1979)

  50. Diaconescu, R., Futatsugi, K., Iida, S.: Component-based algebraic specification and verification in CafeOBJ. In: FM ’99: Proceedings of the Wold Congress on Formal Methods in the Development of Computing Systems-Volume II, pp. 1644–1663 (1999)

  51. Clavel, M., Durán, F., Eker, S., Lincoln, P., Martí-Oliet, N., Meseguer, J., Quesada, J.F.: Maude: specification and programming in rewriting logic. Theor. Comput. Sci. 285(2), 187–243 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  52. Püschel, M., Moura, J.M.F., Singer, B., Xiong, J., Johnson, J., Padua, D., Veloso, M., Johnson, R.W.: Spiral: a generator for platform-adapted libraries of signal processing algorithms. Int. J. High Perform. Comput. Appl. 18(1), 21–45 (2004)

    Article  Google Scholar 

  53. Lowry, M.R., Philpot, A., Pressburger, T., Underwood, I.: Amphion: automatic programming for scientific subroutine libraries. In: ISMIS ’94: Proceedings of the 8th International Symposium on Methodologies for Intelligent Systems, pp. 326–335 (1994)

  54. Thies, W.: Language and Compiler Support for Stream Programs. Ph.D. dissertation. MIT (2008)

  55. Eker, J., Janneck, J., Lee, E.A., Liu, J., Liu, X., Ludvig, J., Sachs, S., Xiong, Y., Neuendorffer, S.: Taming heterogeneity—the Ptolemy approach. Proc. IEEE 91(1), 127–144 (2003)

    Article  Google Scholar 

  56. Combemale, B., Crégut, X., Giacometti, J.-P., Michel, P., Pantel, M.: Introducing simulation and model animation in the MDE topcased toolkit. In: ERTS ’08: 4th European Congress Embedded Real Time Software (2008)

  57. Dotan, D., Kirshin, A.: Debugging and testing behavioral UML models. In: OOPSLA ’07: Companion to the 22nd ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications Companion, pp. 838–839 (2007)

  58. Ratzer, A.V., Wells, L., Lassen, H.M., Laursen, M., Qvortrup, J.F., Stissing, M.S., Westergaard, M., Christensen, S., Jensen, K.: CPN tools for editing, simulating, and analysing coloured petri nets. In: ICATPN ’03: Proceedings of the 24th International Conference on Applications and Theory of Petri Nets, pp. 450–462 (2003)

  59. Fritzsche, M., Johannes, J.: Putting performance engineering into model-driven engineering: model-driven performance engineering. In: Models in Software Engieering, pp. 164–175. Springer, Berlin (2008)

  60. Becker, S.: Coupled model transformations. In: WOSP ’08: Proceedings of the 7th International Workshop on Software and Performance, pp. 103–114 (2008)

  61. Koziolek, H.: Performance evaluation of component-based software systems: a survey. Perform. Eval. 67(8), 634–658 (2010)

    Article  Google Scholar 

  62. Grassi, V., Mirandola, R., Sabetta, A.: From design to analysis models: a kernel language for performance and reliability analysis of component-based systems. In: WOSP ’05: Proceedings of the 5th International Workshop on Software and Performance, pp. 25–36 (2005)

  63. Kounev, S.: Performance modeling and evaluation of distributed component-based systems using queueing petri nets. IEEE Trans. Softw. Eng. 32(7), 486–502 (2006)

    Article  Google Scholar 

  64. Becker, S., Koziolek, H., Reussner, R.: The palladio component model for model-driven performance prediction. J. Syst. Softw. 82(1), 3–22 (2009)

    Article  Google Scholar 

  65. Bunke, H.: Attributed programmed graph grammars and their application to schematic diagram interpretation. IEEE Trans. Pattern Anal. Mach. Intell. 4(6), 574–582 (1982)

    Article  MATH  Google Scholar 

  66. Cousot, P., Cousot, R.: Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: POPL ’77: Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages, pp. 238–252 (1977)

  67. Nielson, F., Nielson, H.R., Hankin, C.: Principles of Program Analysis. Springer, Berlin (1999)

    Book  MATH  Google Scholar 

  68. Guyer, S.Z., Lin, C.: Broadway: a compiler for exploiting the domain-specific semantics of software libraries. Proc. IEEE 93(2), 342–357 (2005)

    Article  Google Scholar 

  69. Perry, D.E.: The inscape environment. In: ICSE ’89: Proceedings of the 11th International Conference on Software Engineering, pp. 2–11. ACM (1989)

  70. Perry, D.E.: The logic of propagation in the inscape environment. ACM SIGSOFT Softw. Eng. Notes 14(8), 114–121 (1989)

    Article  MathSciNet  Google Scholar 

  71. Batory, D., Geraci, B.J.: Composition validation and subjectivity in GenVoca generators. IEEE Trans. Softw. Eng. 23(2), 67–82 (1997)

    Article  Google Scholar 

  72. Feiler, P., Li, J.: Consistency in dynamic reconfiguration. In: ICCDS ’98: Proceedings of the Fourth International Conference on Configurable Distributed Systems, pp. 189–196 (1998)

  73. Ráth, I., Varró, G., Varró, D.: Change-driven model transformations. In: MODELS ’09: Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems, pp. 342–356 (2009)

  74. Eclipse, http://eclipse.org

  75. Eclipse Modeling Framework, http://www.eclipse.org/modeling/emf/

  76. Graphical Editing Framework, http://www.eclipse.org/gef/

  77. Eclipse graphical modeling framework, http://www.eclipse.org/gmf/

Download references

Acknowledgments

We gratefully acknowledge helpful feedback from B. Marker (Texas), T. Riché (National Instruments), R. Silva (Minho), and the anonymous reviewers. This work was supported by NSF Grants CCF 0724979 and OCI-1148125. Rui Gonçalves and João Sobral are funded by ERDF—European Regional Development Fund through the COMPETE Programme (operational programme for competitiveness) and by National Funds through the FCT—Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within project FCOMP-01-0124-FEDER-010152. Rui Gonçalves is additionally funded by FCT grant SFRH/BD/47800/2008.

Author information

Authors and Affiliations

  1. Departamento de Informática, Universidade do Minho, Braga, Portugal

    Rui C. Gonçalves & João L. Sobral

  2. Department of Computer Science, The University of Texas at Austin, Austin, TX, USA

    Don Batory

Authors
  1. Rui C. Gonçalves
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Don Batory
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. João L. Sobral
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rui C. Gonçalves.

Additional information

Communicated by Prof. Marsha Chechik.

Appendix

Appendix

Figure 22 is not the last word on Gamma’s graph. Optimizations identical to those presented in Sect.  are used to optimize the processing of cascading joins, where the output of one join becomes the input of another (see Fig. 28).

Fig. 28
figure 28

graph

Full size image

Applying the refinements and , as described in Sect. 6.2, we get the graph depicted in Fig. 29a. This example further shows the importance of deriving the PnF graphs, instead of just using pre-built optimized implementations for the operations present in the initial PIM (in this case, operations). The use of the optimized implementations for would have resulted in an implementation equivalent to the one depicted in Fig. 29a. However, when we compose two (or more) instances of , new opportunities for optimization arise. We have again a serialization bottleneck, formed by a composition of boxes (that merges the output streams of the first group of s) and (that hash-splits the stream again).

Fig. 29
figure 29

Rotation of and

Full size image

Here again, refinement is insufficient to derive Gamma’s graph; encapsulation boundaries must be broken to eliminate serialization bottlenecks. Unlike the bottlenecks in the previous section, cascading joins use different keys to hash the tuples, so the partitioning of the stream before its merge is different than the partitioning after the hash-split. Therefore, we cannot use algorithm to optimize this subgraph.Footnote 14 Instead, we use a rewrite that removes these bottlenecks by swapping () pairs (algorithm ). Each input stream is hash-split into two substreams that are sent to the each box. The substreams with the same hash values are then merged.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gonçalves, R.C., Batory, D. & Sobral, J.L. ReFlO: an interactive tool for pipe-and-filter domain specification and program generation. Softw Syst Model 15, 377–395 (2016). https://doi.org/10.1007/s10270-014-0403-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-014-0403-7

Keywords

Search

Navigation