Evaluating the Impact of Systems Research

Part of the Translational Systems Sciences book series (TSS, volume 10)


A central challenge of systems research is expressing implicit understanding of change and making it explicit. The goal of this guide is to address, “What distinguishes systems research from other forms of research?” Defining what constitutes good systemic research requires explanation about what is missing from the current practices of research, as driven by the assumptions of science. This requires revisiting assumptions about what we know (ontology), how we learn (epistemology), and how those have shaped our approaches to research thus far. In the seven chapters of this guide, concepts of systems research—philosophy, frameworks, problem structuring and research design, taking action, reporting results, and competencies—have been presented in systematic ways that instill rigor in systemic inquiry. These concepts correspond to the precision expected of science viewed through systemic lenses. Each chapter, and the portion of the research study it represents, needs to be its own coherent “whole,” while also acting as part of a comprehensive study design. Good systems research puts science in context; its evaluation requires more than traditional scientific approaches and critical thinking. The need for systemic evaluation prompts several questions concerning the philosophical principles guiding research, the rationale for the chosen framework, the basis for problem analysis and research question development, and the resulting model. Research must be evaluated for systemic coherence as demonstrated in reporting findings, drawing conclusions, and making recommendations. Have the system and the systems researcher been changed by the inquiry? Essentially asking the question: What is systemic about the research and why does it matter?


Ontology Epistemology Systems research Systematic Systemic Systems model Rigor Coherence Context Critical thinking Credibility Evaluation Change 


  1. Ackoff, R. L. (1953). The design of social research. Chicago, IL: University of Chicago Press.Google Scholar
  2. Ackoff, R. L., & Emery, F. E. (1972). On purposeful systems: An interdisciplinary analysis of individual and social behavior as a system of purposeful events. London, England: Tavistock.Google Scholar
  3. Ackoff, R. L., Magidson, J,. & Addison, H. J. (2006). Idealized design: How to dissolve tomorrow’s crisis… today. Upper Saddle River, NJ: Prentice Hall.Google Scholar
  4. Agassi, J. (1968) The continuing revolution: A history of physics from the Greeks to Einstein. New York, NY: McGraw-Hill.Google Scholar
  5. Argyris, C., & Schön, D. A. (1978). Organizational learning: A theory of action perspective. Reading, MA: Addison-Wesley.Google Scholar
  6. Argyris, C., & Schön, D. A. (1989). Participatory action research and action science compared: A commentary. The American Behavioral Scientist, 32(5), 612–623. CrossRefGoogle Scholar
  7. Arzi-Gonczarowski, Z. (1999). Perceive this as that: Analogies, artificial perception, and category theory. Annals of Mathematics and Artificial Intelligence, 26(1–4), 215–252. doi: 10.1023/A:1018963029743 CrossRefGoogle Scholar
  8. Bahill, A. T., & Gissing, B. (1998). Re-evaluating systems engineering concepts using systems thinking. IEEE Transaction on Systems, Man and Cybernetics, Part C: Applications and Reviews, 28(4), 516–527. Retrieved from
  9. Boorstin, D. J. (1983). The discoverers: A history of Man’s search to know his world and himself. New York, NY: Random House.Google Scholar
  10. Bosch, O. J., Nguyen, N. C., Maeno, T., & Yasui, T. (2013). Managing complex issues through evolutionary learning laboratories. Systems Research and Behavioral Science, 30(2), 116–135. doi: 10.1002/sres.2171 CrossRefGoogle Scholar
  11. Burch, R. (2014). Charles Sanders Peirce. In E. N. Zalta (Ed.), The Stanford encyclopedia of philosophy (Winter ed.). Retrieved from
  12. Checkland, P. (1999). Systems thinking, systems practice: Includes a 30-year retrospective. Chichester, England: Wiley.Google Scholar
  13. Churchman, C. W. (1968). Challenge to reason. New York, NY: McGraw-Hill.Google Scholar
  14. Clagett, M. (1955). Greek science in antiquity. New York, NY: Abelard-Schumann.Google Scholar
  15. Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Thousand Oaks, CA: Sage.Google Scholar
  16. Dewey, J. (1910). How we think. Lexington, MA: D.C. Heath.CrossRefGoogle Scholar
  17. Dewey, J. (1929). The quest for certainty: A study of the relation of knowledge and action. New York, NY: Minton, Balch & Company.Google Scholar
  18. Edson, M. C., & Klein, L. (2016). Problem structuring and research design in systemic inquiry. In M. C. Edson, P. Buckle Henning, & S. Sankaran (Eds.), A guide to systems research: Philosophy, processes and practice (pp. 59–80). Springer.
  19. Freire, P. (1982). Creating alternative research methods: Learning to do it by doing it. In B. Hall, A. Gillette, & R. Tandon (Eds.), Creating knowledge: A monopoly? (pp. 29–37). New Delhi: Society for Participatory Research in Asia.Google Scholar
  20. Forrester, J. W. (1958). Industrial dynamics: A major breakthrough for decision makers. Harvard Business Review, 36(4), 37–66.Google Scholar
  21. Forrester, Jay W. (1961). Industrial dynamics. Waltham, MA: Pegasus Communications.Google Scholar
  22. Golinski, J. (2005). Making natural knowledge: Constructivism and the history of science. Chicago, IL: University of Chicago Press.CrossRefGoogle Scholar
  23. Gunderson, L. H., & Hollling, C. S. (2002). Panarchy: Understanding transformations in human and natural systems. Washington, DC: Island Press.Google Scholar
  24. Hall, E. T. (1959). The silent language. New York, NY: Doubleday.Google Scholar
  25. Hall, E. T. (1966). The hidden dimension. New York, NY: Doubleday.Google Scholar
  26. Hall, E. T. (1976). Beyond culture. New York, NY: Doubleday.Google Scholar
  27. Hall, M. R., & Hall, E. T. (1995). The fourth dimension in architecture: The impact of building on behavior: Eero Saarinen’s administrative center for Deere & Company. Moline, IL: Sunstone Press.Google Scholar
  28. Hammond, D. (2016). Philosophical foundations of systems research. In M. C. Edson, P. Buckle Henning, & S. Sankaran (Eds.), A guide to systems research: Philosophy, processes and practice (pp. 1–19). Springer.
  29. INCOSE. (n.d.). SIMILAR Process: State, investigate, model, integrate, launch, assess, re-evaluate. Retrieved from
  30. Kineman, J. J. (2012). R-Theory: A synthesis of Robert Rosen’s relational complexity. Systems Research and Behavioral Science, 29(5), 527–538. doi: 10.1002/sres.2156 CrossRefGoogle Scholar
  31. Kineman, J. J. (2016a). Modeling and simulation. In M. C. Edson, P. Buckle Henning, & S. Sankaran (Eds.), A guide to systems research: Philosophy, processes and practice (pp. 81–109). Springer.
  32. Kineman, J. J. (2016b). Systems research framework. In M. C. Edson, P. Buckle Henning, & S. Sankaran (Eds.), A guide to systems research: Philosophy, processes and practice (pp. 21–57). Springer.
  33. Koestler, A. (1967). The ghost in the machine. London, UK: Hutchinson.Google Scholar
  34. Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.Google Scholar
  35. Kuhn, W., & Frank, A. U. (1991). A formalization of metaphors and image-schemas in user interfaces. In D. M. Mark & A. U. Frank (Eds.), Cognitive and linguistic aspects of geographic space (pp. 419–434). Dordrecht, The Netherlands: Springer. doi: 10.1007/978-94-011-2606-9_24
  36. Lewin, K. (1947). Group decision and social change. In E. Maccoby, E. Newcomb, & E. Hartley (Eds.), Readings in social psychology (pp. 265–284). New York, NY: Holt.Google Scholar
  37. McIntyre, A. (2007). Participatory action research. Thousand Oaks, CA: Sage.Google Scholar
  38. Metcalf, G. S. (2016). Systems research model: A comparison of research methodologies as action research. Unpublished manuscript.Google Scholar
  39. Miller, J. G. (1978). Living systems. New York, NY: McGraw-Hill.Google Scholar
  40. O’Leary, Z. (2004). The essential guide to doing research. London, United Kingdom: Sage.Google Scholar
  41. O’Sullivan, F. (2016). London may soon have more bikes than cares at rush hour. The Atlantic: CitiLab. Retrieved from
  42. Parent, E. (1996). The living systems theory of James Grier Miller (Primer Project of the International Society for the Systems Sciences). Retrieved from
  43. Pearl, J. (2000). Causality: Models, reasoning, and inference. New York, NY: Cambridge University Press.Google Scholar
  44. Pearl, J., & Russell, S. (1998). Bayesian networks. Retrieved from Google Scholar
  45. Popper, K. R. (1972). Objective knowledge: An evolutionary approach. Oxford, United Kingdom: Oxford University Press.Google Scholar
  46. Rosen, R. (1985). Organisms as causal systems which are not mechanisms. In I. W. Richardson, A. H-Y. Louie, & R. Rosen, Theoretical biology and complexity: Three essays into the nature of complexity (pp. 165–203). Orlando, FL: Academic Press. doi: 10.1016/B978-0-12-597280-2.50008-8 CrossRefGoogle Scholar
  47. Rosen, R. (1991). Life itself: A comprehensive inquiry into the nature, origin, and fabrication of life. New York, NY: Columbia University Press.Google Scholar
  48. Sedley, D. (2003). Plato’s Cratylus. Cambridge, United Kingdom: Cambridge University Press.Google Scholar
  49. Sheldrake, R. (2005). New science of life. London, United Kingdom: Icon Books.Google Scholar
  50. Sheldrake, R. (2011). The presence of the past: Morphic resonance and the habits of nature. London, United Kingdom: Icon Books.Google Scholar
  51. Simms, J. R. (1999). Principles of quantitative living systems science. New York, NY: Springer and Kluwer Academic.Google Scholar
  52. Simon, H. A. (1969). The sciences of the artificial. Boston, MA: MIT Press.Google Scholar
  53. Smithsonian National Air and Space Museum. (n.d.). The Wright’s bicycle shop. Retrieved from
  54. Sullivan, L. (1896). The tall office building artistically reconsidered. Lippicott’s Magazine, 57, 403–409.Google Scholar
  55. Tarnas, R. (1991). The passion of the Western mind: Understanding the ideas that have shaped our world view. New York, NY: Random House.Google Scholar
  56. Ulrich, W. (1988). C. West Churchman–75 years [Editorial]. Systemic Practice and Action Research, 1(4), 341–350. doi: 10.1007/BF01066576
  57. Ulrich, W. (1999). An appreciation of C. West Churchman (with a bibliography from 1938 to 2001 and a postscript of 5 March 2006): Luminaries of the Systems Approach. Retrieved from the International Society for the Systems Sciences website:
  58. Ulrich, W. (2001). The quest for competence in systemic research and practice. Systems Research and Behavioral Science, 1(18), 3–28. doi: 10.1002/sres.366 CrossRefGoogle Scholar
  59. Varey, W. (2016). Systems research reporting. In M. C. Edson, P. Buckle Henning, & S. Sankaran (Eds.), A guide to systems research: Philosophy, processes and practice (pp. 143–175). Springer.
  60. von Bertalanffy, L. (1969). General system theory: Foundations, development, applications (Rev. ed.). New York, NY: George Braziller.Google Scholar
  61. Weissenbacher, G. (2012.) Explaining Heisenbugs. Vienna, Austria: Austrian National Research Network. Retrieved from Google Scholar
  62. Whitehead, A. N. (1967). Science and the modern world. New York, NY: Simon and Schuster. (Original work published in 1925)Google Scholar
  63. Wilber, K. (2007). The integral vision. Boston, MA: Shambhala.Google Scholar

Copyright information

© Springer Science+Business Media Singapore 2017

Authors and Affiliations

  1. 1.International Federation for Systems ResearchViennaAustria

Personalised recommendations