Abstract
Managed Learning Environments (MLEs) in higher education institutions (HEIs) are relatively new to the arena of higher education, even though there are over 90% of institutions in the higher and further education sector who are currently engaged in some kind of MLE development activity (University of Brighton 2003). However, when it comes to the task of assessing the performance of an MLE there are no universally recognisable frameworks for evaluating MLEs in HEIs currently discussed within the literature. The paper advances a general systemic framework for evaluating MLEs based on Checkland’s SSM and reports on the first stages of our attempt to evaluate the MLE at Manchester Metropolitan University involving the team developing the system and the stakeholders concerned. Two of three iterations of this research have been completed and, whilst outside the scope of this paper, have found that SSM has coped with the criteria demanded of the evaluation framework within its context. After completing a stakeholder analysis, the criteria for evaluating an MLE, based on the stakeholders’ requirements, emerged. These iterations have tentatively concluded that by contextualising SSM to the evaluation requirements of an MLE in a UK HEI, the measures of performance suggested by SSM need to be adjusted. The final iteration will check this outcome.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alavi M (1994) Computer-mediated collaborative learning: an empirical evaluation. MIS Q 18(2):159–174
Bullock C, Ory J (2000) Evaluating instructional technology implementation in a higher education environment. Am J Eval 21(3):315–328
Buzan T, Buzan B (1993) The mind map book. BBC Publications, London
Checkland P (1981, 1999) Systems thinking, systems practice. Wiley, Chichester
Checkland P (2000) Soft systems methodology: a thirty year retrospective. Syst Res Behav Sci 17:11–58
Checkland P, Poulter J (2006) Learning for action: a short definitive account of soft systems methodology, and its use for practitioners, teachers and students. Wiley, Chichester
Checkland P, Scholes J (2003) Soft systems methodology in action. Wiley, Chichester
Chen W, Hirschheim R (2004) A paradigmatic and methodological examination of information systems research from 1991 to 2001. Inf Syst J 14:197–235
Davis J, Subrahmanian E, Konda S, Granger H, Collins M, Westerberg A (2001) Creating shared information spaces to support collaborative design work. Inf Syst Frontier 3(3):377–392
de la Teja I, Ganesan R, Lundgren-Cayrol K, Michael Spector J (2003) An introduction to issues in the evaluation of educational technology: international perspectives. Eval Progr Plan 26(2):163–168
Devedzic V (2003) Think ahead: evaluation and standardisation issues for e-learning applications. Int J Continuing Eng Lifelong Learn 13(5/6):556–566
Eden C (1992) On the nature of cognitive maps. J Manag Stud 29(3):261–265
Eden C, Ackermann F (2001a) SODA—the principals. In: Rosenhead J, Mongers J (eds) Rational analysis for a problematic world revisited. Wiley, New York, pp 21–41
Eden C, Ackermann F (2001b) SODA—journey making and mapping in practice. In: Rosenhead J, Mongers J (eds) Rational analysis for a problematic world revisited. Wiley, New York, pp 43–60
Eden C, Jones S, Sims D (1983) Messing about problems. Pergamon, Oxford
Farbey B, Land F, Targett D (1993) How to assess your IT investment: a study of methods and practice. Butterworth-Heinemann, Oxford
Guba E, Lincoln D (1989) Fourth generation evaluation. Sage Publications, Beverly Hills, CA
Holyfield S (2003) Developing a shared understanding of the (MLE)—the role of diagramming and requirements gathering. http://cetis.ac.uk/members/DiVLE/theme3packages/20030919104819/MLE-diagramming-ACF450.pdf. Accessed 20 Jan 2006
Jackson GA (1990) Evaluating learning technology: methods strategies and examples in higher education. J High Educ 61(3):294–311
Laurillard D (1977) Evaluation of student learning in CAL. Comput Educ 2:259–265
McNaught C, Lam P (2005) Building and evaluation culture and evidence base for e-learning in three Hong Kong universities. Br J Educ Technol 36(4):599–614
Meisalo V, Sutinen E, Torvinen S (2003) Choosing appropriate methods for evaluating and improving the learning process in distance programming courses. In: The proceedings of the 33rd ASEE/IEEE frontiers in education conference (FIE2003), pp T2B-11-16
Myers C, Bennett D, Brown G, Henderson T (2004) Emerging online learning environment and student learning: an analysis of faculty perceptions. Educ Technol Soc 7(1):78–86
Oliver M (2000) An introduction to the evaluation of learning technology. Educ Technol Soc 3(4):20–30
Patton M (1997) Utilization-focused evaluation. Sage Publications, Beverly Hills, CA
Quinsee S, Sumner N (2005) How to manage the big bang: evolution or revolution in the introduction of an MLE? Aslib Proc New Inf Perspect 57(2):146–156
Rose J (1997) Soft systems methodology as a social science research tool. Syst Res Behav Sci 14(4):249–258
Rose J, Haynes M (1999) A soft systems approach to evaluation for complex interventions in the public sector. J Appl Manag Stud 8(2):199–216
Serafeimidis V, Smithson S (2003) Information systems evaluation as an organisational institution—experience from a case study. Inf Syst J 13:251–274
Smithson S, Hirsccheim R (1998) Analysing information systems evaluation: another look at an old problem. Eur J Inf Syst 7:158–174
Strauss A, Corbin J (1990) Basics of qualitative research: grounded theory procedures and techniques. Sage Publications, Beverly Hills, CA
Stubbs M (2006) MLE project proposal v1d. Manchester Metropolitan University, Internal Documentation
Swan J (1997) Using cognitive mapping in management research: decisions about technical innovation. Br J Manag 8:183–198
Tricker T, Rangecroft M, Long P, Gilroy P (2001) Evaluating distance education courses: the student perception. Assess Eval High Educ 26(2):165–177
University of Brighton (2003) Managed learning environment activity in further and higher education in the UK. http://www.jisc.ac.uk/index.cfm?name=project_mle_activity. Accessed 1 Feb 2006
Voigt C, Swatman P (2004) Contextual e_learning evaluation: a preliminary framework. J Educ Media 29(3):175–187
Williams D (2002) Improving use of learning technologies in higher education through participant oriented evaluations. Educ Technol Soc 5(3):11–17
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hardman, J., Paucar-Caceres, A. A Soft Systems Methodology (SSM) Based Framework for Evaluating Managed Learning Environments. Syst Pract Action Res 24, 165–185 (2011). https://doi.org/10.1007/s11213-010-9182-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11213-010-9182-4