Identifying factors underlying the quality of online teaching effectiveness: An exploratory study

Abstract

TRADITIONALLY CAMPUS-BASED COURSES rely on student evaluations to provide instructors with feedback about their teaching effectiveness. However, current student evaluations of teaching instruments do not adequately assess many of the essential constructivist-based teaching practices recommended for quality online learning experiences. One of the best known summaries of research-based instructional practices is the widely disseminatedSeven Principles of Effective Teaching authored by Chickering and Gamson (1987). The majority of learner-centered instructional practices which comprise the Seven Principles framework are clearly focused on constructivist-based teaching practices. This study was an initial effort toward the development of a student evaluation of online teaching instrument based on the Seven Principles framework. Four hundred and eighty-nine students enrolled in WebCT courses at Montana State University completed the 26 item instrument. TheStudent Evaluation of Online Teaching Effectiveness (SEOTE) was found to be highly reliable and yielded four interpretable factors. The four factors were interpreted as Student-Faculty Interaction, Active Learning, Time on Task, and Cooperation Among Students.

This is a preview of subscription content, log in to check access.

References

  1. Abrami, P.C., & d’Apollonia, S. (1991). Multidimensional students’ evaluation of teaching effectiveness-Generalizability of N=1" research: Comment on Marsh (1991).Journal of Educational Psychology, 30, 221–227.

    Google Scholar 

  2. Abrami, P.C., d’Apollonia, S., & Rosenfield, S. (1997). The dimensionality of student ratings of instruction: What we know and what we do not. In R.P. Perry & J.C. Smart (Eds.),Effective Teaching In Higher Education: Research and Practice (pp. 321–367). New York: Agathon.

    Google Scholar 

  3. Aleamoni, L.M. (1978). Development and factorial validation of the Arizona Course/Instructor Evaluation Questionnaire.Educational and Psychological Measurement, 38, 1063–1067.

    Article  Google Scholar 

  4. Aleamoni, L.M. (1999). Student rating myths versus research facts from 1924 to 1998. Journal ofPersonal Evaluation in Education, 13(2), 153–166.

    Article  Google Scholar 

  5. American Psychological Association (1997, November).Learner-centered psychological principles: A framework for school design and reform. Retrieved April 21, 2005, from http://www.apa.org/ed/lcp.html#Background.

  6. Anastasi, A., & Urbina, S. (1997).Psychological testing (7th ed.). Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  7. Bandura, A. (1986).Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

    Google Scholar 

  8. Billings, D.M. (2000). A framework for assessing outcomes and practices in Webbased courses in nursing.Journal of Nursing Education, 39(2), 60–67.

    Google Scholar 

  9. Bonk, C.J., & Cunningham, D.J. (1998). Searching for learner-centered, constructivist, and sociocultural components of collaborative educational learning tools. In C.J. Bonk & K.S. King (Eds.),Electronic collaborators: Learner-centered technologies for literacy, apprenticeship, and discourse (pp. 25–50). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  10. Boyer, E.L. (1990).Scholarship reconsidered. Priorities of the professorate. Princeton, NJ: The Carnegie Foundation for the Advancement of Teaching.

    Google Scholar 

  11. Cashin, W.E., & Downey, R.G. (1992). Using global student ratings items for summative evaluation.Journal of Educational Psychology, 84, 563–572.

    Article  Google Scholar 

  12. Cattell, R.B. (1966). The scree test for the number of factors.Multivariate Behavioral Research, 1, 245–276.

    Article  Google Scholar 

  13. Centra, J.A. (1993).Reflective faculty evaluation. San Francisco: Jossey-Bass.

    Google Scholar 

  14. Chickering, A.W., & Erhmann, S.C. (1996). Implementing the seven principles: Technology as lever.AAHE Bulletin, 49(2), 3–6.

    Google Scholar 

  15. Chickering, A.W., & Gamson, Z.F. (March 1987). Seven principles for good practice in undergraduate education.AAHE Bulletin, 39(7), 3–7.

    Google Scholar 

  16. Cohen, P.A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multisection validity studies.Research in Higher Education, 13, 321–341.

    Article  Google Scholar 

  17. Cross, P.K. (1999). What do we know about students’ learning and how do we know it?Innovative Higher Education, 23(2), 255–270.

    Article  Google Scholar 

  18. Donald, J.G. (1999). Motivation for higher-order learning.New Directions for Teaching and Learning, 7, 27–35.

    Article  Google Scholar 

  19. Fabrigar, L.R., Wenger, D.T., MacCallum, R.C., & Strahan, E.J. (1999). Evaluating the use of exploratory factor analysis in psychological research.Psychological Methods, 4(3), 272–299.

    Article  Google Scholar 

  20. Feldman, K.A. (1988). Effective college teaching from the students’ and faculty’s view: Matched or mismatched priorities.Research in Higher Education, 28(4), 291–343.

    Article  Google Scholar 

  21. Feldman, K.A. (1997). Identifying exemplary teachers and teaching: Evidence from student ratings. In R.P. Perry & J.C. Smart (Eds.),Effective teaching in higher education: Research and practice (pp. 368–395). New York: Agathon Press.

    Google Scholar 

  22. Field, A. (2000).Discovering statistics using SPSS for windows. Thousand Oaks, CA: Sage.

    Google Scholar 

  23. Giguere, P., & Minotti, J (2003). Developing high quality Internet-based training for adult learners.Educational Technology, 4, 57–58.

    Google Scholar 

  24. Graham, C., Caglitay, K., Lim, B., Craner, J., &, Duffy, T.M. (2001). Seven principles for effective teaching: A practical lens for evaluating online courses.The Technology Source. Retrieved February 19, 2004, from http://ts.mivu.org/default.asp?show=article&id=839.

  25. Hacker, D.J., & Niederhauser, D.S. (2000). Promoting deep and durable learning in the online classroom.New Directions for Teaching and Learning, 84, 53–63.

    Article  Google Scholar 

  26. Jonassen, D.H. (2000).Computers as mindtools for schools. Upper Saddle River, NJ: Merrill Prentice Hall.

    Google Scholar 

  27. Jonassen, D.H., Peck, K.L., & Wilson, B.G. (1999).Learning with technology, Columbus, OH: Prentice Hall.

    Google Scholar 

  28. Kaiser, H.F. (1960). The application of electronic computers to factor analysis.Educational and Psychological Measurement, 20, 141–151.

    Article  Google Scholar 

  29. Koch, L.C., Holland, L.A., Price, D., Gonzalez, G.L., Lieske, P., Butler, A., Wilson, K., & Holly, M.L. (2002). Engaging new faculty in the scholarship of teaching.Innovative Higher Education, 27(2), 83–94.

    Article  Google Scholar 

  30. Lent, R.W., Brown, S.D., & Larkin, K.C. (1984). Relation of self-efficacy expectations to academic achievement and persistence.Journal of Counseling Psychology, 31, 356–362.

    Article  Google Scholar 

  31. Magnani, L, Nersessian, N.J., & Thagard, P. (1999).Model-based reasoning in scientific discovery. New York: Kluwer Acdemic/Plenum.

    Google Scholar 

  32. Marsh, H.W. (1982). SEEQ:A reliable, valid, and useful instrument for collecting students’ evaluations of university teaching.British Journal of Educational Psychology, 52, 77–95.

    Google Scholar 

  33. Marsh, H.W. (1987). Student evaluations of university teaching: Research findings, methodological issues, and directions for future research.International Journal of Educational Research, 11, 253–388.

    Article  Google Scholar 

  34. Marsh, H.W. (1991). A multidimensional perspective on students’ evaluations of teaching effectiveness: Reply to Abrami and d’Apollinia.Journal of Educational Psychology, 83(1), 416–421.

    Article  Google Scholar 

  35. Marsh, H.W. (1997). Making students’ evaluations of teaching effectiveness effective: The critical issues of validity, bias, and utility.American Psychologist, 52(11), 1187–97.

    Article  Google Scholar 

  36. Marsh, H.W., & Bailey, M. (1993). Multidimensional students’ evaluations of teaching effectiveness. A profile analysis.The Journal of Higher Education, 64(1), 1–18.

    Article  Google Scholar 

  37. Marsh, H.W., & Roche, L.A. (1993). The use of students’ evaluations and an individually structured intervention to enhance university teaching effectiveness. AmericanEducational Research Journal, 30, 217–251.

    Google Scholar 

  38. Millis, B.J., & Cottrell, P.G. (1998).Cooperative learning for higher education faculty. Phoenix, AZ: Oryx Press.

    Google Scholar 

  39. Moskal, P.D., & Dziuban, C.D. (2001). Present and future directions for assessing cyber education: The changing research paradigm. In L.R. Vandervert, L.V. Shavinina & R.A. Cornell (Eds.)Cybereducation: The future of long-Distance Learning. (pp. 157–184). Larchmont, NY: Mary Ann Liebert.

    Google Scholar 

  40. Pajares, F. (2002). Gender and perceived self-efficacy in self-regulated learning.Theory Into Practice, 41(2), 118–125.

    Article  Google Scholar 

  41. Partlow, K.M., & Gibbs, W.J. (2003). Indicators of constructivist principles in Internet-based courses.Journal of Computing in Higher Education, 14(2), 68–97.

    Article  Google Scholar 

  42. Pascarella, E.T., & Terenzini, P.T. (1991).How college affects students. San Francisco: Josey-Bass.

    Google Scholar 

  43. Phipps, R.A., & Merisotis, J.P. (2000).Quality on the line: Benchmarks for success in Internet-based distance education. Washington, DC: The Institute for Higher Education Policy.

    Google Scholar 

  44. Pintrich, P.R., & DeGroot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance.Journal of Educational Psychology, 82, 41–50.

    Article  Google Scholar 

  45. Reeves, T.C., & Reeves, P.M. (1997).Effective dimensions of interactive learning on the World Wide Web. In Bradual H. Kahn (Ed.),Internet-based instruction (pp. 59–66). Englewood Cliffs, NJ: Educational Technology Publications.

    Google Scholar 

  46. Relan, A., & Gillani, B.B. (1997). Internet-based instruction and the traditional classroom: Similarities and differences. In B.H. Kahn (Ed.),Internet-based instruction (pp. 41–46). Englewood Cliffs, NJ: Educational Technology Publications.

    Google Scholar 

  47. Schunk, D. (1983). Developing children’s self-efficacy and skills: The roles of social comparative information and goal setting.Contemporary Educational Psychology, 8, 76–86.

    Article  Google Scholar 

  48. Stevens, J.P. (2002).Applied multivariate statistics for the social sciences (4th ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  49. Svinicki, M.D. (1999). New directions in learning and motivation.New Directions for Teaching and Learning, 80, 5–27.

    Article  Google Scholar 

  50. Vye, N.J., Schwartz, D.L., Bransford, J.D., Barron, B.J., Zech, L., & Cognition and Technology Group at Vanderbilt (1998). SMART environments that support monitoring, reflection, and revision. In D.J. Hacker, J. Dunlosky, & A.C. Graesser, (Eds.),Metacognition in educational theory and practice (pp. 305–346). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  51. Watchtel, H.K. (1998). Student evaluation of college teaching effectiveness: A brief overview.Assessment and Evaluation in Higher Education, 23(2). 191–211.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Additional information

ABOUT THE AUTHORArthur W. Bangert is an Assistant Professor in the Adult and Higher Education Program at Montana State University (MSU) where he teachers courses in educational statistics, research methods, and educational assessment. Prior to his work at MSU, Dr. Bangert was a school psychologist, guidance counselor, and test consultant for a major publishing company. Dr. Bangert’s research interests include designing, teaching, and evaluating online learning environments and the use of Teacher Work Sample Methodology for training pre-service teachers in the design of quality classroom assessments.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Bangert, A.W. Identifying factors underlying the quality of online teaching effectiveness: An exploratory study. J. Comput. High. Educ. 17, 79–99 (2006). https://doi.org/10.1007/BF03032699

Download citation

Keywords

  • online teaching effectiveness
  • Internet-based statistics
  • student evaluations of teaching