Skip to main content
SpringerLink
Log in

Participation in online discussion environments: Is it really effective?

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

This study aimed to develop a rubric to assess participation of students in online discussion environments. For this purpose, the study included 168 students who participated in a course offered online during the spring semester of the 2015–2016 academic year. Developed based on the literature, the rubric consists of two parts (Form and Content, and Number and Density) and seven criteria in total. “Form and Content” consists of congruity of the message in terms of subject, clarity of the message, original value of the message, interactional value of the message, and directing the subject; whereas, “Number and Density” consists of the number of messages and their density. Four different discussion subjects were presented to the students in an online discussion environment. The researchers analyzed the students’ messages individually. Exploratory Factor Analysis was performed in order to obtain evidence for construct validity. After conducting the factor analysis, results showed that the first dimension, which is “form and content” sub-factor, is unidimensional. The “number and density” sub-factor was included in the graded scoring key based on the literature and expert opinion. The findings show that the graded scoring key is reliable and valid.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Total score of the student was calculated by averaging the scores obtained from all messages of that student.

References

  • Acar, M., & Anıl, D. (2009). Sınıf öğretmenlerinin performans değerlendirme sürecindeki değerlendirme yöntemlerini kullanabilme yeterlikleri, karşılaştıkları sorunlar ve çözüm önerileri. TÜBAV Bilim Dergisi, 2(3), 354–363.

    Google Scholar 

  • Ak, Ş. (2016). The role of technology-based scaffolding in problem-based online asynchronous discussion. British Journal of Educational Technology, 47(4), 680–693. https://doi.org/10.1111/bjet.12254

    Article  Google Scholar 

  • An, H., Shin, S., & Lim, K. (2009). The effects of different instructor facilitation approaches on students’ interactions during asynchronous online discussions. Computers & Education, 53(3), 749–760. https://doi.org/10.1016/j.compedu.2009.04.015

    Article  Google Scholar 

  • Andrade, G. H. (2001). The effects of instructional rubrics on learning to write. Current Issues in Education, 4(4), 1-21.

    Google Scholar 

  • Andresen, M. A. (2009). Asynchronous discussion forums: Success factors, outcomes, assessments, and limitations. Educational Technology & Society, 12(1), 249–257.

    MathSciNet  Google Scholar 

  • Arnold, N., & Ducate, L. (2006). Future foreign language teachers’ social and cognitive collaboration in an online environment. Language, Learning and Technology, 10(1), 42–66.

    Google Scholar 

  • Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of pedagogy. ASHE-ERIC Higher Education Report (Vol. 27, No.1). Washington, DC: The George Washington University, Graduate School of Education and Human Development.

  • Bullen, M. (1998). Participation and critical thinking in online university distance education. International Journal of E-Learning & Distance Education, 13(2), 1–32.

    Google Scholar 

  • Büyüköztürk, Ş. (2006). Sosyal Bilimler İçin Veri Analizi El Kitabi. İstatistik, Araştırma Deseni SPSS Uygulamaları ve Yorum (6th ed.). Ankara: Pegem A Yayıncılık.

    Google Scholar 

  • Cheng, C. K., Paré, D. E., Collimore, L. M., & Joordens, S. (2011). Assessing the effectiveness of a voluntary online discussion forum on improving students’ course performance. Computers & Education, 56(1), 253–261. https://doi.org/10.1016/j.compedu.2010.07.024

    Article  Google Scholar 

  • Cho, M. H., & Tobias, S. (2016). Should instructors require discussion in online courses? Effects of online discussion on community of inquiry, learner time, satisfaction, and achievement. International Review of Research in Open and Distributed Learning, 17(2), 123–140.

    Article  Google Scholar 

  • Curtis, D. D., & Lawson, M. J. (2001). Exploring collaborative online learning. Journal of Asynchronous Learning Networks, 5(1), 21–34.

    Google Scholar 

  • Davies, J., & Graff, M. (2005). Performance in e-learning: Online participation and student grades. British Journal of Educational Technology, 36(4), 657–663. https://doi.org/10.1111/j.1467-8535.2005.00542.x

    Article  Google Scholar 

  • Demir, B. P., & Kutlu, Ö. (2016). The effect of electronic portfolio applications on 6th graders’ research skills. Education in Science, 41(188), 227–253. https://doi.org/10.15390/EB.2016.6724

    Google Scholar 

  • Dennen, V. P. (2008). Looking for evidence of learning: Assessment and analysis methods for online discourse. Computers in Human Behavior, 24(2), 205–219. https://doi.org/10.1016/j.chb.2007.01.010

    Article  Google Scholar 

  • De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1), 6–28. https://doi.org/10.1016/j.compedu.2005.04.005

    Article  Google Scholar 

  • De Wever, B., Van Keer, H., Schellens, T., & Valcke, M. (2009). Structuring asynchronous discussion groups: The impact of role assignment and self-assessment on students’ levels of knowledge construction through social negotiation. Journal of Computer Assisted Learning, 25(2), 177–188. https://doi.org/10.1111/j.1365-2729.2008.00292.x

    Article  Google Scholar 

  • Gelbal, S., & Kelecioğlu, H. (2007). Öğretmenlerin ölçme ve değerlendirme yöntemleri hakkındaki yeterlik algıları ve karşılaştıkları sorunlar. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 33, 135–145.

    Google Scholar 

  • Guan, Y.-H., Tsai, C.-C., & Hwang, F.-K. (2006). Content analysis of online discussion on a senior-high-school discussion forum of a virtual physics laboratory. Instructional Science, 34(4), 279–311. https://doi.org/10.1007/s11251-005-3345-1

    Article  Google Scholar 

  • Güler, N. (2011). Rasgele veriler üzerinde Genellenebilirlik Kuramı ve Klasik Test Kuramı’na göre güvenirliğin karşılaştırılması. Eğitim ve Bilim, 36(162), 225–234.

    Google Scholar 

  • Haladyna, T. M. (1997). Writing test item to evaluate higher order thinking. USA: Allyn & Bacon.

    Google Scholar 

  • Hara, N., Bonk, C. J., & Angeli, C. (2002). Content analysis of online discussion in an applied educational psychology course. Instructional Science, 28, 115–152.

    Article  Google Scholar 

  • Haşlaman, T., Demiraslan, Y., Kuşkaya-Mumcu, F., Dönmez, O., & Aşkar, P. (2008). Çevrimiçi ortamda yapılan grup tartışmasındaki iletişim örüntülerinin söylem çözümlemesi yoluyla incelenmesi. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 35, 162–174.

    Google Scholar 

  • Hew, K. F., & Cheung, W. S. (2003). Evaluating the participation and quality of thinking of pre-service teachers in an asynchronous online discussion environment: Part I. International Journal of Instructional Media, 30(3), 247–262.

    Google Scholar 

  • Hew, K. F., & Cheung, W. S. (2012). Student participation in online discussions: Challenges, solutions, and future research. New York: Springer.

    Book  Google Scholar 

  • Hillman, D., Willis, D., & Gunawardena, C. (1994). Learner-interface interaction in distance education: An extension of contemporary models and strategies for practitioners. American Journal of Distance Education, 8(2), 30–42.

    Article  Google Scholar 

  • Hrastinski, S. (2008). What is online learner participation? A literature review. Computers & Education, 51(4), 1755–1765. https://doi.org/10.1016/j.compedu.2008.05.005

    Article  Google Scholar 

  • Hrastinski, S. (2009). A theory of online learning as online participation. Computers & Education, 52(1), 78–82. https://doi.org/10.1016/j.compedu.2008.06.009

    Article  Google Scholar 

  • Hsiao, Y.-T., Wise, A. F., & Marbouti, F. (2012). The impact of task type on learners’ online interaction patterns. Paper presented at the Annual Meeting of the American Education Research Association, Vancouver, Canada.

  • Huang, H. M. (2000). Instructional technologies facilitating online courses. Educational Technology, 40(4), 41–46.

    Google Scholar 

  • Huang, H. M. (2002). Toward constructivism for adult learners in online learning environments. British Journal of Educational Technology, 33(1), 27–37. https://doi.org/10.1111/1467-8535.00236

    Article  Google Scholar 

  • Kalelioglu, F., & Gülbahar, Y. (2010). Çevrimiçi Tartışmaların Değerlendirilmesi için Ölçütlerin Belirlenmesi. Eğitim Teknolojileri Araştırma Dergisi, 1(3). http://www.acarindex.com/dosyalar/makale/acarindex-1423877264.pdf. Accessed 08 July 2017.

  • Kanuka, H., & Anderson, T. (1998). Online social interchange, discord and knowledge construction. Journal of Distance Education, 13(1), 57–74.

    Google Scholar 

  • Kay, R. H. (2006). Developing a comprehensive metric for assessing discussion board effectiveness. British Journal of Educational Technology, 37(5), 761–783. https://doi.org/10.1111/j.1467-8535.2006.00560.x

    Article  Google Scholar 

  • Kızılkaya, G., & Usluel, Y. K. (2008). Web tabanlı öğrenme ortamlarında etkileşim. Paper presented at the 8 th International Educational Technology Conference (IECT). Eskişehir: Anadolu University.

    Google Scholar 

  • Kim, B., & Johnson, P. (2006). Graphical Interface for visual exploration of online discussion forums. Systemics, Cybernetics and Informatics, 4(4), 43–47.

    Google Scholar 

  • Kutlu, Ö., Bilican, S., & Yıldırım, Ö. (2010). A study on the primary school teachers’ attitudes towards rubrics with reference to different variables. Procedia - Social and Behavioral Sciences, 2(2), 5398–5402.

    Article  Google Scholar 

  • Kutlu, Ö., Doğan, C. D., & Karakaya, İ. (2009). Öğrenci başarısının belirlenmesi (2nd ed.). Ankara: PegemaA Yayıncılık.

    Google Scholar 

  • MacKnight, C. B. (2000). Teaching critical thinking through online discussion. Educause Quarterly, 4, 38–41.

    Google Scholar 

  • Marbouti, F., & Wise, A. F. (2016). Starburst: A new graphical interface to support purposeful attention to others’ posts in online discussions. Educational Technology Research and Development, 64(1), 87–113. https://doi.org/10.1007/s11423-015-9400-y

    Article  Google Scholar 

  • Mazzolini, M., & Maddison, S. (2003). Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums. Computers & Education, 40(3), 237–253.

    Article  Google Scholar 

  • Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research & Evaluation, 7(10), 71–81.

    Google Scholar 

  • Newman, D. R., Johnson, C., Cochrane, C., & Webb, B. (1996). An experiment in group learning technology: Evaluating critical thinking in face-to-face and computer-supported seminars. Interpersonal Computing and Technology: An Electronic Journal for the 21st Century, 4(1), 57–74.

    Google Scholar 

  • Ömür, S., & Erkuş, A. (2013). Dereceli puanlama anahtarıyla, genel izlenimle ve ikili karşılaştırmalar yöntemiyle yapılan değerlendirmelerin karşılaştırılması. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 28(2), 308–320.

    Google Scholar 

  • Perkins, C., & Murphy, E. (2006). Identifying and measuring individual engagement in critical thinking in online discussions: An exploratory case study. Educational Technology & Society, 9(1), 298–307.

    Google Scholar 

  • Popham, J. W. (1997). What’s wrong and what’s right with rubric. Educational Leadership, 55(2), 72–75.

    Google Scholar 

  • Rovai, A. (2007). Facilitating online discussions effectively. The Internet and Higher Education, 10(1), 77–88.

    Article  Google Scholar 

  • Swan, K. (2002). Building learning communities in online courses: The importance of interaction. Education, Communication & Information, 2(1), 23–49. https://doi.org/10.1080/1463631022000005016

    Article  Google Scholar 

  • Thomas, M. J. W. (2002). Learning within incoherent structures: The space of online discussion forums. Journal of Computer Assisted Learning, 18(3), 351–366. https://doi.org/10.1046/j.0266-4909.2002.03800.x

    Article  Google Scholar 

  • Tu, C.-H., & McIsaac, M. (2002). The relationship of social presence and interaction in online classes. American Journal of Distance Education, 16(3), 131–150. https://doi.org/10.1207/S15389286AJDE1603_2

    Article  Google Scholar 

  • Wee, M., & Abrizah, A. (2011). An analysis of an assessment model for participation in online forums. Computer Science and Information Systems, 8(1), 121–140. https://doi.org/10.2298/csis100113036c

    Article  Google Scholar 

  • Wickersham, L. E., & Dooley, K. E. (2006). A content analysis of critical thinking skills as an indicator of quality of online discussion in virtual learning communities. Quarterly Review of Distance Education, 7(2), 185–193.

    Google Scholar 

  • Wise, A. F., Saghafian, M., & Padmanabhan, P. (2012). Towards more precise design guidance: Specifying and testing the functions of assigned student roles in online discussions. Educational Technology Research and Development, 60(1), 55–82. https://doi.org/10.1007/s11423-011-9212-7

    Article  Google Scholar 

  • Yang, D., Sinha, T., Adamson, D., & Rose, C. P. (2013). Turn on, tune in, drop out: Anticipating student dropouts in massive open online courses. In NIPS workshop on data driven education, advances in neural information processing systems (NIPS – DDE 2013).  http://lytics.stanford.edu/datadriveneducation/papers/yangetal.pdf. Accessed: 30 June 2017.

  • Zhu, E. (2006). Interaction and cognitive engagement: An analysis of four asynchronous online discussions. Instructional Science, 34(6), 451–480. https://doi.org/10.1007/s11251-006-0004-0

    Article  MathSciNet  Google Scholar 

  • Zhu, Y., Au, W., & Yates, G. (2016). University students’ self-control and self-regulated learning in a blended course. The Internet and Higher Education, 30, 54–62. https://doi.org/10.1016/j.iheduc.2016.04.001

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported with the project number of KBÜBAP-17- BM-330 by SRP Office of Karabük University, thank you.

Author information

Authors and Affiliations

  1. Letters of Faculty, Educational Sciences Department, Karabuk University, Demir Celik Kampusu, 78050, Karabuk, Turkey

    Fatma Betül Kurnaz

  2. TOBB Techical Science Vocational School, Computer Technology Department, Karabuk University, Demir Celik Kampusu, 78050, Karabuk, Turkey

    Esin Ergün

  3. Distance Education Center, Ankara University, 06830 Golbasi, Ankara, Turkey

    Hale Ilgaz

Authors
  1. Fatma Betül Kurnaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Esin Ergün
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hale Ilgaz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Esin Ergün.

Appendix

Appendix

Table 11 Rubric about participation in online discussion environments
Full size table

CALCULATING THE ONLINE PARTICIPATION SCORE

Total Score about Message Content (TSCM)Footnote 1= (CMS*7) + (OVM*7) + (IVM*4) + (CM*3) + (DS*7)

Total Score about Number and Density of Messages (TSNDM) = (NM*4) + (DM*4)

Online Participation Score = TSCM+TSNDM

TOTAL SCORE ABOUT THE LEVEL OF PARTICIPATION IN ONLINE DISCUSSION ENVIRONMENTS: ...............................................

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kurnaz, F.B., Ergün, E. & Ilgaz, H. Participation in online discussion environments: Is it really effective?. Educ Inf Technol 23, 1719–1736 (2018). https://doi.org/10.1007/s10639-018-9688-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-018-9688-4

Keywords

Search

Navigation