Abstract
We report on the design, development, implementation, research, and iteration of two educational data science (EDS) workshops focused on using R and RStudio for 44 doctoral students with limited and varied EDS backgrounds. Through qualitative and quantitative analysis of pre- and post-workshop surveys, we found that participants in EDS workshops are concerned and even fearful, yet possess relevant assets that come from a wide range of background experiences. Pedagogical factors—including being patient amidst errors, coding collaboratively, and explaining technical concepts in an accessible and rigorous manner—were strengths of the workshops, while the need for more time was a key possible improvement. Furthermore, participants’ self-reported confidence grew from before to after the workshop. Based on our initial design, revisions, and research findings, we describe five formative design guidelines for educational data science workshops that address doctoral students’ goals and needs. In total, this work implies that well-designed workshops and short courses can offer opportunities for a wide range of educational researchers to extend their expertise with newer methods to carry out impactful work—in turn shaping the emerging EDS field.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
References
Baker, R. S., Martin, T., & Rossi, L. M. (2016). Educational data mining and learning analytics. The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications, pp. 379–396. Wiley. https://doi.org/10.1002/9781118956588.ch16
Bell, J., Sawaya, S., & Cain, W. (2014). Synchromodal classes: Designing for shared learning experiences between face-to-face and online students. International Journal of Designs for Learning, 5(1). https://doi.org/10.14434/ijdl.v5i1.12657
Blaschke, L. M. (2012). Heutagogy and lifelong learning: A review of heutagogical practice and self-determined learning. The International Review of Research in Open and Distributed Learning, 13(1), 56–71. https://doi.org/10.19173/irrodl.v13i1.1076
Carlson, S., & Gadio, C. T. (2002). Teacher professional development in the use of technology. In W. D. Haddad & A. Draxler (Eds.), Technologies for education: Potentials, parameters, and prospects (pp. 118–132). UNESCO.
Chikasanda, V. K. M., Otrel-Cass, K., Williams, J., & Jones, A. (2013). Enhancing teachers’ technological pedagogical knowledge and practices: A professional development model for technology teachers in Malawi. International Journal of Technology and Design Education, 23, 597–622. https://doi.org/10.1007/s10798-012-9206-8
Daniel, B. K. (2019). Big data and data science: A critical review of issues for educational research. British Journal of Educational Technology, 50(1), 101–113. https://doi.org/10.1111/bjet.12595
Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199. https://doi.org/10.3102/0013189X08331140
Estrellado, R. A. (2022). The K-12 educator’s data guidebook: Reimagining practical data use in Schools. Routledge. https://doi.org/10.4324/9781003139751
Estrellado, R. A., Freer, E. A., Mostipak, J., Rosenberg, J. M., & Velásquez, I. C. (2020). Data science in education using R. Routledge.
Feldon, D. F., Jeong, S., Peugh, J., Roksa, J., Maahs-Fladung, C., Shenoy, A., & Oliva, M. (2017). Null effects of boot camps and short-format training for PhD students in life sciences. Proceedings of the National Academy of Sciences, 114(37), 9854–9858. https://doi.org/10.1073/pnas.1705783114
Fernández-Batanero, J. M., Montenegro-Rueda, M., Fernández-Cerero, J., & García-Martínez, I. (2022). Digital competences for teacher professional development. Systematic review. European Journal of Teacher Education, 45(4), 513–531. https://doi.org/10.1080/02619768.2020.1827389
Fiesler, & Proferes, N. (2018). “Participant” perceptions of Twitter research ethics. Social Media + Society, 4(1), 1–14. https://doi.org/10.1177/2056305118763366
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020). Mining big data in education: Affordances and challenges. Review of Research in Education, 44(1), 130–160. https://doi.org/10.3102/0091732X20903304
Francom, G. M. (2010). Teach me how to learn: Principles for fostering students’ self-directed learning skills. International Journal of Self-Directed Learning, 7(1), 29–44. https://www.oltraining.com/SDLwebsite/IJSDL/IJSDL7.1-2010.pdf#page=34
Gee, J. P. (2005). Semiotic social spaces and affinity spaces: From The Age of Mythology to today’s schools. In D. Barton & K. Tusting (Eds.), Beyond communities of practice: Language, power and social context (pp. 214–232). Cambridge University Press.
Greenhalgh, S. P., Staudt Willet, K. B., Rosenberg, J. M., & Koehler, M. J. (2020). Lessons learned from applying Twitter research methods to educational technology phenomena. In E. Romero-Hall (Ed.), Research methods in learning design & technology (pp. 64–77). Routledge.
Hardin, J., Hoerl, R., Horton, N. J., Nolan, D., Baumer, B., Hall-Holt, O., Murrell, P., Peng, R., Roback, P., Temple Lang, D., & Ward, M. D. (2015). Data science in statistics curricula: Preparing students to “think with data.” The American Statistician, 69(4), 343–353. https://doi.org/10.1080/00031305.2015.1077729
Hazzan, O., & Mike, K. (2023). Data science teacher preparation: Implementation of the TPACK framework. ACM Inroads, 14(3), 39–44. https://doi.org/10.1145/3614100
Horton, N. J., & Hardin, J. S. (2021). Integrating computing in the statistics and data science curriculum: Creative structures, novel skills and habits, and ways to teach computational thinking. Journal of Statistics and Data Science Education, 29(sup1), S1–S3. https://doi.org/10.1080/10691898.2020.1870416
Ifenthaler, & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64(5), 923–938. https://doi.org/10.1007/s11423-016-9477-y
Jordan, K. L., Marwick, B., Weaver, B., Zimmerman, N., Williams, J., Teal, T., Becker, E., Duckles, J., Duckles, B., & Wickes, E. (2017). Analysis of the Carpentries’ Long-Term Feedback Survey. https://doi.org/10.5281/zenodo.1039944
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler Publishers.
Kizilcec, R. F., & Davis, D. (2022). Learning analytics education: A case study, review of current programs, and recommendations for instructors [pre-print]. https://doi.org/10.35542/osf.io/4uyex
Knowles, M. S. (1975). Self-directed learning: A guide for learners and teachers. Cambridge Adult Education.
Krumm, A., Means, B., & Bienkowski, M. (2018). Learning analytics goes to school: A collaborative approach to improving education. Routledge. https://doi.org/10.4324/9781315650722
Institute for Education Sciences. (2022). Methods Training for Education Research. https://ies.ed.gov/ncer/projects/program.asp?ProgID=82
Lang, C., Siemens, G. Wise, A. F., Gašević, D, & Merceron, A. (2022). Handbook of learning analytics (2nd Ed.). https://www.solaresearch.org/publications/hla-22/
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge University Press.
Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabási, A.-L., Brewer, D., & Alstyne, M. (2009). Computational social science. Science, 323, 721–723. https://doi.org/10.1126/science.1167742
Lee, V. R., Wilkerson, M. H., & Lanouette, K. (2021). A call for a humanistic stance toward K–12 data science education. Educational Researcher, 50(9), 664–672. https://doi.org/10.3102/0013189X211048810
Louws, M. L., Meirink, J. A., van Veen, K., & van Driel, J. H. (2017). Teachers’ self- directed learning and teaching experience: What, how, and why teachers want to learn. Teaching and Teacher Education, 66, 171–183. https://doi.org/10.1016/j.tate.2017.04.004
Mayer-Schönberger, V. (2015). Big data for cardiology: Novel discovery? European Heart Journal, 37(12), 996–1001. https://doi.org/10.1093/eurheartj/ehv648
McFarland, D. A., Khanna, S., Domingue, B. W., & Pardos, Z. A. (2021). Education data science: Past, present, future. AERA Open, 7(1), 1–12. https://doi.org/10.1177/23328584211052055
McNamara, A. (2018). Key attributes of a modern statistical computing tool. The American Statistician.
Munroe, R. (n.d.). Tech support cheat sheet. xkcd. Retrieved from https://xkcd.com/627/
National Center for Education Statistics. (2022). National assessment of educational progress. https://nces.ed.gov/nationsreportcard/
Nolan, D., & Temple Lang, D. (2010). Computing in the statistics curricula. The American Statistician, 64(2), 97–107. https://doi.org/10.1198/tast.2010.09132
Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64. Retrieved from https://www.jstor.org/stable/jeductechsoci.17.4.49
Pimentel, D. R., Horton, N. J., & Wilkerson, M. H. (2022). Tools to support data analysis and data science in K-12 education. National Academies of Science.
Plair, S. K. (2008). Revamping professional development for technology integration and fluency. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 82(2), 70–74. https://doi.org/10.3200/TCHS.82.2.70-74
Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3), e1355. https://doi.org/10.1002/widm.1355
Rosenberg, J. M., & Jones, R. S. (2022). A secret agent? K-12 data science learning through the lens of agency. EdArXiv. https://doi.org/10.35542/osf.io/eyzkv
Rosenberg, J. M., & Staudt Willet, K. B. (2021). Balancing privacy and open science in the context of COVID-19: A response to Ifenthaler & Schumacher (2016). Educational Technology Research and Development, 69, 347–351. https://doi.org/10.1007/s11423-020-09860-8
Salganik, M. J. (2019). Bit by bit: Social research in the digital age. Princeton University Press.
Saldaña, J. (2011). Fundamentals of qualitative research. Oxford University Press.
Sane, M., & Becker, E. (2018). The Carpentries programmatic assessment report: 2018 Q1. Zenodo. https://doi.org/10.5281/zenodo.1404426
Schwab-McCoy, A., Baker, C. M., & Gasper, R. E. (2021). Data science in 2020: Computing, curricula, and challenges for the next 10 years. Journal of Statistics and Data Science Education, 29(sup1), S40–S50. https://doi.org/10.1080/10691898.2020.1851159
Selwyn, N., Hillman, T., Eynon, R., Ferreira, G., Knox, J., Macgilchrist, F., & Sancho-Gil, J. M. (2020). What’s next for Ed-Tech? Critical hopes and concerns for the 2020s. Learning, Media and Technology, 45(1), 1–6. https://doi.org/10.1080/17439884.2020.1694945
Slater, S., Joksimović, S., Kovanovic, V., Baker, R. S., & Gasevic, D. (2017). Tools for educational data mining: A review. Journal of Educational and Behavioral Statistics, 42(1), 85–106. https://doi.org/10.3102/1076998616666808
Thayer, K., & Ko, A. J. (2017, August). Barriers faced by coding bootcamp students. In Proceedings of the 2017 ACM Conference on International Computing Education Research (pp. 245–253). https://doi.org/10.1145/3105726.3106176
The Carpentries. (2022a). The Carpentries pre-workshop survey. https://carpentries.github.io/assessment-archives/pre-workshop/pre-workshop.html
The Carpentries. (2022b). The Carpentries post-workshop survey. https://carpentries.github.io/assessment-archives/post-workshop/post-workshop.html
Welser, H. T., Smith, M., Fisher, D., & Gleave, E. (2008). Distilling digital traces: Computational social science approaches to studying the Internet. In N. Fielding, R. M. Lee, & G. Blank (Eds.), The SAGE handbook of online research methods (pp. 116–141). SAGE. https://doi.org/10.4135/9780857020055
Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency. Learning, Media and Technology, 45(2), 107–114. https://doi.org/10.1080/17439884.2020.1761641
Word, K. R. (2017). When do workshops work? A response to the ‘null effects’ paper from Feldon et al. https://software-carpentry.org/blog/2017/12/response-null-effects.html
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Staudt Willet, K.B., Rosenberg, J.M. The Design and Effects of Educational Data Science Workshops for Early Career Researchers. J Form Des Learn 7, 83–97 (2023). https://doi.org/10.1007/s41686-023-00083-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41686-023-00083-7