Abstract
Makerspaces, especially in their diverse proliferating forms, support a broad variety of learning outcomes. There is rich work in attempting to understand and describe these learning goals. Yet, there is a lack of support for practitioners and educators to assess the learning in events and programming at makerspaces (and similar environments) without extensive videorecording and documentation. In this paper, we present our design iterations at adapting the Tinkering Studio’s Learning Dimensions Framework (LDF) into tools usable by makerspace facilitators. These tools are intended to support recording observations, to inform the design of events they organize. Coupling an activity theory perspective (Cole and Engeström in The Cambridge handbook of sociocultural psychology. Cambridge University Press, Cambridge, 2007) with Tatar’s (2007) Design Tensions framework, we highlight key categories of considerations that emerge in creating and implementing such an assessment system, namely, tools, terminology, and practice. These interlinked categories foreground the following tensions which expand our considerations for the practice of assessment in makerspaces: supporting real-time, informative observation increases granularity of data collected, but also imposes a cost on facilitator attention; using a common assessment framework across different facilitators requires developing and establishing shared vocabulary and understanding; and tool-driven assessments need repeated adaptation and responsiveness to different facilitator practices. Additionally, this analysis also surfaces the learning for facilitators themselves in such a co-design process of creating and implementing tools to understand, recognize and assess learning experiences through the lenses of personal and shared values around productive learning.
Similar content being viewed by others
References
Austin, K., Ehrlich, S. B., Puckett, C., & Singleton, J. (2010). YOUmedia Chicago: Reimagining Learning, Literacies, and Libraries—A Snapshot Of Year 1. Working Paper. Consortium on Chicago School Research.
Benjamin, R. (2016). Catching our breath: Critical race STS and the carceral imagination. Engaging Science, Technology, and Society, 2, 145–156.
Bergner, Y., & Chen, O. (2018). Deep making: curricular modules for transferable content-knowledge and scientific literacy in makerspaces and FabLabs. In Proceedings of the 17th ACM Conference on Interaction Design and Children—IDC ’18 (pp. 551–556). https://doi.org/10.1145/3202185.3210774
Bernstein, B. B. (2003). Class, codes, and control. Routledge.
Bevan, B., Gutwill, J. P., Petrich, M., & Wilkinson, K. (2015). Learning through STEM-rich tinkering: Findings from a jointly negotiated research project taken up in practice: LEARNING THROUGH STEM-RICH TINKERING. Science Education, 99(1), 98–120. https://doi.org/10.1002/sce.21151
Blikstein, P. (2013). Digital fabrication and ‘making’in education: The democratization of invention. FabLabs: Of Machines Makers and Inventors, 4, 1–21.
Blikstein, P., Kabayadondo, Z., Martin, A., & Fields, D. (2017). An Assessment Instrument of Technological Literacies in Makerspaces and FabLabs: Assessment of Technological Literacies in Makerspaces and FabLabs. Journal of Engineering Education, 106(1), 149–175. https://doi.org/10.1002/jee.20156
Brahms, L., & Wardrip, P. (2014). The learning practices of making: An evolving framework for design. Children’s Museum of Pittsburgh.
Buldu, M. (2010). Making learning visible in kindergarten classrooms: Pedagogical documentation as a formative assessment technique. Teaching and Teacher Education, 26(7), 1439–1449.
Chang, S., Penney, L., Wardrip, P., Anderson, A., Craddock, I., Martin, C, & Dow, K. (2019). Opportunities and vignettes for library makerspaces. MakerEd and the University of Wisconsin Madison. White Paper.
Coburn, C. E., & Penuel, W. R. (2016). Research-practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/0013189X16631750
Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45(6), 378–387.
Cole, M., & Engeström, Y. (2007). Cultural-historical approaches to designing for development. Cambridge University Press.
Dijkstra, P., Kuyper, H., Van der Werf, G., Buunk, A. P., & van der Zee, Y. G. (2008). Social comparison in the classroom: A review. Review of Educational Research, 78(4), 828–879.
Dix, A. (2007, September 3). Designing for appropriation (pp. 27–30). Retrieved from http://dl.acm.org.ezproxy.library.wisc.edu/citation.cfm?id=1531407.1531415
Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic fieldnotes. University of Chicago Press.
Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. O. R. A. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. Teachers College Record, 115(14), 136–156.
Gitomer, D. (2014). Development of the Dimensions of Success (DoS) Observation Tool for the Out of School Time STEM Field: Refinement, Field-testing and Establishment of Psychometric Properties. Retrieved from http://www.pearweb.org/tools/dos.html
Gutiérrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated praxis, transformative learning, and social design experiments. Journal of Teacher Education, 61(1–2), 100–117.
Halverson, E., Lakind, A., & Willett, R. (2017). The bubbler as systemwide makerspace: A design case of how making became a core service of the public libraries. International Journal of Designs for Learning. https://doi.org/10.14434/ijdl.v8i1.22653
Hill, H., & Grossman, P. (2013). Learning from teacher observations: Challenges and opportunities posed by new teacher evaluation systems. Harvard Educational Review, 83(2), 371–384.
Hirshberg, P., Dougherty, D., & Kadanoff, M. (2016). Maker city: A practical guide for reinventing American Cities. Maker Media Inc.
Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., & Pascoe, C. J. (2010). Hanging out, messing around, and geeking out. MIT Press.
Körkkö, M., Kyrö-Ämmälä, O., & Turunen, T. (2016). Professional development through reflection in teacher education. Teaching and Teacher Education, 55, 198–206.
Kumar, V., Millerjohn, R., & Wardrip, P. (2019). Designing tools for observation and assessment in makerspaces. In Proceedings of FabLearn 2019 (pp. 197–200).
Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), Article 4. https://doi.org/10.7771/2157-9288.1099
Murai, Y., Kim, Y. J., Martin, E., Kirschmann, P., Rosenheck, L., & Reich, J. (2019). Embedding assessment in school-based making: preliminary exploration of principles for embedded assessment in maker learning. In Proceedings of FabLearn 2019 (pp. 180–183).
Martin, C. K., Reyes, E., Ramirez, E., Brahms, L., & Wardrip, P. (2020). Supporting Educator Reflection and Agency Through the Co-Design of Observation Tools and Practices for Informal Learning Environment (JH Kalir & D. Filipiak, Eds.; Vol. 1). Pittsburgh.
Niess, M. L. (2011). Investigating TPACK: Knowledge growth in teaching with technology. Journal of Educational Computing Research, 44(3), 299–317.
Ocumpaugh, J. (2015). Baker Rodrigo Ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences.
Penuel, W. R., Allen, C. D., Manz, E., & Heredia, S. C. (2022). Design-based implementation research as an approach to studying teacher learning in research-practice partnerships focused on equity. Teacher learning in changing contexts (pp. 217–237). Routledge.
Peppler, K., & Bender, S. (2013). Maker movement spreads innovation one project at a time. Phi Delta Kappan, 95(3), 22–27. https://doi.org/10.1177/003172171309500306
Peppler, K., Maltese, A., Keune, A., Chang, S., & Regalla, L. (2015). Survey of Makerspaces, Part II, Maker Ed Initiative.
Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In Handbook of research on student engagement (pp. 365–386). Springer, Boston, MA.
Reilly, J. M., Ravenell, M., & Schneider, B. (2018). Exploring Collaboration Using Motion Sensors and Multi-Modal Learning Analytics. Retrieved from https://eric.ed.gov/?id=ED593204
Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18–36.
Sheridan, K., Halverson, E. R., Litts, B., Brahms, L., Jacobs-Priebe, L., & Owens, T. (2014). Learning in the Making: A comparative case study of three makerspaces. Harvard Educational Review, 84(4), 505–531. https://doi.org/10.17763/haer.84.4.brr34733723j648u
Smith, C., & Hohmann, C. (2005). Full findings from the Youth PQA validation study. High/Scope Educational Research Foundation.
Tekkumru-Kisa, M., & Stein, M. K. (2015). Learning to see teaching in new ways: A foundation for maintaining cognitive demand. American Educational Research Journal, 52(1), 105–136.
Tseng, T. (2016). Build in progress: Building process-oriented documentation. Makeology: Makerspaces as learning environments. Routledge.
Wardrip, P. (2014). Badging to support teaching and student engagement: An implementation of a school-based badging system (Doctoral dissertation, University of Pittsburgh).
Wardrip, P. S., & Brahms, L. (2015). Learning practices of making: developing a framework for design. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15), pp. 375–378. New York, NY: Association for Computing Machinery. https://doi.org/10.1145/2771839.2771920
Wardrip, P. S., & Brahms, L. (2016). Taking making to school. Makeology: Makerspaces as learning environments, 1, 97–106.
Wardrip, P. S., & Brahms, L. (2020). Supporting learning in museum makerspaces: A National Framework. Journal of Museum Education, 45(4), 476–483.
Wardrip, P., Chang, S., Penney, L., Abramovich, S., Millerjohn, R., Kumar, V., Martin, C., Widman, S., Penuel, W. R., Chang-Order, J., & Halverson, E. (2020). Assessment in Hands-On Library Learning Spaces. In M. Gresalfi & I. S. Horn (Eds.), The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS) 2020 (Vol. 3, pp. 1525–1530). International Society of the Learning Sciences.
Wardrip, P., Evancho, J., & McNamara, A. (2018). Identifying what matters. Phi Delta Kappan, 99(6), 60–63.
Wardrip, P. S., White, A., & Brahms, L. (2023). Learning practices in a museum art studio: Designing a learning framework for an arts-based learning team. International Journal of Designs for Learning, 14(2), 112–120.
White, A. M., Saplan, K., Wardrip, P. S., Bank, A., Akiva, T., & Brahms, L. (2022). The seek & share resources tool: Measuring a learning practice of making. Visitor Studies, 25(2), 217–237.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kumar, V., Wardrip, P. & Millerjohn, R. Design tensions in developing and using observation and assessment tools in makerspaces. Education Tech Research Dev 72, 261–280 (2024). https://doi.org/10.1007/s11423-023-10330-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11423-023-10330-0