Skip to main content
Log in

Design tensions in developing and using observation and assessment tools in makerspaces

  • Development Article
  • Published:
Educational technology research and development Aims and scope Submit manuscript

Abstract

Makerspaces, especially in their diverse proliferating forms, support a broad variety of learning outcomes. There is rich work in attempting to understand and describe these learning goals. Yet, there is a lack of support for practitioners and educators to assess the learning in events and programming at makerspaces (and similar environments) without extensive videorecording and documentation. In this paper, we present our design iterations at adapting the Tinkering Studio’s Learning Dimensions Framework (LDF) into tools usable by makerspace facilitators. These tools are intended to support recording observations, to inform the design of events they organize. Coupling an activity theory perspective (Cole and Engeström in The Cambridge handbook of sociocultural psychology. Cambridge University Press, Cambridge, 2007) with Tatar’s (2007) Design Tensions framework, we highlight key categories of considerations that emerge in creating and implementing such an assessment system, namely, tools, terminology, and practice. These interlinked categories foreground the following tensions which expand our considerations for the practice of assessment in makerspaces: supporting real-time, informative observation increases granularity of data collected, but also imposes a cost on facilitator attention; using a common assessment framework across different facilitators requires developing and establishing shared vocabulary and understanding; and tool-driven assessments need repeated adaptation and responsiveness to different facilitator practices. Additionally, this analysis also surfaces the learning for facilitators themselves in such a co-design process of creating and implementing tools to understand, recognize and assess learning experiences through the lenses of personal and shared values around productive learning.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Austin, K., Ehrlich, S. B., Puckett, C., & Singleton, J. (2010). YOUmedia Chicago: Reimagining Learning, Literacies, and Libraries—A Snapshot Of Year 1. Working Paper. Consortium on Chicago School Research.

  • Benjamin, R. (2016). Catching our breath: Critical race STS and the carceral imagination. Engaging Science, Technology, and Society, 2, 145–156.

    Article  Google Scholar 

  • Bergner, Y., & Chen, O. (2018). Deep making: curricular modules for transferable content-knowledge and scientific literacy in makerspaces and FabLabs. In Proceedings of the 17th ACM Conference on Interaction Design and Children—IDC ’18 (pp. 551–556). https://doi.org/10.1145/3202185.3210774

  • Bernstein, B. B. (2003). Class, codes, and control. Routledge.

    Google Scholar 

  • Bevan, B., Gutwill, J. P., Petrich, M., & Wilkinson, K. (2015). Learning through STEM-rich tinkering: Findings from a jointly negotiated research project taken up in practice: LEARNING THROUGH STEM-RICH TINKERING. Science Education, 99(1), 98–120. https://doi.org/10.1002/sce.21151

    Article  ADS  Google Scholar 

  • Blikstein, P. (2013). Digital fabrication and ‘making’in education: The democratization of invention. FabLabs: Of Machines Makers and Inventors, 4, 1–21.

    Google Scholar 

  • Blikstein, P., Kabayadondo, Z., Martin, A., & Fields, D. (2017). An Assessment Instrument of Technological Literacies in Makerspaces and FabLabs: Assessment of Technological Literacies in Makerspaces and FabLabs. Journal of Engineering Education, 106(1), 149–175. https://doi.org/10.1002/jee.20156

    Article  Google Scholar 

  • Brahms, L., & Wardrip, P. (2014). The learning practices of making: An evolving framework for design. Children’s Museum of Pittsburgh.

  • Buldu, M. (2010). Making learning visible in kindergarten classrooms: Pedagogical documentation as a formative assessment technique. Teaching and Teacher Education, 26(7), 1439–1449.

    Article  Google Scholar 

  • Chang, S., Penney, L., Wardrip, P., Anderson, A., Craddock, I., Martin, C, & Dow, K. (2019). Opportunities and vignettes for library makerspaces. MakerEd and the University of Wisconsin Madison. White Paper.

  • Coburn, C. E., & Penuel, W. R. (2016). Research-practice partnerships in education: Outcomes, dynamics, and open questions. Educational Researcher, 45(1), 48–54. https://doi.org/10.3102/0013189X16631750

    Article  Google Scholar 

  • Cohen, J., & Goldhaber, D. (2016). Building a more complete understanding of teacher evaluation using classroom observations. Educational Researcher, 45(6), 378–387.

    Article  Google Scholar 

  • Cole, M., & Engeström, Y. (2007). Cultural-historical approaches to designing for development. Cambridge University Press.

    Google Scholar 

  • Dijkstra, P., Kuyper, H., Van der Werf, G., Buunk, A. P., & van der Zee, Y. G. (2008). Social comparison in the classroom: A review. Review of Educational Research, 78(4), 828–879.

    Article  Google Scholar 

  • Dix, A. (2007, September 3). Designing for appropriation (pp. 27–30). Retrieved from http://dl.acm.org.ezproxy.library.wisc.edu/citation.cfm?id=1531407.1531415

  • Emerson, R. M., Fretz, R. I., & Shaw, L. L. (2011). Writing ethnographic fieldnotes. University of Chicago Press.

    Book  Google Scholar 

  • Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. O. R. A. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. Teachers College Record, 115(14), 136–156.

    Article  Google Scholar 

  • Gitomer, D. (2014). Development of the Dimensions of Success (DoS) Observation Tool for the Out of School Time STEM Field: Refinement, Field-testing and Establishment of Psychometric Properties. Retrieved from http://www.pearweb.org/tools/dos.html

  • Gutiérrez, K. D., & Vossoughi, S. (2010). Lifting off the ground to return anew: Mediated praxis, transformative learning, and social design experiments. Journal of Teacher Education, 61(1–2), 100–117.

    Article  Google Scholar 

  • Halverson, E., Lakind, A., & Willett, R. (2017). The bubbler as systemwide makerspace: A design case of how making became a core service of the public libraries. International Journal of Designs for Learning. https://doi.org/10.14434/ijdl.v8i1.22653

    Article  Google Scholar 

  • Hill, H., & Grossman, P. (2013). Learning from teacher observations: Challenges and opportunities posed by new teacher evaluation systems. Harvard Educational Review, 83(2), 371–384.

    Article  Google Scholar 

  • Hirshberg, P., Dougherty, D., & Kadanoff, M. (2016). Maker city: A practical guide for reinventing American Cities. Maker Media Inc.

    Google Scholar 

  • Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., & Pascoe, C. J. (2010). Hanging out, messing around, and geeking out. MIT Press.

    Google Scholar 

  • Körkkö, M., Kyrö-Ämmälä, O., & Turunen, T. (2016). Professional development through reflection in teacher education. Teaching and Teacher Education, 55, 198–206.

    Article  Google Scholar 

  • Kumar, V., Millerjohn, R., & Wardrip, P. (2019). Designing tools for observation and assessment in makerspaces. In Proceedings of FabLearn 2019 (pp. 197–200).

  • Martin, L. (2015). The promise of the maker movement for education. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), Article 4. https://doi.org/10.7771/2157-9288.1099

  • Murai, Y., Kim, Y. J., Martin, E., Kirschmann, P., Rosenheck, L., & Reich, J. (2019). Embedding assessment in school-based making: preliminary exploration of principles for embedded assessment in maker learning. In Proceedings of FabLearn 2019 (pp. 180–183).

  • Martin, C. K., Reyes, E., Ramirez, E., Brahms, L., & Wardrip, P. (2020). Supporting Educator Reflection and Agency Through the Co-Design of Observation Tools and Practices for Informal Learning Environment (JH Kalir & D. Filipiak, Eds.; Vol. 1). Pittsburgh.

  • Niess, M. L. (2011). Investigating TPACK: Knowledge growth in teaching with technology. Journal of Educational Computing Research, 44(3), 299–317.

    Article  MathSciNet  Google Scholar 

  • Ocumpaugh, J. (2015). Baker Rodrigo Ocumpaugh monitoring protocol (BROMP) 2.0 technical and training manual. New York, NY and Manila, Philippines: Teachers College, Columbia University and Ateneo Laboratory for the Learning Sciences.

  • Penuel, W. R., Allen, C. D., Manz, E., & Heredia, S. C. (2022). Design-based implementation research as an approach to studying teacher learning in research-practice partnerships focused on equity. Teacher learning in changing contexts (pp. 217–237). Routledge.

    Chapter  Google Scholar 

  • Peppler, K., & Bender, S. (2013). Maker movement spreads innovation one project at a time. Phi Delta Kappan, 95(3), 22–27. https://doi.org/10.1177/003172171309500306

  • Peppler, K., Maltese, A., Keune, A., Chang, S., & Regalla, L. (2015). Survey of Makerspaces, Part II, Maker Ed Initiative.

  • Pianta, R. C., Hamre, B. K., & Allen, J. P. (2012). Teacher-student relationships and engagement: Conceptualizing, measuring, and improving the capacity of classroom interactions. In Handbook of research on student engagement (pp. 365–386). Springer, Boston, MA.

  • Reilly, J. M., Ravenell, M., & Schneider, B. (2018). Exploring Collaboration Using Motion Sensors and Multi-Modal Learning Analytics. Retrieved from https://eric.ed.gov/?id=ED593204

  • Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18–36.

    Article  Google Scholar 

  • Sheridan, K., Halverson, E. R., Litts, B., Brahms, L., Jacobs-Priebe, L., & Owens, T. (2014). Learning in the Making: A comparative case study of three makerspaces. Harvard Educational Review, 84(4), 505–531. https://doi.org/10.17763/haer.84.4.brr34733723j648u

    Article  Google Scholar 

  • Smith, C., & Hohmann, C. (2005). Full findings from the Youth PQA validation study. High/Scope Educational Research Foundation.

    Google Scholar 

  • Tekkumru-Kisa, M., & Stein, M. K. (2015). Learning to see teaching in new ways: A foundation for maintaining cognitive demand. American Educational Research Journal, 52(1), 105–136.

    Article  Google Scholar 

  • Tseng, T. (2016). Build in progress: Building process-oriented documentation. Makeology: Makerspaces as learning environments. Routledge.

    Google Scholar 

  • Wardrip, P. (2014). Badging to support teaching and student engagement: An implementation of a school-based badging system (Doctoral dissertation, University of Pittsburgh).

  • Wardrip, P. S., & Brahms, L. (2015). Learning practices of making: developing a framework for design. In Proceedings of the 14th International Conference on Interaction Design and Children (IDC '15), pp. 375–378. New York, NY: Association for Computing Machinery. https://doi.org/10.1145/2771839.2771920

  • Wardrip, P. S., & Brahms, L. (2016). Taking making to school. Makeology: Makerspaces as learning environments, 1, 97–106.

    Article  Google Scholar 

  • Wardrip, P. S., & Brahms, L. (2020). Supporting learning in museum makerspaces: A National Framework. Journal of Museum Education, 45(4), 476–483.

    Article  Google Scholar 

  • Wardrip, P., Chang, S., Penney, L., Abramovich, S., Millerjohn, R., Kumar, V., Martin, C., Widman, S., Penuel, W. R., Chang-Order, J., & Halverson, E. (2020). Assessment in Hands-On Library Learning Spaces. In M. Gresalfi & I. S. Horn (Eds.), The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (ICLS) 2020 (Vol. 3, pp. 1525–1530). International Society of the Learning Sciences.

    Google Scholar 

  • Wardrip, P., Evancho, J., & McNamara, A. (2018). Identifying what matters. Phi Delta Kappan, 99(6), 60–63.

    Article  Google Scholar 

  • Wardrip, P. S., White, A., & Brahms, L. (2023). Learning practices in a museum art studio: Designing a learning framework for an arts-based learning team. International Journal of Designs for Learning, 14(2), 112–120.

    Article  Google Scholar 

  • White, A. M., Saplan, K., Wardrip, P. S., Bank, A., Akiva, T., & Brahms, L. (2022). The seek & share resources tool: Measuring a learning practice of making. Visitor Studies, 25(2), 217–237.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vishesh Kumar.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, V., Wardrip, P. & Millerjohn, R. Design tensions in developing and using observation and assessment tools in makerspaces. Education Tech Research Dev 72, 261–280 (2024). https://doi.org/10.1007/s11423-023-10330-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-023-10330-0

Keywords

Navigation