Skip to main content
SpringerLink
Log in

Design-based implementation research: milestones and trade-offs in designing a collaborative representation tool for engineering classrooms

  • Development Article
  • Published:
Educational technology research and development Aims and scope Submit manuscript

Abstract

In response to the criticism that theory-driven researcher-developed learning tools lack scalability and sustainability in the real world, the design-based implementation research (DBIR) approach was proposed. However, few empirical studies actually describe what a DBIR study looks like and how it can inform readers about learning tool design. We engaged in a retrospective reflection to reconstruct our multi-year DBIR project experience based on team’s research and design documents and artifacts accumulated over 4 years, alongside conversations with the interdisciplinary design team members. Through constant comparison and ethnographic conversations, we describe our project in terms of the five DBIR milestones identified and four design tensions. We discuss how our project showcases evidence of scalability and sustainability of the tool, while effectiveness is addressed differently from design experiments. Implications and future directions are also provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

Not applicable.

References

  • Akkerman, S. F., & Bakker, A. (2011). Boundary crossing and boundary objects. Review of Educational Research, 81(2), 132–169.

    Article  Google Scholar 

  • Akkerman, S., Vandenbossche, P., Admiraal, W., Gijselaers, W., Segers, M., Simons, R., & Kirschner, P. (2007). Reconsidering group cognition: From conceptual confusion to a boundary area between cognitive and socio-cultural perspectives? Educational Research Review, 2(1), 39–63. https://doi.org/10.1016/j.edurev.2007.02.001

    Article  Google Scholar 

  • Al-Said, K. (2023). Effect of‗ Bring Your Own Device ‘(BYOD) on student behavior, well-being, and learning economic disciplines. International Journal of Information and Education Technology, 13(4).

  • Alvarez, C., Brown, C., & Nussbaum, M. (2011). Comparative study of netbooks and tablet PCs for fostering face-to-face collaborative learning. Computers in Human Behavior, 27(2), 834–844.

    Article  Google Scholar 

  • Aviani, I., Erceg, N., & Mešić, V. (2015). Drawing and using free body diagrams: Why it may be better not to decompose forces. Physical Review Special Topics-Physics Education Research, 11(2), 020137.

    Article  Google Scholar 

  • Barron, B. (2000). Achieving coordination in collaborative problem-solving groups. The Journal of the Learning Sciences, 9(4), 403–436.

    Article  Google Scholar 

  • Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences, 12(3), 307–359.

    Article  Google Scholar 

  • Barron, B., Martin, C. K., Mercier, E., Pea, R., Steinbock, D., Walter, S., Herrenkohl, L., Mertl, V., & Tyson, K. (2009). Repertoires of collaborative practice. In Proceedings of the 9th international conference on computer supported collaborative learning—CSCL’09 (pp. 25–27). https://doi.org/10.3115/1599503.1599513

  • Bause, I. M., Brich, I. R., Wesslein, A. K., & Hesse, F. W. (2018). Using technological functions on a multi-touch table and their affordances to counteract biases and foster collaborative problem solving. International Journal of Computer-Supported Collaborative Learning, 13(1), 7–33.

    Article  Google Scholar 

  • Beers, P. J., Boshuizen, H. P. E., Kirschner, P. A., & Gijselaers, W. H. (2005). Computer support for knowledge construction in collaborative learning environments. Computers in Human Behavior, 21(4), 623–643.

    Article  Google Scholar 

  • Boling, E. (2010). The need for design cases: Disseminating design knowledge. International Journal of Designs for Learning, 1(1).

  • Borge, M., & Mercier, E. (2019a). Towards a micro-ecological approach to CSCL. International Journal of Computer-Supported Collaborative Learning, 14(2), 219–235. https://doi.org/10.1007/s11412-019-09301-6

    Article  Google Scholar 

  • Borge, M., & Shimoda, T. (2019b). Designing a computer-supported-collective regulation system: A theoretically informed approach. Technology, Instruction, Cognition, & Learning, 11(2–3), 163–192.

    Google Scholar 

  • Borge, M., & Xia, Y. (2023). Beyond the individual: The regulation and negotiation of socioemotional practices across a learning ecosystem. Journal of the Learning Sciences, 1–51.

  • Bratman, M. E. (1992). Shared cooperative activity. The Philosophical Review, 101(2), 327–341.

    Article  Google Scholar 

  • Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178.

    Article  Google Scholar 

  • Collins, A. (1996). Design issues for learning environments. International Perspectives on the Design of Technology-Supported Learning Environments, 347–361.

  • Cress, U., Oshima, J., Rosé, C., & Wise, A. F. (2021). Foundations, processes, technologies, and methods: An overview of CSCL through its handbook. International handbook of computer-supported collaborative learning, 3–22.

  • Davis, B., Sinclair, N., & Whiteley, W. (2015). What is spatial reasoning? In Spatial reasoning in the early years (pp. 13–24). Routledge.

  • De Vries, E. (2006). Students’ construction of external representations in design-based learning situations. Learning and Instruction, 16(3), 213–227.

    Article  Google Scholar 

  • Dede, C. (2010). Technological supports for acquiring 21st century skills. International Encyclopedia of Education, 3, 158–166.

    Article  Google Scholar 

  • Dillenbourg, P., & Evans, M. (2011). Interactive tabletops in education. International Journal of Computer-Supported Collaborative Learning, 6, 491–514. https://doi.org/10.1007/s11412-011-9127-7

    Article  Google Scholar 

  • Ehrlich, G.S., Rajarathinam, R. J., & Mercier, E. (2022) Developing a boundary practice for collaborative task design in a design-centric research-practice partnership. In Chinn, C., Tan, E., Chan, C.,& Kali, Y.(Eds.). (2022). In Proceedings of the 16th international conference of the learning sciences-ICLS2022 (pp. 2082–2083). International Society of the Learning Sciences.

  • Ewert, D., Schuster, K., Johansson, D., Schilberg, D., & Jeschke, S. (2016). Intensifying learner’s experience by incorporating the virtual theatre into engineering education. In Engineering education 4.0 (pp. 91–103). Springer.

  • Fishman, B. J., Penuel, W. R., Allen, A. R., Cheng, B. H., & Sabelli, N. (2013). Design-based implementation research: An emerging model for transforming the relationship of research and practice. National Society for the Study of Education, 112(2), 136–156.

    Google Scholar 

  • Fleck, R., Rogers, Y., Yuill, N., Marshall, P., Carr, A., Rick, J., & Bonnett, V. (2009). Actions speak loudly with words: Unpacking collaboration around the table. In Proceedings of the ACM international conference on interactive tabletops and surfaces (pp. 189–196).

  • Gergen, M. M., & Gergen, K. J. (2000). Qualitative inquiry: Tensions and transformations. Handbook of Qualitative Research, 2, 1025–1046.

    Google Scholar 

  • Glaser, B. G., & Strauss, A. L. (1967). The constant comparative method of qualitative analysis. The Discovery of Grounded Theory: Strategies for Qualitative Research, 101, 158.

    Google Scholar 

  • Haßler, B., Major, L., & Hennessy, S. (2016). Tablet use in schools: A critical review of the evidence for learning outcomes. Journal of Computer Assisted Learning, 32(2), 139–156.

    Article  Google Scholar 

  • Hakkarainen, K., Paavola, S., Kangas, K. A. I. I. U., & Seitamaa-Hakkarainen, P. (2013). Chapter 3: Toward collaborative knowledge creation. In International handbook of collaborative learning (pp. 57–73). Routledge. https://doi.org/10.4324/9780203837290

  • He, W., & Zhao, L. (2020). Exploring undergraduates’ learning engagement via BYOD in the blended learning classroom (EULEBYODBLC). International Journal of Information and Education Technology, 10(2), 159–164.

    Article  Google Scholar 

  • Henderson, K. (1991). Flexible sketches and inflexible data bases: Visual communication, conscription devices, and boundary objects in design engineering. Science, Technology, & Human Values, 16, 448.

    Article  Google Scholar 

  • Herman, G. L., Zilles, C., & Loui, M. C. (2011). How do students misunderstand number representations? Computer Science Education, 21(3), 289–312.

    Article  Google Scholar 

  • Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A framework for teachable collaborative problem solving skills. In Assessment and teaching of 21st century skills (pp. 37–56). Springer.

  • Hibbeler, R. C. (2014). Mechanics of materials (9th ed.). Pearson Prentice Hall.

    Google Scholar 

  • Higgins, S. E., Mercier, E., Burd, E., & Hatch, A. (2011). Multi-touch tables and the relationship with collaborative classroom pedagogies: A synthetic review. International Journal of Computer-Supported Collaborative Learning, 6(4), 515–538. https://doi.org/10.1007/s11412-011-9131-y

    Article  Google Scholar 

  • Hung, W. (2016). All PBL starts here: The problem. Interdisciplinary Journal of Problem-Based Learning, 10(2), 2.

    Article  Google Scholar 

  • Johnson-Glauch, N., Choi, D. S., & Herman, G. (2020). How engineering students use domain knowledge when problem-solving using different visual representations. Journal of Engineering Education, 109(3), 443–469.

    Article  Google Scholar 

  • Johri, A., Roth, W. M., & Olds, B. M. (2013). The role of representations in engineering practices: Taking a turn towards inscriptions. Journal of Engineering Education, 102(1), 2–19.

    Article  Google Scholar 

  • Jonassen, D., Strobel, J., & Lee, C. B. (2006). Everyday problem solving in engineering: Lessons for engineering educators. Journal of Engineering Education, 95(2), 139–151.

    Article  Google Scholar 

  • Juhl, J., & Lindegaard, H. (2013). Representations and visual synthesis in engineering design. Journal of Engineering Education. https://doi.org/10.1002/jee.20001

    Article  Google Scholar 

  • Koike, H., Sato, Y., Kobayashi, Y., Tobita, H., & Kobayashi, M. (2000, April). Interactive textbook and interactive Venn diagram: natural and intuitive interfaces on augmented desk system. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 121–128). ACM.

  • Korbach, A., Ginns, P., Brünken, R., & Park, B. (2020). Should learners use their hands for learning? Results from an eye-tracking study. Journal of Computer Assisted Learning, 36(1), 102–113.

    Article  Google Scholar 

  • Lajoie, S. P., & Lu, J. (2012). Supporting collaboration with technology: Does shared cognition lead to co-regulation in medicine? Metacognition and Learning, 7(1), 45–62.

    Article  Google Scholar 

  • Law, N., Ko, P., Superfine, A. C., Goldman, S. R., & Ko, M. L. M. (2022). Design for multilevel connected learning in pedagogical innovation networks. Teacher learning in changing contexts: Perspectives from the learning sciences. Routledge.

    Google Scholar 

  • Lawrence, L., & Mercier, E. (2019). Co-design of an orchestration tool: Supporting engineering teaching assistants as they facilitate collaborative learning. Interaction Design and Architectures, 42, 111–130.

    Article  Google Scholar 

  • Lawrence L., Shehab, S. Livingston, L., & Margotta, A. (2019) Collaborative teaching sequence. Retrieved September, 2023, from https://www.colearnlab.org/_files/ugd/26a9a4_4aec40e94f79491aac3560722499d89b.pdf.

  • Lee, C. D. (2003). Toward a framework for culturally responsive design in multimedia computer environments: Cultural modeling as a case. Mind, Culture, and Activity, 10(1), 42–61. https://doi.org/10.1207/S15327884MCA1001_05

    Article  Google Scholar 

  • Lee, C. P. (2007). Boundary negotiating artifacts: Unbinding the routine of boundary objects and embracing chaos in collaborative work. Computer Supported Cooperative Work (CSCW), 16(3), 307–339.

    Article  Google Scholar 

  • Lesh, R., & Doerr, H. (2003). Foundations of a model and modeling perspective on mathematics teaching, learning, and problem solving.

  • Lesh, R., & Doerr, H. M. (2012). Alternatives to trajectories and pathways to describe development in modeling and problem solving. In Mathematikunterricht im Kontext von Realität, Kultur und Lehrerprofessionalität (pp. 138–147). Vieweg+ Teubner Verlag.

  • Li, L., Frizell, S., & Yang, Y. (2010). Infusing tablet PCs and interactive learning technology into computer science education to enhance student learning. American Society for Engineering Education.

    Book  Google Scholar 

  • Lundin, J., Lymer, G., Holmquist, L. E., Brown, B., & Rost, M. (2009). Integrating students’ mobile technology in higher education. International Journal of Mobile Learning and Organisation, 4(1), 1–14.

    Article  Google Scholar 

  • Mercier, E., Goldstein, M. H., Baligar, P., & Rajarathinam, R. J. (2023). Collaborative learning in engineering education. In A. Johri (Ed.), International handbook of engineering education. Routledge.

    Google Scholar 

  • Mercier, E., & Higgins, S. (2014). Creating joint representations of collaborative problem solving with multi-touch technology. Journal of Computer Assisted Learning, 30(6), 497–510.

    Article  Google Scholar 

  • Mercier, E., Higgins, S., Burd, E., & Joyce-Gibbons, A. (2012). Multi-touch technology to support multiple levels of collaborative learning in the classroom. In 10th international conference of the learning sciences: The future of learning, ICLS 2012—Proceedings 2 (pp. 187–191).

  • Mercier, E. M., & Higgins, S. E. (2013). Collaborative learning with multi-touch technology: Developing adaptive expertise. Learning and Instruction, 25, 13–23. https://doi.org/10.1016/j.learninstruc.2012.10.004

    Article  Google Scholar 

  • Mercier, E. & Shehab, S. (2018, April) Adaptive expertise in the teaching of collaborative problem solving in undergraduate engineering courses. In S. Athanases (Chair) Adaptive expertise for teaching in academic domains: argumentative writing, literature, collaborative problem-solving, and technology-based inquiry. Presented at the annual meeting of the American Educational Research Association.

  • Mercier, E., Shehab, S., Sun, J., & Capell, N. (2015). The development of collaborative practices in introductory engineering courses. In O. Lindwall, P. Häkkinen, T. Koschmann, P. Tchounikine, & S. Ludvigsen (Eds.), Exploring the material conditions of learning: computer supported collaborative learning (CSCL) conference 2015 (pp. 657–658). The International Society of the Learning Sciences.

    Google Scholar 

  • Mercier, E., Vourloumi, G., & Higgins, S. (2017). Student interactions and the development of ideas in multi-touch and paper-based collaborative mathematical problem solving. British Journal of Educational Technology, 48(1), 162–175. https://doi.org/10.1111/bjet.12351

    Article  Google Scholar 

  • Moore, T. J., Miller, R. L., Lesh, R. A., Stohlmann, M. S., & Kim, Y. R. (2013). Modeling in engineering: The role of representational fluency in students’ conceptual understanding. Journal of Engineering Education, 102(1), 141–178.

    Article  Google Scholar 

  • Nokes-Malach, T. J., Richey, J. E., & Gadgil, S. (2015). When is it better to learn together? Insights from research on collaborative learning. Educational Psychology Review, 27(4), 645–656.

    Article  Google Scholar 

  • OECD. (2017). PISA 2015 results (Volume V): Collaborative problem solving. OECD Publishing. https://doi.org/10.1787/19963777

    Book  Google Scholar 

  • Overdijk, M., & van Diggelen, W. (2008). Appropriation of a shared workspace: Organizing principles and their application. International Journal of Computer-Supported Collaborative Learning, 3, 165–192.

    Article  Google Scholar 

  • Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40(7), 331–337.

    Article  Google Scholar 

  • Penuel, W. R., & Potvin, A. S. (2021). Design-based implementation research to support inquiry learning. International handbook of inquiry and learning, 74–87.

  • Rosengrant, D., Van Heuvelen, A., & Etkina, E. (2009). Do students use and understand free-body diagrams? Physical Review Special Topics-Physics Education Research, 5(1), 010108.

    Article  Google Scholar 

  • Rossing, J. P., Miller, W. M., Cecil, A. K., & Stamper, S. E. (2012). iLearning: The future of higher education? Student perceptions on learning with mobile tablets. Journal of the Scholarship of Teaching and Learning, 12(2), 1–26.

    Google Scholar 

  • Roth, W. M. (2014). The social nature of representational engineering knowledge (pp. 67–82). Cambridge handbook of engineering education research.

    Google Scholar 

  • Sandoval, W. (2014). Conjecture mapping: An approach to systematic educational design research. Journal of the Learning Sciences, 23(1), 18–36.

    Article  Google Scholar 

  • Schwartz, D. L. (1995). The emergence of abstract representations in dyad problem solving. The Journal of the Learning Sciences, 4(3), 321–354.

    Article  Google Scholar 

  • Sharma, G. V. S. S., & Kumar, S. (2023). Thinking through art—A creative insight into mechanical engineering education. Thinking Skills and Creativity, 101341.

  • Shehab, S., & Mercier, E. (2019). Visualizing representations of interaction states during CSCL. In A wide lens: Combining embodied, enactive, extended, and embedded learning in collaborative settings. The International Society of the Learning Sciences.

  • Shehab, S., & Mercier, E. (2020). Exploring the relationship between the types of interactions and progress on a task during collaborative problem solving. In Proceedings of the international conference of the learning sciences (Vol. 3, pp. 1285–1292).

  • Shehab, S., Mercier, E., Kersh, M., Juarez, G. & Zhao, H. (2017). Designing engineering tasks for collaborative problem solving. In B. K. Smith, M. Borge, E. Mercier, & K. Y. Lim (Eds.), Making a difference—prioritizing equity and access in CSCL: The 12th international conference on computer supported collaborative learning. The International Society of the Learning Sciences.

  • Shehab, S. S. (2019). Collaborative problem solving in higher education classrooms: Exploring student interactions, group progress, and the role of the Teacher (order no. 29024028). Available from Dissertations & Theses @ Big Ten Academic Alliance; ProQuest Dissertations & Theses Global. (2634881997). https://www.proquest.com/dissertations-theses/collaborative-problem-solving-higher-education/docview/2634881997/se-2

  • Shuman, L. J., Besterfield-Sacre, M., & McGourty, J. (2005). The ABET “professional skills”—Can they be taught? Can they be assessed? Journal of Engineering Education, 94(1), 41–55.

    Article  Google Scholar 

  • Spradley, J. P. (1979). The ethnographic interview (pp. 7–247). Holt, Rinehart and Winston.

    Google Scholar 

  • Stahl, G. (2013). Theories of cognition in collaborative learning. In The international handbook of collaborative learning (pp. 74–90).

  • Tierney, W. G., & Clemens, R. F. (2011). Qualitative research and public policy: The challenges of relevance and trustworthiness. In Higher education: Handbook of theory and research (pp. 57–83). Springer.

  • Tse, E., Greenberg, S., Shen, C., Forlines, C., & Kodama, R. (2008). Exploring true multi-user multimodal interaction over a digital table. Proceedings of the 7th ACM conference on designing interactive systems—DIS ’08 (pp. 109–118). https://doi.org/10.1145/1394445.1394457

  • Tucker, T., Lawrence, L., & Mercier, E. (2021a). Work in progress: Investigating the effectiveness of an orchestration tool on the nature of students’ collaborative interactions during group work. In 2021 ASEE virtual annual conference content access.

  • Tucker, T., Lawrence, L., & Mercier, E. (2021b). Investigating the effectiveness of an orchestration tool on the nature of students’ collaborative interactions during group work. American Society for Engineering Education.

    Google Scholar 

  • Tucker, T., & Shehab, S., & Mercier, E., & Silva, M. (2019). Work in progress: Evidence-based analysis of the design of collaborative problem-solving engineering tasks. Paper presented at 2019 ASEE Annual Conference & Exposition. https://peer.asee.org/32366

  • Tversky, B. (2015). Keynote address: tools for thinking. In The impact of pen and touch technology on education (pp. 1–4). Springer.

  • Wai, J., Lubinski, D., & Benbow, C. P. (2009). Spatial ability for STEM domains: Aligning over 50 years of cumulative psychological knowledge solidifies its importance. Journal of Educational Psychology, 101(4), 817.

    Article  Google Scholar 

  • Webb, N. M. (2013). Information processing approaches to collaborative learning. In The international handbook of collaborative learning (pp. 19–40). Routledge.

  • Wise, A. F., & Schwarz, B. B. (2017). Visions of CSCL: Eight provocations for the future of the field. International Journal of Computer-Supported Collaborative Learning, 12(4), 423–467.

    Article  Google Scholar 

  • Zaqoot, W., Oh, L. B., Seah, L. H., Koh, E., Zhou, F., Tan, W. K., & Teo, H. H. (2019, December). Representational fluency in education: a literature review and the proposal of a new instrument. In 2019 IEEE International Conference on Engineering, Technology and Education (TALE) (pp. 1–7). IEEE.

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant no. 1441149 and 1628976. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors also thank the members of the CSTEPS team, collaborators in the College of Engineering and all the students who participated in this research.

Author information

Authors and Affiliations

  1. Leadership, Technology, and Workforce Development, College of Education and Human Services, Valdosta State University, 1500 N. Patterson St, Valdosta, GA, 31698, USA

    Jiyoon Jung

  2. Curriculum and Instruction, College of Education, University of Illinois, Urbana-Champaign, Champaign, IL, USA

    Emma Mercier

Authors
  1. Jiyoon Jung
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emma Mercier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiyoon Jung.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Research involving human subjects or animal rights

All research complied with all relevant federal guidelines and institutional policies.

Informed consent

Informed consent for participation and publication of data was obtained from all individual participants for whom identifying information is included in the article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jung, J., Mercier, E. Design-based implementation research: milestones and trade-offs in designing a collaborative representation tool for engineering classrooms. Education Tech Research Dev 71, 2457–2481 (2023). https://doi.org/10.1007/s11423-023-10288-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-023-10288-z

Keywords

Search

Navigation