Skip to main content
SpringerLink
Log in

A delphi consensus checklist for assessing arts design: a case for miniature robots in a STEAM contest

  • Published:
International Journal of Technology and Design Education Aims and scope Submit manuscript

Abstract

Arts design has been proven to enable designers to have a meaningful learning of STEAM, but research on a checklist for assessing Arts design based on evaluating entries in a STEAM contest is still rare. Therefore, this study developed a checklist for assessing Arts design for evaluating miniature robots in a STEAM contest. Using an example of STEAM contest, named PowerTech, which has been held 22 years in Taiwan, we developed five dimensions of the checklist are: aesthetics, material usage, bionic, performance, and additional devices. There are two sub-indices in each aspect to be considered as the identification of form characteristics. Eight international experts were invited to review the content validity of the Arts design scale, and 30 completed checklists were used to test the reliability and validity though Kendall’s coefficient of concordance. The results showed that the Arts design scale had high reliability and good validity, and so rubric index could be used to measure and assess Arts design with miniature robots in that integrated STEAM contest. Based on this, it is suggested that educational authorities can refer this study and for encourage students to create Arts in any integrated STEAM contests.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Albay, E.M., & Eisma, D.V. (2021). Performance task assessment supported by the design thinking process: Results from a true experimental research. Social Sciences & Humanities Open, 3, 100116.

  • Alfakhri, D., Harness, D., Nicholson, J., & Harness, T. (2018). The role of aesthetics and design in hotelscape: A phenomenological investigation of cosmopolitan consumers. Journal of Business Research, 85, 523–531.

  • Anderson, K., Kachorsky, D.(2019). Assessing students’ multimodal compositions: an analysis of the literature. English Teaching: Practice & Critique, 18(3), 312–334.

  • Ball, L. J., Christensen, B. T., & Halskov, K. (2021). Sticky notes as a kind of design material: how sticky notes support design cognition and design collaboration. Design Studies, 76, 101034.

  • Barlex, D. (2004). Creativity in school technology education: A chorus of voices. In H. Middleton, M. Pavlova, & D. Roebuck (Eds.), Learning for innovation in technology education (pp. 24–37). Brisbane, Australia: Centre for Learning Research, Griffith University.

  • Bequette, J. W., & Bequette, M. B. (2012). A place for art and design education in the STEM conversation. Art Education, 65(2), 40–47.

  • Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400–413.

  • Cai, H., He, W. and Zhang, D. (2003). A semantic style driving method for products’ appearance design. Journal of Materials Processing Technology, 139(1–3), 233–236.

  • Chapelle, F. and Bidaud, P. (2006). Evaluation functions synthesis for optimal design of hyper-redundant robotic systems. Mechanism and Machine Theory, 41(10), 1196–1212.

  • Clayton, M. J. (1997). Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educational Psychology: 17(4), 373–386.

  • Coates, D. (2003). Watches tell more than time: Product design, information and the quest for elegance London: McGraw-Hill.

  • Coburn, A., Vartanian, O., & Chatterjee, A. (2017). Buildings, Beauty, and the Brain: a Neuroscience of Architectural Experience. Journal of Cognitive Neuroscience, 29(9), 1521–1531.

  • Crilly, N., Moultrie, J., & Clarkson, P. J. (2004). Seeing things: consumer response to the visual domain in product design. Design Studies, 25(6), 547–577.

  • Daisy, F., Andrew, S., & Dorina, C. (2018). Cultural engagement and cognitive reserve: museum attendance and dementia incidence over a 10-year period. The British Journal of Psychiatry, 213, 1–3.

  • Davis, L. L. (1992). Instrument review: Getting the most from a panel of experts. Applied Nursing Research,5(4), 194–197.

  • Fackrell, K.Smith, H., Colley, V. Thacker, B. Horobin, A. Haider, H.F. et al.(2017). Core outcome domains for early phase clinical trials of sound-, psychology-, and pharmacology-based interventions to manage chronic subjective tinnitus in adults: the COMIT’ID study protocol for using a Delphi process and face-to-face meetings to establish consensus.BioMed Research International.18 (1) (2017), 1–11.

  • Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research, and applications Cambridge: MIT Press.

  • Fuchs, S., & Schmidt, K. D. (2021). On order statistics and Kendall’s tau. Statistics and Probability Letters, 169, 108972.

  • Gerrisk, K., & Lathlean, J. (2015). The Research process in nursing (7th ed) Wiley Blackwell.

  • Gonzalez-Zugasti, J., & Otto, K. (2000). A method for architecting product platform with an application to interplanetary mission design. Researches in Engineering Design, 12, 61–72.

  • Hohl, L., Tellez, R., Michel, O., & Ijspeert, A. J. (2006). Aibo and Webots: Simulation, wireless remote control and controller transfer. Robotics and Automatic Systems, 54(6), 472–485.

  • Hong, J.C., Lin, C. L., and Lin,Y. L. (2007). Operating a successful PowerTech creativity contest. Journal of Technology Studies, 33(1), 25–31.

  • Hong, J. C., Ye, J., & Fan, J. (2019). STEM in fashion design: The roles of creative self-efficacy and epistemic curiosity in creative performance. Eurasia Journal of Mathematics, Science and Technology Education 15(9), em1742.

  • Hong, J. C., Chen, M. L., Wang, C. M., Ye, J. N., & Ye, J. H. (2020). Relationship between the urban and rural students’ cooperative attitude, creative task engagements and competition value in participating a STEAM co-creation contest. International Journal of Information and Education Technology, 10(12), 873–881.

  • Hong, J. C., Tsai, C. R., & Tai, K. H. (2021). iSTEAM contest on enhancing self-confidence in making miniature models: Correlate to mastery orientation, engagement and interest. Research in Science and Technological Educationhttp://dx.doi.org/10.1080/02635143.2021.1909554

  • Hong, J. C., Tsai, C. R., & Tai, K. H. (in press). iSTEAM contest on enhancing self-confidence in making miniature models: Correlate to mastery orientation, engagement and interest. Research in Science and Technological Education, http://dx.doi.org/10.1080/02635143.2021.1909554

  • Hughes, J. (2017). Digital making with “At-risk” youth.” The International Journal of Information and Learning Technology, 34(2), 102–113.

  • Hu, Y., Ren, Z. Z., Du, X., Lan, L., Yu, W., & Yang, S. (2021). The shifting patterns based on six thinking hats and its relationship with design creativity. Thinking Skills and Creativity. 42, 321–334.

  • Imbler, A. C., Clark, S. K., Young, T. A., & Feinauer, E. (in press). Teaching second-grade students to write science expository text: Does a holistic or analytic instrument provide more meaningful results? Assessing Writing.

  • Isaksen, S. G., Lauer, K. J., Ekvall, G., & Britz, A. (2001). Perceptions of the best and worst climates for creativity: preliminary validation evidence for the situational outlook questionnaire. Creativity Research Journal, 13(2), 171–184.

  • Jiang, Z. H. & Yan, J. Q. (2003). Research and development on constraint-based product family design and assembly simulation. Journal of Materials Processing Technology, 139 (1–3), 257–262.

  • Keeney, S., Hasson, F., & Mckenna, H. (2006). Consulting the oracle: Ten lessons from using the delphi technique in nursing research. Journal of Advanced Nursing, 53(2), 25–29.

  • Kezar, A., & Maxey, D. (2016). The Delphi technique: an untapped approach of participatory research. International Journal of Social Research Methodology, 19(2), 143–160.

  • Krasnow, D., & Chatfield, S. J. (2009). Development of the ‘performance competence evaluation measure’ assessing qualitative aspects of dance performance. Journal of Dance Medicine & Science, 13, 101–107.

  • Kost, G. G. & Zdanowicz, R. (2005). Modeling of manufacturing systems and robot motions. Journal of Materials Processing Technology, 164–165, 1369–1378.

  • Lena, J. (2019). Entitled: Discriminating tastes and the expansion of the arts. Princeton: Princeton UP.

  • Liu, C.-Y., & Wu, C.-J. (2022). STEM without art: A ship without a sail. Thinking Skills and Creativity, 43, 100977.

  • Martinez, S. L., & Stager, G. (2013). Invent to learn: Making, tinkering, and engineering in the classroom New York, NY: Constructing Modern Knowledge Press.

  • Mckenna, H., Keeney, S., & Hasson, F. (2011). The Delphi Technique in nursing and health research John Wiley & Sons.

  • Medeiros, K.E., Steele, L. M., Watts, L. L., & Mumford, M. D. (2018). Timing is everything: Examining the role of constraints throughout the creative process. Psychology of Aesthetics, Creativity, and the Arts, 12(4), 471–488.

  • Menéndez-Varela, J.-L., & Gregori-Giralt, E. (2018). instruments for developing students’ professional judgement: A study of sustainable assessment in arts education. Studies in Educational Evaluation, 58, 70–79.

  • Mostyn, V. and Skarupa, J (2004). Improving mechanical model accuracy for simulation purposes. Mechtronics, 14(7), 777–787.

  • Mourtos,N.J. (2012).Defining, teaching, and assessing engineering design skills. Industrial Engineering: Concepts, Methodologies, Tools, and Applications, 1, 1–13.

  • Olawumi, T. O., & Chan, D. W. (2018). Identifying and prioritizing the benefits of integrating BIM and sustainability practices in construction projects: A Delphi survey of international experts. Sustainable Cities and Society, 40, 16–27.

  • Park, H., Byun, S., Sim, J., Han, H., & Baek, Y. S. (2016). Teachers’ perceptions and practices of STEAM education in South Korea. Eurasia Journal of Mathematics, Science and Technology Education, 12(7), 1739–1753.

  • Peng, H., Hu, J., Wang, H., Ren, H., Sun, C., Hu, H., & Li, J. (2021). Multiple visual feature integration based automatic aesthetics evaluation of robotic dance motions. Information, 12 (3), 95.

  • Pirzadeh, P., Lingard, H., & Blismas, N. (2020). Effective communication in the context of safe design decision making. Safety Science, 131(1), 10–21.

  • Rakovic, M., Bernacki, M. L., Greene, J. A., Plumley, R. D., Hogan, K.A., Gates, K. M., & Panter, A. T. (2022). Examining the critical role of evaluation and adaptation in self-regulated learning. Contemporary Educational Psychology, 68, 102027.

  • Rieger, K. L., Chernomas, W. M., McMillan, D. E., & Morin, F. L. (2020). Navigating creativity within arts-based pedagogy: Implications of a constructivist grounded theory study. Nurse Education Today, 91, 104465.

  • Rosso, B. D. (2014). Creativity and constraint: Exploring the role of constraint in the creative processes of new product and technology development teams. Organization Studies, 35(4), 551–585.

  • Rowe, G., & Wright, G. (2001). Expert opinions in forecasting: The R of the Delphi Technique. In J. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners (pp. 125–144). Boston: Kluwer.

  • Samuel A.A., AmarBennadji. B, Firdaus Muhammad-Sukki. C., Nazmi Sellamia. C. (2021). Myth or gold? the power of aesthetics in the adoption of building integrated photovoltaics (bipvs). Energy Nexus, 4

  • Stefano, Mastandrea, Sabrina, Fagioli, Valeria, & Biasi. (2019). Art and psychological well-being: linking the brain to the aesthetic emotion. Frontiers in psychology,739

  • Thalamy, P., Piranda, B., Naz, A., & Bourgeois. J. (2021). VisibleSim: A behavioral simulation framework for lattice modular robots. Robotics and Autonomous Systems.147, 34–56.

  • Tromp, C., & Baer, J. (2022). Creativity from constraints: Theory and applications to education. Thinking Skills and Creativity, 46, 101184.

  • Vaajakallio, K., & MattelmäkiKi, T. (2014). Design games in codesign: as a tool, a mindset and a structure. International Journal of CoCreation in Design and the Arts 10(1), 63–77.

  • Vanbelle, S. (2016). A new interpretation of the weighted kappa coefficients. Psychometrika, 81(2), 399–410.

  • Veryzer, R.W. J. (1993). Aesthetic response and the influence of design principles on product preferences. Advances in Consumer Research, 20, 224–228.

  • Vernon,W. (2009). The Delphi technique: A review. International Journal of Therapy and Rehabilitation, 16(2), 1759-1779X.

  • Viera, A. J., & Garrett, J. M. (2005). Understanding interobserver agreement: the kappa statistic. Family Medicine, 37(5), 360-3.

  • Wang, Y., Yu, S., Ma, N., Wang, J., Hu, Z., Liu, Z., & He, J. (2020). Prediction of product design decision Making: An investigation of eye movements and EEG features. Advanced Engineering Informatics.45,101095.

  • Wohl, H. (2021). Bound by creativity: How contemporary art is created and judged. Chicago: University of Chicago Press.

  • Wu, Z. (2022). Understanding teachers’ cross-disciplinary collaboration for STEAM education: Building a digital community of practice. Thinking Skills and Creativity, 46, 101178.

  • Zhang, T., & Cheng, C. (2020). Understanding of Kendall’s Coefficient of Concordance. Statistics and Application, 9(4), 578–581.

Download references

Author information

Authors and Affiliations

  1. Normal School of Vocational and Techniques, Hubei University of Technology, Wuhan, China

    Qiuyue Yang

  2. Institute for Research Excellence in Learning Sciences, National Taiwan Normal University, Taiwan, China

    Jon-Chao Hong

  3. School of Education Science, Nanjing Normal University, Nanjing, China

    Jianjun Gu

  4. No. 122 Ninghai Road, Gulou District, Nanjing City, Jiangsu Province, China

    Jianjun Gu

Authors
  1. Qiuyue Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jon-Chao Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianjun Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The first author conducted the research; the second and third authors analyzed the data; all authors wrote the paper; all authors have approved the final version.

Corresponding author

Correspondence to Jianjun Gu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Q., Hong, JC. & Gu, J. A delphi consensus checklist for assessing arts design: a case for miniature robots in a STEAM contest. Int J Technol Des Educ 34, 249–265 (2024). https://doi.org/10.1007/s10798-023-09823-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10798-023-09823-z

Keywords

Search

Navigation