Skip to main content
SpringerLink
Log in

Recent Findings About the Novelty Metric of Shah

  • Conference paper
  • First Online:
Design Tools and Methods in Industrial Engineering II (ADM 2021)

Part of the book series: Lecture Notes in Mechanical Engineering ((LNME))

  • 2374 Accesses

Abstract

The work presented in this paper aims at presenting the last findings about one of the most acknowledged and used novelty metrics in the field of engineering design. The a-posteriori novelty metric of Shah, Vargas-Hernandez and Smith is often considered in design research, but after two decades, there are still some critical issues to be solved. This paper highlights the issues identified in the last research works, by referring to the few published works that directly face the argument. It emerged that notwithstanding the presence of metric variants proposed by scholars, none of them is capable to overcome the problems. In particular, the assessment of set of ideas implementing heterogeneous numbers of attributes can lead to misleading results. In absence of a comprehensive solution, this paper provides a list of practical hints in order to allow the application of the metric (or the related variants) and ensuring the repeatability of the experiments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amabile, T.M.: The Social Psychology of Creativity. Springer-Verlag, New York (1983). https://doi.org/10.1080/02640828708723925.

  2. Moreno, D.P., Blessing, L.T., Yang, M.C., Hernández, A.A., Wood, K.L.: Overcoming design fixation: design by analogy studies and nonintuitive findings. Artif. Intell Eng. Des. Anal. Manuf. AIEDAM. 30, 185–199 (2016). https://doi.org/10.1017/S0890060416000068

    Article  Google Scholar 

  3. Sarkar, P., Chakrabarti, A.: Studying engineering design creativity - developing a Common Definition and Associated Measures. In: Gero, J.S. (ed.) NSF International Workshop on Studying Design Creativity 2008, Aix-en-Provence (2008)

    Google Scholar 

  4. Daly, S.R., Mosyjowski, E.A., Seifert, C.M.: Teaching creativity in engineering courses. J. Eng. Educ. 103, 417–449 (2014). https://doi.org/10.1002/jee.20048

    Article  Google Scholar 

  5. Shah, J.J., Vargas-Hernandez, N., Smith, S.M.: Metrics for measuring ideation effectiveness. Des. Stud. 24, 111–134 (2003). https://doi.org/10.1016/S0142-694X(02)00034-0

    Article  Google Scholar 

  6. Jansson, D.G., Smith, S.M.: Design fixation. Des. Stud. 12, 3–11 (1991). https://doi.org/10.1016/0142-694X(91)90003-F

    Article  Google Scholar 

  7. Linsey, J.S., Clauss, E.F., Kurtoglu, T., Murphy, J.T., Wood, K.L., Markman, A.B.: An experimental study of group idea generation techniques: understanding the roles of idea representation and viewing methods. J. Mech. Des. 133(3), 1–15 (2011). https://doi.org/10.1115/1.4003498

    Article  Google Scholar 

  8. Vargas-Hernandez, N., Schmidt, L.C., Okudan, G.E.: Systematic ideation effectiveness study of TRIZ. J. Mech. Des. 135, 1 (2013). https://doi.org/10.1115/1.4024976

    Article  Google Scholar 

  9. Fiorineschi, L., Frillici, F.S., Rotini, F., Tomassini, M.: Exploiting TRIZ tools for enhancing systematic conceptual design activities. J. Eng. Des. 29, 259–290 (2018). https://doi.org/10.1080/09544828.2018.1473558

    Article  Google Scholar 

  10. Brown, D.C.: Problems with the calculation of novelty metrics. In: Proceedings of the 6th International Conference on Design Computing and Cognition (2014)

    Google Scholar 

  11. Eckert, C., Alink, T., Ruckpaul, A.: Different notions of function : results from an experiment on the analysis of an existing product. J. Eng. Des. 22, 811–837 (2011). https://doi.org/10.1080/09544828.2011.603297

  12. Vermaas, P.E., Eckert, C.: My functional description is better! Artif. Intell. Eng. Des. Anal. Manuf. 27, 187–190 (2013). https://doi.org/10.1017/S089006041300019X

    Article  Google Scholar 

  13. Fiorineschi, L., Frillici, F.S., Rotini, F.: Subjectivity of novelty metrics based on idea decomposition. Int. J. Des. Creativity Innov. 8, 1–17 (2020). https://doi.org/10.1080/21650349.2020.1811775

    Article  Google Scholar 

  14. Sluis-Thiescheffer, W., Bekker, T., Eggen, B., Vermeeren, A., De Ridder, H.: Measuring and comparing novelty for design solutions generated by young children through different design methods. Des. Stud. 43, 48–73 (2016). https://doi.org/10.1016/j.destud.2016.01.001

    Article  Google Scholar 

  15. Fiorineschi, L., Frillici, F.S., Rotini, F.: Issues related to missing attributes in a-posteriori novelty assessments. In: International Design Conference - Design 2018, Dubrovnik, Croatia, pp. 1067–1078 (2018)

    Google Scholar 

  16. Fiorineschi, L., Frillici, F.S., Rotini, F.: Impact of missing attributes on a posteriori novelty assessments. Res. Eng. Design 31(2), 221–234 (2020). https://doi.org/10.1007/s00163-020-00332-x

    Article  Google Scholar 

  17. Peeters, J., Verhaegen, P.A., Vandevenne, D., Duflou, J.R.: Refined metrics for measuring novelty in ideation. In: Proceedings of IDMME - Virtual Concept 2010, Bordeaux, France, pp. 1–4 (2010)

    Google Scholar 

  18. Johnson, T.A., Caldwell, B.W., Cheeley, A., Green, M.G.: Comparison and extension of novelty metrics for problem-solving tasks. In: Proceedings of the ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC/CIE 2016, Charlotte (2016). https://doi.org/10.1115/DETC2016-60319

  19. Vargas-Hernandez, N., Okudan, G.E., Schmidt, L.C.: Effectiveness metrics for ideation: merging genealogy trees and improving novelty metric. In: Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2012, Chicago (2012). https://doi.org/10.1115/DETC2012-70295

  20. Vargas-Hernandez, N., Schmidt, L.C., Okudan, G.E.: Systematic ideation effectiveness study of TRIZ. In: Proceedings of the ASME 2012 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference IDETC/CIE 2012, pp. 1–10 (2012)

    Google Scholar 

  21. Pahl, G., Beitz, W., Feldhusen, J., Grote, K.H.: Engineering Design, 3rd ed. Springer-Verlag, London (2007). https://doi.org/10.1007/978-1-84628-319-2

  22. Fiorineschi, L., Rotini, F.: A-posteriori novelty metrics based on idea decomposition. Int. J. Des. Sci. Technol. 23, 187–209 (2019)

    Google Scholar 

  23. Nelson, B.A., Wilson, J.O., Rosen, D., Yen, J.: Refined metrics for measuring ideation effectiveness. Des. Stud. 30, 737–743 (2009). https://doi.org/10.1016/j.destud.2009.07.002

    Article  Google Scholar 

  24. Cohen, J.: Weighted kappa: Nominal scale agreement with pro-vision for scaled disagreement or partial credit. Psychol. Bull. 70, 213–220 (1968)

    Article  Google Scholar 

  25. Hayes, A.F., Krippendorff, K.: Answering the call for a standard reliability measure for coding data. Commun. Methods Meas. 1, 77–89 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial Engineering, University of Florence, Via Santa Marta 3, 50139, Firenze, Italy

    Lorenzo Fiorineschi, Francesco Saverio Frillici, Federico Rotini & Monica Carfagni

Authors
  1. Lorenzo Fiorineschi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Saverio Frillici
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Federico Rotini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Monica Carfagni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lorenzo Fiorineschi .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria Gestionale, dell’Informazione e della Produzione, Univesita Degli Studi di Bergamo, Dalmine, Bergamo, Italy

    Caterina Rizzi

  2. Dipartimento di Ingegneria Meccanica e Aerospaziale, Sapienza Università di Roma, Rome, Italy

    Francesca Campana

  3. Dipartimento di Ingegneria Meccanica e Aerospaziale, Sapienza Università di Roma, Roma, Italy

    Michele Bici

  4. Dipartimento di Ingegneria "Enzo Ferrari", Università di Modena e Reggio Emilia, Modena, Italy

    Francesco Gherardini

  5. Dipartimento di Ingegneria, Università degli Studi di Palermo, Palermo, Italy

    Tommaso Ingrassia

  6. Dipartimento di Ingegneria, Università degli Studi Roma Tre, Roma, Italy

    Paolo Cicconi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fiorineschi, L., Frillici, F.S., Rotini, F., Carfagni, M. (2022). Recent Findings About the Novelty Metric of Shah. In: Rizzi, C., Campana, F., Bici, M., Gherardini, F., Ingrassia, T., Cicconi, P. (eds) Design Tools and Methods in Industrial Engineering II. ADM 2021. Lecture Notes in Mechanical Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-91234-5_71

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-91234-5_71

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-91233-8

  • Online ISBN: 978-3-030-91234-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation