Advertisement

“I Know It When I See It”: How Experts and Novices Recognize Good Design

  • Kesler TannerEmail author
  • James Landay
Chapter
Part of the Understanding Innovation book series (UNDINNO)

Abstract

Design novices have limited design experience and typically lack the skills or confidence to create good design, however, they may be able to recognize good design. To assess this ability, 53 novice designers and 52 expert designers participated in an online study where they evaluated a series of websites based on aesthetic appeal using two different modes of comparison. Results show that both experts and novices are able to recognize good design and that novices are able to do so almost as well as experts (76.5% accuracy compared to 81.2%). The greatest determinant of whether a participant would correctly identify a higher-rated design was the difference in the two websites’ ground-truth aesthetic ratings. However, expertise and the mode by which the comparison was presented had a significant impact on accuracy (Keep-the-Best = 83.6% and Tournament = 74.1%).

Keywords

Novice designers Comparative design Design subjectivity Design expertise Visual design Web design 

Notes

Acknowledgments

We’d like to thank Jacob O. Wobbrock, Rob Semmens, Kurt Gee and Roy Roy for their assistance with statistics. Any opinions, findings, conclusions or recommendations expressed in this work are those of the authors and do not necessarily reflect those of any supporter listed above.

References

  1. Bogumil, R. J. (1985). The reflective practitioner: How professionals think in action. Proceedings of the IEEE [Internet]. [cited 2017 Sep 15], 73(4), 845–846. Available from: http://ieeexplore.ieee.org/document/1457478/CrossRefGoogle Scholar
  2. Buxton, B. (2010). Sketching user experiences: Getting the design right and the right design [Internet]. [cited 2017 Sep 15]. Available from: https://books.google.com/books?hl=en&lr=&id=2vfPxocmLh0C&oi=fnd&pg=PP1&dq=Sketching+User+Experiences:+Getting+the+Design+Right+and+the+Right+Design&ots=06Kuhjl6VO&sig=5VvNqRGGzjzH0ufiVvL8A8UlZXo
  3. Christiaans, H. H. C. M., & Dorst, K. H. (1992). Cognitive models in industrial design engineering: A protocol study. American Society of Mechanical Engineers Design Engineering Division.Google Scholar
  4. Cross, N. (2004). Expertise in design: An overview. Design Studies. [Internet]. Elsevier; Sep 1 [cited 2017 Sep 15];25(5), 427–441. Available from: http://www.sciencedirect.com/science/article/pii/S0142694X04000316CrossRefGoogle Scholar
  5. Dow, S., & Klemmer, S. R. (2010). The efficacy of prototyping under time constraints. Design Thinking. [Internet]. [cited 2017 Sep 15], pp. 111–128. Available from: http://dl.acm.org/citation.cfm?id=1640260Google Scholar
  6. Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2012). Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. Design Thinking Research Studies. Co-Creation Practice [Internet]. [cited 2017 Sep 15], pp. 127–153. Available from: http://link.springer.com/chapter/10.1007/978-3-642-21643-5_8Google Scholar
  7. Faste, R. (1995). The role of aesthetics in engineering. Japan Society of Mechanical Engineering Journal. [Internet]. [cited 2017 Sep 15]. Available from: http://www.fastefoundation.org/publications/the_role_of_aesthetics.pdf
  8. Faste, H. (2017). Intuition in design: Reflections on the iterative aesthetics of form. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems – CHI EA ’17 [Internet]. [cited 2017 Sep 15], pp. 3403–3413. Available from: http://dl.acm.org/citation.cfm?id=3025534
  9. Hartmann, B., Klemmer, S., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., et al. (2006) Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of 19th Annual ACM Symposium on User interface Software Technology [Internet]. [cited 2017 Sep 15], pp. 299–308. Available from: http://dl.acm.org/citation.cfm?id=1166300
  10. Hassenzahl, M. (2004, December 1). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interactions. [Internet]. L. Erlbaum Associates [cited 2017 Sep 15], 19(4), 319–349. Available from: http://www.tandfonline.com/doi/abs/10.1207/s15327051hci1904_2CrossRefGoogle Scholar
  11. Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., & Cheng, J., et al. (2013). Peer and self assessment in massive online classes. ACM Transactions on Computer-Human Interaction. [Internet]. [cited 2017 Sep 15], 20(6), 1–31. Available from: http://link.springer.com/chapter/10.1007/978-3-319-06823-7_9CrossRefGoogle Scholar
  12. Kurosu, M., & Kashimura, K. (1995). Creativity of. Apparent usability vs. inherent usability: Experimental analysis on the determinants of the apparent usability. In Proceedings of ACM Conference on Human Factors in Computer Systems [Internet]. New York, New York, USA: ACM Press; [cited 2017 Sep 15], 292–293. Available from: http://portal.acm.org/citation.cfm?doid=223355.223680
  13. Lloyd, P., & Scott, P. (1994). Discovering the design problem. Design Studies. [Internet]. [cited 2017 Sep 15], 15(2), 125–140. Available from: http://www.sciencedirect.com/science/article/pii/0142694X94900205CrossRefGoogle Scholar
  14. Luther, K., Pavel, A., Wu, W., Tolentino, J., Agrawala, M., & Hartmann, B., et al. (2014) CrowdCrit: Crowdsourcing and aggregating visual design critique. In Proceedings of Companion Publ. 17th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW Companion ’14 [Internet]. [cited 2017 Sep 15], pp. 21–24. Available from: http://dl.acm.org/citation.cfm?id=2556788
  15. Luther, K., Tolentino, J., Wu, W., Pavel, A., Bailey, B. P., Agrawala, M., et al. (2015) Structuring, aggregating, and evaluating crowdsourced design critique. In Proceedings of 18th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’15 [Internet]. [cited 2017 Sep 15], pp. 473–485. Available from: http://dl.acm.org/citation.cfm?id=2675283
  16. Marks, J., Ruml, W., Ryall, K., Seims, J., Shieber, S., & Andalman, B., et al. (1997). Design galleries: a general approach to setting parameters for computer graphics and animation. In Proceedings of 24th Annual Conference on Computer Graphics Interactive Technology – SIGGRAPH ’97 [Internet]. [cited 2017 Sep 15], pp. 389–400. Available from: http://dl.acm.org/citation.cfm?id=258887
  17. O’Donovan, P., Agarwala, A., & Hertzmann, A. (2015). DesignScape: Design with interactive layout suggestions. In Proceedings of 33rd Annual Conference on Human Factors in Computer Systems – CHI ’15 [Internet]. [cited 2017 Sep 15], pp. 1221–1224. Available from: http://dl.acm.org/citation.cfm?id=2702149
  18. Petitmengin-peugeot, C. (1999). The intuitive experience. Journal of Consciousness Studies. [Internet]. [cited 2017 Sep 15], 2, 43–77. Available from: http://www.ingentaconnect.com/content/imp/jcs/1999/00000006/f0020002/928
  19. Reinecke, K., & Gajos, K. Z. (2014). Quantifying visual preferences around the world. In Proceedings of 32rd Annual ACM Conference on Human Factors in Computer Systems – CHI ’14 [Internet]. ACM Press, New York, USA; [cited 2017 Sep 15], pp. 11–20. Available from: http://dl.acm.org/citation.cfm?doid=2556288.2557052
  20. Salter, A., & Whyte, J. (2004). Serious play: How the world’s best companies simulate to innovate [Internet]. Technovation. [cited 2017 Sep 15], pp. 277–278. Available from: https://books.google.com/books?hl=en&lr=&id=3f6UdmTaAH0C&oi=fnd&pg=PR9&dq=Serious+Play:+How+the+World’s+Best+Companies+Simulate+to+Innovate&ots=RFdJu21jIC&sig=rHiLvl2c91_MH8Zao_nEZmabVhU
  21. Tidwell, J. (2005). Designing interfaces: Patterns for effective interaction design [Internet]. OReilly Media Inc. [cited 2017 Sep 19]. p. 352. Available from: https://books.google.com/books?hl=en&lr=&id=5gvOU9X0fu0C&oi=fnd&pg=PR11&dq=Designing+Interfaces:+Patterns+for+Effective+Interaction+Design&ots=sSZ0N7X9VT&sig=eeDGYaJ91-vKulFUKyqqGI2fhBk
  22. Todi, K., Weir, D., & Oulasvirta, A. (2016). Sketchplore: Sketch and explore with a layout optimiser. In Proceedings of 2016 ACM Conference [Internet]. [cited 2017 Sep 15]; Available from: http://dl.acm.org/citation.cfm?id=2901817
  23. Tohidi, M., Buxton, W., Baecker, R., & Sellen, A. (2006). Getting the right design and the design right: Testing many is better than one. In Proceedings of CHI 2006 Conference on Human Factors on Computer Systems [Internet]. [cited 2017 Sep 15], pp. 1243–1252. Available from: http://dl.acm.org/citation.cfm?id=1124960
  24. Van Duyne, D. K., Landay, J. A., & Hong, J. I. (2007). The design of sites: Patterns for creating winning web sites [Internet]. [cited 2017 Sep 19]. Available from: https://books.google.com/books?hl=en&lr=&id=eE2TxLtDsL8C&oi=fnd&pg=PR13&dq=The+Design+of+Sites:+Patterns+for+Creating+Winning+Web+Sites&ots=v9JzHZPmCj&sig=FwQSMbSU8bI_Nno-Gu0-1Wgh9uc
  25. Willett, W., Heer, J., & Agrawala, M. (2012). Strategies for crowdsourcing social data analysis. In Proceedings of 2012 ACM Annual Conference on Human Factors on Computer Systems – CHI ’12 [Internet]. [cited 2017 Sep 15], p. 227. Available from: http://dl.acm.org/citation.cfm?id=2207709
  26. Xu, A., & Bailey, B. P. (2012) What do you think? A case study of benefit, expectation, and interaction in a large online critique community. In Proceedings of 15th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW’12 [Internet]. [cited 2017 Sep 15], 295. Available from: http://dl.acm.org/citation.cfm?id=2145252
  27. Xu, A., Huang, S.-W., & Bailey, B. (2014). Voyant: generating structured feedback on visual designs using a crowd of non-experts. In Proceedings of 17th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’14 [Internet]. [cited 2017 Sep 15], pp. 1433–1444. Available from: http://dl.acm.org/citation.cfm?id=2531604
  28. Xu, A., Rao, H., Dow, S. P., & Bailey, B. P. (2015). A classroom study of using crowd feedback in the iterative design process. In Proceedings of 18th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’15 [Internet]. [cited 2017 Sep 15], pp. 1637–1648. Available from: http://dl.acm.org/citation.cfm?id=2675140
  29. Yuan, A., Luther, K., Krause, M., Vennix, S. I., Dow, S. P., & Hartmann, B. (2016). Almost an expert: The effects of rubrics and expertise on perceived value of crowdsourced design critiques. In Proceedings of 19th ACM Conference on Computer Supported Cooperative Work and Social Computing – CSCW ’16 [Internet]. [cited 2017 Sep 15], pp. 1003–1015. Available from: http://dl.acm.org/citation.cfm?id=2819953

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Stanford UniversityStanfordUSA

Personalised recommendations