Skip to main content

Improving Usability with Think Aloud and Focus Group Methods. A Case Study: An Intelligent Police Patrolling System (I-Pat)

  • Conference paper
  • First Online:
Advances in Human Factors and Systems Interaction (AHFE 2019)


This study proposes the use of Think Aloud and Focus Group methods for evaluating the usability of the Intelligent Police Patrolling System (I-Pat). The study was conducted with twenty-one students of computer engineering. The study included two evaluation methods. First, the application of Think Aloud using audio recordings, image capturing and questionnaires. Second, the application of a Focus Group for brainstorming. The total number of the usability problems identified was fifteen. Comprehensiveness (46%) and layout (43%) problems were the most frequently found. The study showed that the problems encountered were due to the lack of understanding of the system’s functions, so it is recommended increasing the users’ levels of knowledge about the system. The application of these methods caused the students to find a greater number of errors than when applying a single method, allowing them to generate a report with usability improvements according to the reported errors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. ISO Standards: 9126 standard (2018).

  2. ISTQB: International software testing qualifications board, advanced level syllabus usability tester (2016).

  3. Brhel, M., Meth, H., Maedche, A., Werder, K.: Exploring principles of user-centered agile software development: a literature review. Inf. Softw. Technol. 61, 163–181 (2015)

    Article  Google Scholar 

  4. Pilco, H., Sanchez-Gordon, S., Calle-Jimenez, T., Rybarczyk, Y., Jadán, J., Villarreal, S., Esparza, W., Acosta-Vargas, P., Guevara, C., Nunes, I.L.: Analysis and improvement of the usability of a tele-rehabilitation platform for hip surgery patients. In: International Conference on Applied Human Factors and Ergonomics, pp. 197–209. Springer (2018)

    Google Scholar 

  5. Maniega-Legarda, D.: Aplicación de criterios de usabilidad en sitios web: consejos y pautas para una correcta interpretación. Observatorio TIC: REBIUN Red de Bibliotecas Universitarias (2006)

    Google Scholar 

  6. User Experience Professionals’ Association: The usability body of knowledge (2018).

  7. Guan, Z., Lee, S., Cuddihy, E., Ramey, J.: The validity of the stimulated retrospective think-aloud method as measured by eye tracking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1253–1262. ACM (2006)

    Google Scholar 

  8. Olmsted-Hawala, E.L., Murphy, E.D., Hawala, S., Ashenfelter, K.T.: Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data dissemination web sites for usability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2381–2390. ACM (2010)

    Google Scholar 

  9. McDonald, S., McGarry, K., Willis, L.M.: Thinking-aloud about web navigation: the relationship between think-aloud instructions, task difficulty and performance. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 57, pp. 2037–2041. SAGE Publications, Los Angeles (2013)

    Google Scholar 

  10. Hertzum, M., Hansen, K.D., Andersen, H.H.: Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload? Behav. Inf. Technol. 28(2), 165–181 (2009)

    Article  Google Scholar 

  11. Barnum, C.M.: Usability testing essentials: ready, set… test!. Elsevier, Amsterdam (2010)

    Google Scholar 

  12. Van Den Haak, M.J., De Jong, M.D., Schellens, P.J.: Constructive interaction: an analysis of verbal interaction in a usability setting. IEEE Trans. Prof. Commun. 49(4), 311–324 (2006)

    Article  Google Scholar 

  13. Makri, S., Blandford, A., Cox, A.L.: This is what im doing and why: methodological reflections on a naturalistic think-aloud study of interactive information behaviour. Inf. Process. Manage. 47(3), 336–348 (2011)

    Article  Google Scholar 

  14. Nørgaard, M., Hornbæk, K.: What do usability evaluators do in practice? An explorative study of think-aloud testing. In: Proceedings of the 6th Conference on Designing Interactive Systems, pp. 209–218. ACM (2006)

    Google Scholar 

  15. Molich, R., Ede, M.R., Kaasgaard, K., Karyukin, B.: Comparative usability evaluation. Behav. Inf. Technol. 23(1), 65–74 (2004)

    Article  Google Scholar 

  16. Gulliksen, J., Boivie, I., Persson, J., Hektor, A., Herulf, L.: Making a difference: a survey of the usability profession in Sweden. In: Proceedings of the Third Nordic-Conference on Human-Computer Interaction, pp. 207–215. ACM (2004)

    Google Scholar 

  17. Nyumba, T.O., Wilson, K., Derrick, C.J., Mukherjee, N.: The use of focus group discussion methodology: Insights from two decades of application in conservation. Methods Ecol. Evol. 9(1), 20–32 (2018)

    Article  Google Scholar 

  18. Plummer-D’Amato, P.: Focus group methodology part 2: considerations for analysis. Int. J. Ther. Rehabil. 15(3), 123–129 (2008)

    Article  Google Scholar 

  19. Onwuegbuzie, A.J., Dickinson, W.B., Leech, N.L., Zoran, A.G.: A qualitative framework for collecting and analyzing data in focus group research. Int. J. Qual. Methods 8(3), 1–21 (2009)

    Article  Google Scholar 

  20. Kontio, J., Lehtola, L., Bragge, J.: Using the focus group method in software engineering: obtaining practitioner and user experiences. In: Proceedings. 2004 International Symposium on Empirical Software Engineering, ISESE 2004, pp. 271–280. IEEE (2004)

    Google Scholar 

  21. Sunikka, A.: Usability evaluation of the helsinki school of economics website. Master’s thesis (2004)

    Google Scholar 

  22. Nielsen, J.: The use and misuse of focus groups. IEEE Softw. 14(1), 94–95 (1997)

    Article  Google Scholar 

  23. Obeso, M.E.A., Lovelle, J.M.C., Prieto, A.B.M.: Metodología de medición y evaluación de la usabilidad en sitios web educativos. Universidad de Oviedo (2005)

    Google Scholar 

  24. Guevara, C., Jadan, J., Zapata, C., Martínez, L., Pozo, J., Manjarres, E.: Model of dynamic routes for intelligent police patrolling. In: Multidisciplinary Digital Publishing Institute Proceedings, vol. 2, p. 1214 (2018)

    Google Scholar 

  25. Habibi, M.R.M., Khajouei, R., Eslami, S., Jangi, M., Ghalibaf, A.K., Zangouei, S., et al.: Usability testing of bed information management system: a think-aloud method. J. Adv. Pharm. Technol. Res. 9(4), 153 (2018)

    Article  Google Scholar 

  26. Harper, B.D., Norman, K.L.: Improving user satisfaction: the questionnaire for user interaction satisfaction version 5.5. In: Proceedings of the 1st Annual Mid-Atlantic Human Factors Conference, pp. 224–228 (1993)

    Google Scholar 

  27. Arh, T., Blazic, B.J.: A case study of usability testing-the sumi evaluation approach of the educanext portal. WSEAS Trans. Inf. Sci. Appl. 5(2), 175–181 (2008)

    Google Scholar 

  28. Brooke, J., et al.: Sus-a quick and dirty usability scale. Usabil. Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  29. Nielsen, J.: Severity ratings for usability problems. Pap. Essays 54, 1–2 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Tania Calle-Jimenez .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Calle-Jimenez, T. et al. (2020). Improving Usability with Think Aloud and Focus Group Methods. A Case Study: An Intelligent Police Patrolling System (I-Pat). In: Nunes, I. (eds) Advances in Human Factors and Systems Interaction. AHFE 2019. Advances in Intelligent Systems and Computing, vol 959. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-20039-8

  • Online ISBN: 978-3-030-20040-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics