Developing Instrument for Handset Usability Evaluation: A Survey Study

  • Ting Zhang
  • Pei-Luen Patrick Rau
  • Gavriel Salvendy
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4550)


Handset is transforming from a traditional cellular phone to an integrated content delivery platform for communications, entertainment and commerce. Their increasing capabilities and value-added features provide more utilities, and at the same time, make the design more complicated and the device more difficult to use. An online survey was conducted to measure user’s perspective of the usability level of their current handset using a psychometric type of instrument. A total of 9 usability factors were derived from the results of exploratory factor analysis. The total percentage variance explained by these 9 factors of the overall variance of the data was 65.20%. The average internal consistency in this study is 0.70.


Handset Usability Usability measurements Usability factors Instrument Survey 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Han, S.H., et al.: Evaluation of product usability: development and validation of usability dimensions and design elements based on empirical models. International Journal of Industrial Ergonomics 26(4), 477–488 (2000)CrossRefGoogle Scholar
  2. 2.
    Davis, F.D.: Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13(3), 318–340 (1989)CrossRefGoogle Scholar
  3. 3.
    Han, S.H., et al.: Usability of consumer electronic products. International Journal of Industrial Ergonomics 28(3-4), 143–151 (2001)CrossRefGoogle Scholar
  4. 4.
    Jordan, P.W.: Human factors for pleasure in product use. Applied Ergonomics 29(1), 25–33 (1998)CrossRefGoogle Scholar
  5. 5.
    Jokela, T., et al.: The standard of user-centered design and the standard definition of usability: Analyzing ISO 13407 against ISO 9241-11. In: de Janeiro, R. (ed.) Proceedings of the Latin American conference on Human-Computer Interaction, Brazil, ACM Press, New York (2003)Google Scholar
  6. 6.
    Zhang, D., Adipat, B.: Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction 18(3), 293–308 (2005)CrossRefGoogle Scholar
  7. 7.
    Kjeldskov, J., Graham, C.: A review of mobile HCI research methods. In: Chittaro, L. (ed.) Mobile HCI 2003. LNCS, vol. 2795, pp. 317–335. Springer, Heidelberg (2003)Google Scholar
  8. 8.
    Scholtz, J.: Usability evaluation (2004) (cited 2006 October 27th) Available from
  9. 9.
    van Schaik, P., Ling, J.: Five psychometric scales for online measurement of the quality of human-computer interaction in Web sites. International Journal of Human-Computer Interaction 18(3), 309–322 (2005)CrossRefGoogle Scholar
  10. 10.
    Lee, Y.S., et al.: Systematic evaluation methodology for cell phone user interfaces. Interacting with Computers 18(2), 304–325 (2006)CrossRefGoogle Scholar
  11. 11.
    Klockar, T., et al.: Usability of mobile phones. In: Proceedings of the 19th International Symposium on Human Factors in Telecommunications, Berlin, Germany (2003)Google Scholar
  12. 12.
    Ziefle, M., Bay, S., Schwade, A.: On keys’ meanings and modes: The impact of different key solutions on children’s efficiency using a mobile phone. Behaviour & Information Technology 25(5), 413–431 (2006)CrossRefGoogle Scholar
  13. 13.
    Bederson, B.B., et al.: A fisheye calendar interface for PDAs: Providing overviews for small displays. In: Proceedings of CHI 2003 Conference on Human Factors in Computing Systems, ACM Press, Ft. Lauderdale, Florida, USA (2003)Google Scholar
  14. 14.
    Bederson, B.B., et al.: DateLens: a fisheye calendar interface for PDAs. ACM Transactions on Computer-Human Interaction (TOCHI) 11(1), 90–119 (2004)CrossRefGoogle Scholar
  15. 15.
    Bloom, L. et al.: Investigating the relationship between battery life and user acceptance of dynamic, energy-aware interfaces on handhelds. In: Mobile Human-Computer Interaction - Mobilehci, Proceedings. pp. 13–24 (2004)Google Scholar
  16. 16.
    Buchanan, G., et al.: Improving mobile internet usability. In: Proceedings of the 10th international conference on World Wide Web, ACM Press, Hong Kong (2001)Google Scholar
  17. 17.
    Ziefle, M., Bay, S.: Mental models of a cellular phone menu. Comparing older and younger novice users, in Mobile Human-Computer Interaction - Mobilehci 2004, Proceedings. pp. 25–37 (2004)Google Scholar
  18. 18.
    Chuang, M.C., Chang, C.C., Hsu, S.H.: Perceptual factors underlying user preferences toward product form of mobile phones. International Journal of Industrial Ergonomics 27(4), 247–258 (2001)CrossRefGoogle Scholar
  19. 19.
    Ling, C., Hwang, W., Salvendy, G.: A survey of what customers want in a cell phone design. Behaviour & Information Technology (2005)Google Scholar
  20. 20.
    Ling, C., Hwang, W., Salvendy, G.: Diversified users’ satisfaction with advanced mobile phone features. Universal Access in the Information Society 5(2), 239–249 (2006)CrossRefGoogle Scholar
  21. 21.
    Albers, M.J., Kim, L.: User Web browsing characteristics using palm handhelds for information retrieval. In: Professional Communication Conference, 2000. Proceedings of 2000 Joint IEEE International and 18th Annual Conference on Computer Documentation (IPCC/SIGDOC 2000), IEEE, Cambridge, MA (2000)Google Scholar
  22. 22.
    Chittaro, L., Dal Cin, P.: Evaluating interface design choices on WAP phones: navigation and selection. Personal and Ubiquitous Computing 6(4), 237–244 (2002)CrossRefGoogle Scholar
  23. 23.
    Christie, J., Klein, R.M., Watters, C.: A comparison of simple hierarchy and grid metaphors for option layouts on small-size screens. International Journal of Human-Computer Studies 60(5-6), 564–584 (2004)CrossRefGoogle Scholar
  24. 24.
    Kjeldskov, J., Stage, J.: New techniques for usability evaluation of mobile systems. International Journal of Human-Computer Studies 60(5-6), 599–620 (2004)CrossRefGoogle Scholar
  25. 25.
    Marila, J., Ronkainen, S.: Time-out in mobile text input: The effects of learning and feedback. In: Chittaro, L. (ed.) Mobile HCI 2003. LNCS, vol. 2795, pp. 91–103. Springer, Heidelberg (2003)Google Scholar
  26. 26.
    Ziefle, M.: The influence of user expertise and phone complexity on performance, ease of use and learnability of different mobile phones. Behaviour & Information Technology 21(5), 303–311 (2002)CrossRefGoogle Scholar
  27. 27.
    Ji, Y.G., et al.: A usability checklist for the usability evaluation of mobile phone user interface. International Journal of Human-Computer Interaction 20(3), 207–231 (2006)CrossRefGoogle Scholar
  28. 28.
    Jones, M., et al.: Improving Web interaction on small displays. Computer Networks 31(11-16), 1129–1137 (1999)CrossRefGoogle Scholar
  29. 29.
    Rao, H., et al.: iMail: a WAP mail retrieving system. Information Sciences 151, 71–91 (2003)zbMATHCrossRefGoogle Scholar
  30. 30.
    Watters, C., Duffy, J., Duffy, K.: Using large tables on small screen display devices. International Journal of Human Computer Studies 58(1), 21–37 (2003)CrossRefGoogle Scholar
  31. 31.
    Watters, C., Zhang, R.: PDA access to Internet content: Focus on forms. In: Proceedings of the 36th Annual Hawaii International Conference on System Sciences (HICSS’03) - Track 4., IEEE Computer Society, Hawaii (2003)Google Scholar
  32. 32.
    Anderson, C.R., Domingos, P., Weld, D.S.: Personalizing web sites for mobile users. In: Proceedings of the 10th international conference on World Wide Web, ACM Press, Hong Kong (2001)Google Scholar
  33. 33.
    Bruner, G.C., Kumar, A.: Explaining consumer acceptance of handheld Internet devices. Journal of Business Research 58(5), 553–558 (2005)CrossRefGoogle Scholar
  34. 34.
    Ho, S.Y., Kwok, S.H.: The attraction of personalized service for users in mobile commerce: an empirical study. ACM SIGecom Exchanges 3(4), 10–18 (2002)CrossRefGoogle Scholar
  35. 35.
    Venkatesh, V., Ramesh, V., Massey, A.P.: Understanding usability in mobile commerce. Communications of the ACM 46(12), 53–56 (2003)CrossRefGoogle Scholar
  36. 36.
    Tarasewich, P.: Designing mobile commerce applications. Communications of the ACM 46(12), 57–60 (2003)CrossRefGoogle Scholar
  37. 37.
    Malloy, A.D., Varshney, U., Snow, A.P.: Supporting mobile commerce applications using dependable wireless networks. Mobile Networks and Applications 7(3), 225–234 (2002)zbMATHCrossRefGoogle Scholar
  38. 38.
    Koutsiouris, V., Vlachos, P., Vrechopoulos, A.: Developing and evaluating mobile entertainment applications: The case of the music industry. In: Rauterberg, M. (ed.) ICEC 2004. LNCS, vol. 3166, pp. 513–517. Springer, Heidelberg (2004)Google Scholar
  39. 39.
    Kaasinen, E.: User acceptance of location-aware mobile guides based on seven field studies. Behaviour & Information Technology 24(1), 37–49 (2005)CrossRefGoogle Scholar
  40. 40.
    Fithian, R., et al.: The design and evaluation of a mobile location-aware handheld event planner. In: Proceedings of Human-Computer Interaction with Mobile Devices and Services: 5th International Symposium, Mobile HCI, 2003. Udine, Italy (2003)Google Scholar
  41. 41.
    Ciavarella, C., Paternò, F.: Design criteria for location-aware, indoor, PDA applications. In: Dignum, F.P.M., Cortés, U. (eds.) Agent-Mediated Electronic Commerce III. LNCS (LNAI), vol. 2003, pp. 131–144. Springer, Heidelberg (2001)Google Scholar
  42. 42.
    Howell, M., Love, S., Turner, M.: The impact of interface metaphor and context of use on the usability of a speech-based mobile city guide service. Behaviour & Information Technology 24(1), 67–78 (2005)CrossRefGoogle Scholar
  43. 43.
    Folmer, E., Bosch, J.: Architecting for usability: a survey. Journal of Systems and Software 70(1-2), 61–78 (2004)CrossRefGoogle Scholar
  44. 44.
    Hornbaek, K.: Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies 64(2), 79–102 (2006)CrossRefGoogle Scholar
  45. 45.
    Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of SIGCHI ’88, Washington, DC: New York: ACM/SIGCHI (1988)Google Scholar
  46. 46.
    Lewis, J.R.: Psychometric evaluation of the post-study system usability questionnaire: the PSSUQ. In: Proceedings of the Human Factors Society 36th Annual Meeting, Human Factors Society, Atlanta, GA (1992)Google Scholar
  47. 47.
    Kirakowski, J., Corbett, M.: SUMI: The software usability measurement inventory. British Journal of Educational Technology 24(3), 210–212 (1993)CrossRefGoogle Scholar
  48. 48.
    Lewis, J.R.: IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 7(1), 57–58 (1995)CrossRefGoogle Scholar
  49. 49.
    Lin, H.X., Choong, Y.Y., Salvendy, G.: A proposed index of usability: A method for comparing the relative usability of different software systems. Behaviour & Information Technology 16(4-5), 267–278 (1997)CrossRefGoogle Scholar
  50. 50.
    Kirakowski, J., Claridge, N.: Website Analysis and MeasureMent Inventory (Web Usability Questionnaire). (1998) (cited 2006 Dec. 05th), Available from:
  51. 51.
    Muylle, S., Moenaert, R., Despontin, M.: The conceptualization and empirical validation of web site user satisfaction. Information & Management 41(5), 543–560 (2004)CrossRefGoogle Scholar
  52. 52.
    Hair, J.E., et al.: Multivariate data analysis: with readings, 4th edn. Prentice-Hall, Inc, Upper Saddle River, NJ, USA (1995)Google Scholar
  53. 53.
    Stiggelbout, A.M., et al.: Ideals of patient autonomy in clinical decision making: a study on the development of a scale to assess patients’ and physicians’ views. Journal of Medical Ethics 30(3), 268–274 (2004)CrossRefGoogle Scholar
  54. 54.
    Chiou, C.F., et al.: Development and validation of the revised Cedars-Sinai Health-Related Quality of Life for Rheumatoid Arthritis Instrument. Arthritis & Rheumatism-Arthritis Care. & Research 55(6), 856–863 (2006)CrossRefGoogle Scholar
  55. 55.
    Smith, B., CapUti, P., Rawstorne, P.: The development of a measure of subjective computer experience. Computers in Human Behavior 23(1), 127–145 (2007)CrossRefGoogle Scholar
  56. 56.
    Wang, Y.-S., Liao, Y.-W.: The conceptualization and measurement of m-commerce user satisfaction. Computers in Human Behavior 23(1), 381–398 (2007)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Ting Zhang
    • 1
  • Pei-Luen Patrick Rau
    • 1
  • Gavriel Salvendy
    • 1
  1. 1.Department of Industrial Engineering, Tsinghua University, Beijing 100084China

Personalised recommendations