Mobile Networks and Applications

, Volume 18, Issue 1, pp 81–97 | Cite as

Design and Implementation of a Toolkit for Usability Testing of Mobile Apps

  • Xiaoxiao Ma
  • Bo Yan
  • Guanling Chen
  • Chunhui Zhang
  • Ke Huang
  • Jill Drury
  • Linzhang Wang
Article

Abstract

The usability of mobile applications is critical for their adoption because of the relatively small screen and awkward (sometimes virtual) keyboard, despite the recent advances of smartphones. Traditional laboratory-based usability testing is often tedious, expensive, and does not reflect real use cases. In this paper, we propose a toolkit that embeds into mobile applications the ability to automatically collect user interface (UI) events as the user interacts with the applications. The events are fine-grained and useful for quantified usability analysis. We have implemented the toolkit on Android devices and we evaluated the toolkit with a real deployed Android application by comparing event analysis (state-machine based) with traditional laboratory testing (expert based). The results show that our toolkit is effective at capturing detailed UI events for accurate usability analysis.

Keywords

Toolkit Usability testing Mobile application Automated Logging method 

References

  1. 1.
    Kerr R (2009) US mobile internet usage to near 50 % in 2013. In: Vator newsGoogle Scholar
  2. 2.
    Budiu R, Nielsen J (2009) Usability of mobile websites: 85 design guidelines for improving access to web-based content and services through mobile devices. Nielsen Norman Group Research ReportGoogle Scholar
  3. 3.
    Gomez Inc (2009) Why the mobile web is disappointing end-users. Equation Research ReportGoogle Scholar
  4. 4.
    Tullis T, Fleischman S, McNulty M, Cianchette C, Bergel M (2002) An empirical comparison of lab and remote usability testing of web sites. In: Usability professional association conference, OrlandoGoogle Scholar
  5. 5.
    Bastien JMC (2010) Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform 79:e18–23CrossRefGoogle Scholar
  6. 6.
    West R, Lehman K (2006) Automated summative usability studies: an empirical evaluation. In: Proceedings of the SIGCHI conference on human factors in computing systems, ser. CHI ’06. ACM, New York, pp 631–639Google Scholar
  7. 7.
    Waterson S, Landay JA (2002) In the lab and out in the wild: remote web usability testing for mobile devices. In: Conference on human factors in computing systems, pp 296–297Google Scholar
  8. 8.
    Hilbert DM, Redmiles DF (2000) Extracting usability information from user interface events. ACM Comput Surv 32:384–421CrossRefGoogle Scholar
  9. 9.
    Yan B, Chen G (2011) Appjoy: personalized mobile application discovery. In: Proceedings of the 9th international conference on mobile systems, applications, and services, ser. MobiSys ’11. ACM, New York, pp 113–126. Available: http://doi.acm.org/10.1145/1999995.2000007 CrossRefGoogle Scholar
  10. 10.
    Rosenbaum S, Rohn J, Humburg J (2000) A toolkit for strategic usability: results from workshops, panels, and surveys. In: Proceedings of the ACM CHI 2000 conference on human factors in computing systems, New York, pp 337–344Google Scholar
  11. 11.
    Usability Professionals’ Association (2008) UPA 2007 Salary SurveyGoogle Scholar
  12. 12.
    Ericsson KA, Simon HA (1980) Verbal reports as data. Psychol Rev 87:215–251CrossRefGoogle Scholar
  13. 13.
    Nieslen J (2007) Severity ratings for usability problems. Retrieved June 4th from UseIt. Available: http://www.useit.com/papers/heuristic/severityrating.html
  14. 14.
    Akers D (2009) Backtracking events as indicators of software usability problems. Ph.D. dissertationGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Xiaoxiao Ma
    • 1
  • Bo Yan
    • 1
  • Guanling Chen
    • 2
  • Chunhui Zhang
    • 1
  • Ke Huang
    • 1
  • Jill Drury
    • 1
  • Linzhang Wang
    • 2
  1. 1.Computer Science DepartmentUniversity of Massachusetts LowellLowellUSA
  2. 2.State Key Laboratory for Novel Software Technology, Department of Computer Science and TechnologyNanjing UniversityNanjingPeople’s Republic of China

Personalised recommendations