The 100,000 Participant Laboratory - A Crowd-Centered Approach to Design and Evaluate the Usability of Mobile Apps

  • Ming-Hui WenEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9746)


Mobile apps play important roles in our daily lives. Software developers typically follow the user-centered design (UCD) approach to develop products that optimize usability. However, most usability methodologies require lab experiments or field studies. Usability experiments require higher costs and efforts to prepare related tasks, such as selecting the environmental setting, subject recruiting and conducting the study. Compared to the laboratory experiment of personal computers (PC), mobile usability laboratory settings present greater challenges related to controlling the research context variables.

The purpose of this study is to build an online usability lab by applying Internet and crowdsourcing technologies. The lab allows researchers to invite Internet users to participate in the usability experiment online. This study follows the usability engineering theoretical framework to define mobile usability contextual factors. The clouded-based online usability lab was introduced as a conceptual framework to support both Android and iOS mobile apps. Mobile app developers can upload their apps to the lab system to collect and monitor usability data and track the behavioral data of human-computer interaction. The ultimate goal of this lab was to help researchers conduct usability experiments in real world contexts and develop the dynamic mental model of product users.


Mobile usability Mental model User experience Crowdsourcing 



The author thank the Ministry of Science and Technology (MOST) in Taiwan for providing financial support under grant MOST 103-2221-E-163 -003 -MY2.


  1. Allen, C.D., Ballman, D., Begg, V., Miller-Jacobs, H. H., Muller, M., Nielsen, J., Spool, J.: User involvement in the design process: why, when & how?. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 251–254. ACM (May 1993)Google Scholar
  2. Roy, M.C., Dewit, O., Aubert, B.A.: The impact of interface usability on trust in web retailers. Internet Res. 11(5), 388–398 (2001)CrossRefGoogle Scholar
  3. Costa, P.T., McCrae, R.R.: Four ways five factors are basic. Personality Individ. Differ. 13(6), 653–665 (1992)CrossRefGoogle Scholar
  4. Deterding, S., Dixon, D., Khaled, R., Nacke, L.: From game design elements to gamefulness: defining gamification. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, pp. 9–15. ACM (September 2011)Google Scholar
  5. Devaraj, S., Easley, R.F., Crant, J.M.: Research note-how does personality matter? relating the five-factor model to technology acceptance and use. Inf. Syst. Res. 19(1), 93–105 (2008)CrossRefGoogle Scholar
  6. Engeström, Y.: Activity theory and individual and social transformation. Perspect. Act. Theor., 19–38 (1999)Google Scholar
  7. Gould, J.D.: How to design usable systems. In: Handbook of Human-Computer Interaction, pp. 757–789, Amsterdam (1988)Google Scholar
  8. Hsu, S.H., Wen, M.H., Wu, M.C.: Exploring user experiences as predictors of MMORPG addiction. Comput. Educ. 53(3), 990–995 (2009)CrossRefGoogle Scholar
  9. Huotari, K., Hamari, J.: Gamification” from the perspective of service marketing. In: Proceedings of CHI 2011 Workshop Gamification (May 2011)Google Scholar
  10. Kapp, K.M. The gamification of learning and instruction: game-based methods and strategies for training and education. Wiley. com (2012)Google Scholar
  11. Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 453–456. ACM (April 2008)Google Scholar
  12. Kumar, J.: Gamification at work: designing engaging business software. In: Marcus, A. (ed.) DUXU 2013, Part II. LNCS, vol. 8013, pp. 528–537. Springer, Heidelberg (2013)Google Scholar
  13. Leontev, A.N.: Activity, consciousness, and personality. Moscow: Progress (1978)Google Scholar
  14. McCrae, R.R., Costa Jr, P.T.: A five-factor theory of personality. Handb. pers. Theor. Res. 2, 139–153 (1999)Google Scholar
  15. Levanthal, L., Teasley, B., Stone, D., Lancaster, A.M., Marcus, A., Nardi, B., Nielsen, J., Kurosu, M., Heller, R.: Designing for diverse users: will just a better interface do?. In: Conference Companion on Human Factors in Computing Systems, pp. 191–192. ACM (April 1994)Google Scholar
  16. Nielsen, J.: Finding usability problems through heuristic evaluation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 373–380. ACM (June 1992)Google Scholar
  17. Norman, D.A.: Some observations on mental models. In: Mental Models, pp. 7–14 (1983)Google Scholar
  18. Norman, D.A.: The psychology of everyday things. Basic books, New York (1988)Google Scholar
  19. Raymer, R., Design, E.L.: Gamification: using game mechanics to enhance eLearning. Elearn Mag. 2011(9), 3 (2011)CrossRefGoogle Scholar
  20. Reynolds, F.D., Darden, W.R.: Intermarket patronage: a psychographic study of consumer outshoppers. J. Mark. Res. 36(4), 50–54 (1972)Google Scholar
  21. Viitamaki, S.: The FLIRT model of crowdsourcing: planning and executing collective customer collaboration. Marketing Master Thesis, Helsinki School of Economics, Spring (2008)Google Scholar
  22. Wansink, B., Park, S.B.: Methods and measures that profile heavy users. J. Advertising Res. 40(2000), 61–72 (2000)CrossRefGoogle Scholar
  23. Wickens, C.D.: Computational models of human performance. CSERIAC Gateway 3, 8–12 (1992)Google Scholar
  24. William, D.W., Douglas, J.T.: Activities, interests and opinions. J. Advertising Res. 11(4), 27–35 (1971)Google Scholar
  25. Marzilli, M., Yan, T., Holmes, R., Ganesan, D., Corner, M.: mCrowd: a platform for mobile crowdsourcing. In: Proceedings of the 7th ACM Conference on Embedded Networked Sensor Systems, pp. 347–348. ACM (November 2009)Google Scholar
  26. Yen, Y.C., Chu, C.Y., Yeh, S.L., Chu, H.H., Huang, P.: Lab Experiment vs. Crowdsourcing: A Comparative User Study on Skype Call Quality (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Commercial Design and ManagementNational Taipei University of BusinessTaipeiTaiwan

Personalised recommendations