Further into the Wild: Running Worldwide Trials of Mobile Systems

  • Donald McMillan
  • Alistair Morrison
  • Owain Brown
  • Malcolm Hall
  • Matthew Chalmers
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6030)


Many studies of ubiquitous computing systems involve deploying a system to a group of users who will be studied through direct observation, interviews and the gathering of system log data. However, such studies are often limited in the number of participants and duration of the trial, particularly if the researchers are providing the participants with hardware. Apple’s App Store and similar application repositories have become popular with smartphone users, yet few ubiquitous computing studies have yet utilised these distribution mechanisms. We describe our experiences of running a very large scale trial where such a distribution model is used to recruit thousands of users for a mobile system trial that can be run continuously with no constrained end date. We explain how we conducted such a trial, covering issues such as data logging and interviewing users based in several different continents. Benefits and potential shortcomings of running a trial in this way are discussed and we offer guidance on ways to help manage a large and disparate user-base using in-application feedback measures and web-based social networking applications. We describe how, through these methods, we were able to further the development of a piece of ubiquitous computing software through user-informed design on a mass scale.


Evaluation Techniques Large Scale Deployment Trial Methods 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rogers, Y., et al.: Why it’s worth the hassle: The value of in-situ studies when designing UbiComp. In: Krumm, J., Abowd, G.D., Seneviratne, A., Strang, T. (eds.) UbiComp 2007. LNCS, vol. 4717, pp. 336–353. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    IDC, Worldwide Converged Mobile Device Market Grows 39.0% Year Over Year in Fourth Quarter,
  3. 3.
    Zhai, S., et al.: Shapewriter on the iPhone: from the laboratory to the real world. In: Proc. ACM CHI Extended Abstracts, pp. 2667–2670 (2009)Google Scholar
  4. 4.
    Bell, M., et al.: Interweaving Mobile Games with Everyday Life. In: Proc. ACM CHI, pp. 417–426 (2006)Google Scholar
  5. 5.
    Hall, M., et al.: Adapting Ubicomp Software and its Evaluation. In: Proc. ACM Engineering Interactive Computing Systems, pp. 143–148 (2009)Google Scholar
  6. 6.
    O’Neill, E., et al.: Instrumenting the city: developing methods for observing and understanding the digital cityscape. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 315–332. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Reades, J., et al.: Cellular Census: Explorations in Urban Data Collection. Pervasive Computing 6(3), 30–38 (2007)CrossRefGoogle Scholar
  8. 8.
    Chin, A.: Finding Cohesive Subgroups and Relevant Members in the Nokia Friend View Mobile Social Network. In: Proc. Social Computing, pp. 278–283 (2009)Google Scholar
  9. 9.
    Licoppe, C., Inada, Y.: Emergent Uses of a Multiplayer Location-aware Mobile Game: the Interactional Consequences of Mediated Encounters. Mobilities 1(1), 39–61 (2006)CrossRefGoogle Scholar
  10. 10.
    Crabtree, A., et al.: Supporting Ethnographic Studies of Ubiquitous Computing in the Wild. In: Proc. ACM DIS, pp. 60–69 (2006)Google Scholar
  11. 11.
    Froehlich, J., et al.: Voting With Your Feet: An Investigative Study of the Relationship Between Place Visit Behavior and Preference. In: Dourish, P., Friday, A. (eds.) UbiComp 2006. LNCS, vol. 4206, pp. 333–350. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Carter, S., et al.: Support for Situated Ubicomp Experimentation. In: Proc. ACM CHI, pp. 125–134 (2007)Google Scholar
  13. 13.
    Morrison, A., et al.: Using Location and Motion Data to Filter System Logs. In: LaMarca, A., Langheinrich, M., Truong, K.N. (eds.) Pervasive 2007. LNCS, vol. 4480, pp. 109–126. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Common European Framework of Reference for Languages,
  15. 15.
  16. 16.
    FAST, Federation Asks: Do you know what you’re agreeing to?,

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Donald McMillan
    • 1
  • Alistair Morrison
    • 1
  • Owain Brown
    • 1
  • Malcolm Hall
    • 1
  • Matthew Chalmers
    • 1
  1. 1.Department of Computing ScienceUniversity of GlasgowUK

Personalised recommendations