Development of Mobile Data Collection Applications by Domain Experts: Experimental Results from a Usability Study

  • Johannes Schobel
  • Rüdiger Pryss
  • Winfried Schlee
  • Thomas Probst
  • Dominic Gebhardt
  • Marc Schickler
  • Manfred Reichert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10253)


Despite their drawbacks, paper-based questionnaires are still used to collect data in many application domains. In the QuestionSys project, we develop an advanced framework that enables domain experts to transform paper-based instruments to mobile data collection applications, which then run on smart mobile devices. The framework empowers domain experts to develop robust mobile data collection applications on their own without the need to involve programmers. To realize this vision, a configurator component applying a model-driven approach is developed. As this component shall relieve domain experts from technical issues, it has to be proven that domain experts are actually able to use the configurator properly. The experiment presented in this paper investigates the mental efforts for creating such data collection applications by comparing novices and experts. Results reveal that even novices are able to model instruments with an acceptable number of errors. Altogether, the QuestionSys framework empowers domain experts to develop sophisticated mobile data collection applications by orders of magnitude faster compared to current mobile application development practices.


Process-driven applications End-user programming Experimental results 


  1. 1.
    Bachmann, A., Zetzsche, R., Schankin, A., Riedel, T., Beigl, M., Reichert, M., Santangelo, P., Ebner-Priemer, U.: ESMAC: a web-based configurator for context-aware experience sampling apps in ambulatory assessment. In: 5th International Conference on Wireless Mobile Communication and Healthcare, pp. 15–18 (2015)Google Scholar
  2. 2.
    Basili, V.R.: Software Modeling and Measurement: The Goal/Question/Metric Paradigm (1992)Google Scholar
  3. 3.
    Begel, A., Klopfer, E.: Starlogo TNG: an introduction to game development. J. E-Learn. (2007)Google Scholar
  4. 4.
    Brooke, J., et al.: SUS - a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)Google Scholar
  5. 5.
    Carlbring, P., Brunt, S., Bohman, S., Austin, D., Richards, J., Öst, L.G., Andersson, G.: Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput. Hum. Behav. 23(3), 1421–1434 (2007)CrossRefGoogle Scholar
  6. 6.
    Claes, J., Vanderfeesten, I., Pinggera, J., Reijers, H.A., Weber, B., Poels, G.: A visual analysis of the process of process modeling. Inf. Syst. e-Bus. Manag. 13(1), 147–190 (2015)CrossRefGoogle Scholar
  7. 7.
    Cook, T.D., Campbell, D.T., Day, A.: Quasi-Experimentation: Design and Analysis Issues for Field Settings, vol. 351. Houghton Mifflin, Boston (1979)Google Scholar
  8. 8.
    Fernandez-Ballesteros, R.: Self-report questionnaires. Compr. Handb. Psychol. Assess. 3, 194–221 (2004)Google Scholar
  9. 9.
    Höst, M., Regnell, B., Wohlin, C.: Using students as subjects - a comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Eng. 5(3), 201–214 (2000)CrossRefMATHGoogle Scholar
  10. 10.
    Kandogan, E., Haber, E., Barrett, R., Cypher, A., Maglio, P., Zhao, H.: A1: end-user programming for web-based system administration. In: Proceedings of 18th ACM Symposium on User Interface Software and Technology. ACM (2005)Google Scholar
  11. 11.
    Lane, S.J., Heddle, N.M., Arnold, E., Walker, I.: A review of randomized controlled trials comparing the effectiveness of hand held computers with paper methods for data collection. BMC Med. Inform. Decis. Mak. 6(1), 1 (2006)CrossRefGoogle Scholar
  12. 12.
    Marcano Belisario, J.S., Jamsek, J., Huckvale, K., O’Donoghue, J., Morrison, C.P., Car, J.: Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. The Cochrane Library (2015)Google Scholar
  13. 13.
    Martini, M., Pinggera, J., Neurauter, M., Sachse, P., Furtner, M.R., Weber, B.: The impact of working memory and the process of process modelling on model quality: investigating experienced versus inexperienced modellers. Sci. Rep. 6, 1–11 (2016)CrossRefGoogle Scholar
  14. 14.
    Palermo, T.M., Valenzuela, D., Stork, P.P.: A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain 107(3), 213–219 (2004)CrossRefGoogle Scholar
  15. 15.
    Pavlović, I., Kern, T., Miklavčič, D.: Comparison of paper-based and electronic data collection process in clinical trials: costs simulation study. Contemp. Clin. Trials 30(4), 300–316 (2009)CrossRefGoogle Scholar
  16. 16.
    Rafique, I., Lew, P., Abbasi, M.Q., Li, Z.: Information quality evaluation framework: extending ISO 25012 data quality model. World Acad. Sci. Eng. Technol. 65, 523–528 (2012)Google Scholar
  17. 17.
    Reichert, M., Weber, B.: Enabling Flexibility in Process-Aware Information Systems: Challenges, Methods, Technologies. Springer, Heidelberg (2012)CrossRefMATHGoogle Scholar
  18. 18.
    Renaud, G., Azzopardi, L.: SCAMP: a tool for conducting interactive information retrieval experiments. In: IIiX, pp. 286–289 (2012)Google Scholar
  19. 19.
    Schobel, J., Pryss, R., Schickler, M., Reichert, M.: A configurator component for end-user defined mobile data collection processes. In: Demo Track of the 14th International Conference on Service Oriented Computing, October 2016Google Scholar
  20. 20.
    Schobel, J., Pryss, R., Schickler, M., Reichert, M.: A lightweight process engine for enabling advanced mobile applications. In: Debruyne, C., et al. (eds.) OTM 2016. LNCS, vol. 10033, pp. 552–569. Springer, Cham (2016). doi:10.1007/978-3-319-48472-3_33 CrossRefGoogle Scholar
  21. 21.
    Schobel, J., Pryss, R., Schickler, M., Ruf-Leuschner, M., Elbert, T., Reichert, M.: End-user programming of mobile services: empowering domain experts to implement mobile data collection applications. In: IEEE 5th International Conference on Mobile Services. IEEE Computer Society Press, June 2016Google Scholar
  22. 22.
    Weber, B., Pinggera, J., Neurauter, M., Zugal, S., Martini, M., Furtner, M., Sachse, P., Schnitzer, D.: Fixation patterns during process model creation: initial steps toward neuro-adaptive process modeling environments. In: 2016 49th Hawaii International Conference on System Sciences (HICSS), pp. 600–609. IEEE (2016)Google Scholar
  23. 23.
    Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer Science & Business Media, Heidelberg (2012)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Johannes Schobel
    • 1
  • Rüdiger Pryss
    • 1
  • Winfried Schlee
    • 2
  • Thomas Probst
    • 1
  • Dominic Gebhardt
    • 1
  • Marc Schickler
    • 1
  • Manfred Reichert
    • 1
  1. 1.Institute of Databases and Information SystemsUlm UniversityUlmGermany
  2. 2.Department of Psychiatry and PsychotherapyRegensburg UniversityRegensburgGermany

Personalised recommendations