Abstract
Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yardley, L., Morrison, L., Bradbury, K., Muller, I.: The person-based approach to intervention development: application to digital health-related behavior change interventions. J. Med. Internet Res. 17(1), 30 (2015)
Ramachandran, A., Snehalatha, C., Ram, J., Selvam, S., Simon, M., Nanditha, A., Oliver, N., Shetty, A.S., Godsland, I.F., Chaturvedi, N., Majeed, A., Toumazou, C., Alberti, K.G., Johnston, D.G.: Effectiveness of mobile phone messaging in prevention of type 2 diabetes by lifestyle modification in men in India: a prospective, parallel-group, randomised controlled trial. Lancet Diabetes Endocrinol. 1(3), 191–198 (2013)
Eaton, C.B., Hartman, S.J., Perzanowski, E., Pan, G., Roberts, M.B., Risica, P.M., Gans, K.M., Jakicic, J.M., Marcus, B.H.: A randomized clinical trial of a tailored lifestyle intervention for obese, sedentary, primary care patients. Ann. Fam. Med. 14(4), 311–319 (2016)
Nahum-Shani, I., Smith, S.N., Tewari, A., Witkiewitz, K., Collins, L.M., Spring, B., Murphy, S.: Just in time adaptive interventions (JITAIs): an organizing framework for ongoing health behavior support. Methodology Center Technical report (14-126) (2014)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems, vol. 37. University of Cambridge, Department of Engineering (1994)
Pellegrini, C.A., Pfammatter, A.F., Conroy, D.E., Spring, B.: Smartphone applications to support weight loss: current perspectives. Adv. Health Care Technol. 1, 13 (2015)
Dantzig, S., Geleijnse, G., Halteren, A.T.: Toward a persuasive mobile application to reduce sedentary behavior. Pers. Ubiquit. Comput. 17(6), 1237–1246 (2013)
Kelly, J., Gooding, P., Pratt, D., Ainsworth, J., Welford, M., Tarrier, N.: Intelligent real-time therapy: harnessing the power of machine learning to optimise the delivery of momentary cognitive-behavioural interventions. J. Ment. Health 21(4), 404–414 (2012)
Hammond, R.A., Ornstein, J.T., Fellows, L.K., Dubé, L., Levitan, R., Dagher, A.: A model of food reward learning with dynamic reward exposure. Front. Comput. Neurosci. 6, 82 (2012). https://doi.org/10.3389/fncom.2012.00082
Mohan, S., Venkatakrishnan, A., Silva, M., Pirolli, P.: On designing a social coach to promote regular aerobic exercise. In: AAAI, pp. 4721–4727 (2017)
Navarro-Barrientos, J.E., Rivera, D.E., Collins, L.M.: A dynamical model for describing behavioural interventions for weight loss and body composition change. Math. Comput. Model. Dyn. Syst. 17(2), 183–203 (2011)
Pielot, M., Cardoso, B., Katevas, K., Serrà , J., Matic, A., Oliver, N.: Beyond interruptibility: predicting opportune moments to engage mobile phone users. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 1(3), 91 (2017)
Oh, H., Jalali, L., Jain, R.: An intelligent notification system using context from real-time personal activity monitoring. In: IEEE International Conference on 2015 Multimedia and Expo (ICME), pp. 1–6. IEEE (2015)
Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: a survey. J. Mach. Learn. Res. 10, 1633–1685 (2009)
Acknowledgement
The research leading to these results has received funding partially from the European Community’s H2020 Programme under grant agreement no H2020-PHC-689444, POWER2DM project (Predictive Model-Based Decision Support for Diabetes Patient Empowerment) and partially from The Scientific and Technological Research Council of Turkey (TÜBİTAK).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Gonul, S., Namli, T., Baskaya, M., Sinaci, A.A., Cosar, A., Toroslu, I.H. (2018). Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning. In: Mouhoub, M., Sadaoui, S., Ait Mohamed, O., Ali, M. (eds) Recent Trends and Future Technology in Applied Intelligence. IEA/AIE 2018. Lecture Notes in Computer Science(), vol 10868. Springer, Cham. https://doi.org/10.1007/978-3-319-92058-0_32
Download citation
DOI: https://doi.org/10.1007/978-3-319-92058-0_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92057-3
Online ISBN: 978-3-319-92058-0
eBook Packages: Computer ScienceComputer Science (R0)