POMDPs for Assisting Homeless Shelters – Computational and Deployment Challenges

  • Amulya YadavEmail author
  • Hau Chan
  • Albert Jiang
  • Eric Rice
  • Ece Kamar
  • Barbara Grosz
  • Milind Tambe
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10003)


This paper looks at challenges faced during the ongoing deployment of HEALER, a POMDP based software agent that recommends sequential intervention plans for use by homeless shelters, who organize these interventions to raise awareness about HIV among homeless youth. HEALER’s sequential plans (built using knowledge of social networks of homeless youth) choose intervention participants strategically to maximize influence spread, while reasoning about uncertainties in the network. In order to compute its plans, HEALER (i) casts this influence maximization problem as a POMDP and solves it using a novel planner which scales up to previously unsolvable real-world sizes; (ii) and constructs social networks of homeless youth at low cost, using a Facebook application. HEALER is currently being deployed in the real world in collaboration with a homeless shelter. Initial feedback from the shelter officials has been positive but they were surprised by the solutions generated by HEALER as these solutions are very counter-intuitive. Therefore, there is a need to justify HEALER’s solutions in a way that mirrors the officials’ intuition. In this paper, we report on progress made towards HEALER’s deployment and detail first steps taken to tackle the issue of explaining HEALER’s solutions.


Intervention Participant Homeless Youth Partially Observable Markov Decision Process Homeless Shelter Influence Spread 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This research was supported by MURI Grant W911NF-11-1-0332 and NIMH Grant number R01-MH093336.


  1. 1.
    Borgs, C., Brautbar, M., Chayes, J., Lucier, B.: Maximizing social influence in nearly optimal time. In: Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2014, pp. 946–957. SIAM (2014)Google Scholar
  2. 2.
    Cohen, E., Delling, D., Pajor, T., Werneck, R.F.: Sketch-based influence maximization and computation: scaling up with guarantees. In: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 629–638. ACM (2014)Google Scholar
  3. 3.
    Cointet, J.-P., Roth, C.: How realistic should knowledge diffusion models be? J. Artif. Soc. Soc. Simul. 10(3), 5 (2007)Google Scholar
  4. 4.
    Council, N.H.: HIV/AIDS among persons experiencing homelessness: risk factors, predictors of testing, and promising testing strategies (2012).
  5. 5.
    Dibangoye, J.S., Shani, G., Chaib-Draa, B., Mouaddib, A.-I.: Topological order planner for POMDPs. In: International Joint Conference on Artificial Intelligence (IJCAI) (2009)Google Scholar
  6. 6.
    Eck, A., Soh, L.-K.: To ask, sense, or share: ad hoc information gathering. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, pp. 367–376. International Foundation for Autonomous Agents and Multiagent Systems (2015)Google Scholar
  7. 7.
    Golovin, D., Krause, A.: Adaptive submodularity: theory and applications in active learning and stochastic optimization. J. Artif. Intell. Res. 42, 427–486 (2011)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Kelly, J.A., Murphy, D.A., Sikkema, K.J., McAuliffe, T.L., Roffman, R.A., Solomon, L.J., Winett, R.A., Kalichman, S.C.: Randomised, controlled, community-level HIV-prevention intervention for sexual-risk behaviour among homosexual men in US cities. Lancet 350(9090), 1500 (1997)CrossRefGoogle Scholar
  9. 9.
    Kempe, D., Kleinberg, J., Tardos, É.: Maximizing the spread of influence through a social network. In: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 137–146. ACM (2003)Google Scholar
  10. 10.
    Khan, O.Z., Poupart, P., Black, J.P.: Minimal sufficient explanations for factored Markov decision processes. Citeseer (2009)Google Scholar
  11. 11.
    Kim, M., Leskovec, J.: The network completion problem: inferring missing nodes and edges in networks. In: Proceedings of the SIAM Conference on Data Mining. SIAM (2011)Google Scholar
  12. 12.
    LaSalle, D., Karypis, G.: Multi-threaded graph partitioning. In: 2013 IEEE 27th International Symposium on Parallel & Distributed Processing (IPDPS), pp. 225–236. IEEE (2013)Google Scholar
  13. 13.
    Leskovec, J., Krause, A., Guestrin, C., Faloutsos, C., VanBriesen, J., Glance, N.: Cost-effective outbreak detection in networks. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 420–429. ACM (2007)Google Scholar
  14. 14.
    Marcolino, L., Lakshminarayanan, A., Yadav, A., Tambe, M.: Simultaneous influencing and mapping social networks. In: Proceedings of the Fifteenth International Conference on Autonomous Agents and Multiagent Systems (Short Paper) (AAMAS 2016) (2016)Google Scholar
  15. 15.
    Paquet, S., Tobin, L., Chaib-Draa, B.: An online POMDP algorithm for complex multiagent environments. In: Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 970–977. ACM (2005)Google Scholar
  16. 16.
    Poupart, P., Kim, K.-E., Kim, D.: Closing the gap: improved bounds on optimal POMDP solutions. In: International Conference on Automated Planning and Scheduling (2011)Google Scholar
  17. 17.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2009)zbMATHGoogle Scholar
  18. 18.
    Rice, E.: The positive role of social networks and social networking technology in the condom-using behaviors of homeless young people. Public Health Rep. 125(4), 588 (2010)Google Scholar
  19. 19.
    Rice, E., Barman-Adhikari, A., Milburn, N.G., Monro, W.: Position-specific HIV risk in a large network of homeless youths. Am. J. Pub. Health 102(1), 141–147 (2012)CrossRefGoogle Scholar
  20. 20.
    Rice, E., Fulginiti, A., Winetrobe, H., Montoya, J., Plant, A., Kordic, T.: Sexuality and homelessness in Los Angeles public schools. Am. J. Pub. Health 102(2), 200a–201a (2012)CrossRefGoogle Scholar
  21. 21.
    Rice, E., Tulbert, E., Cederbaum, J., Adhikari, A.B., Milburn, N.G.: Mobilizing homeless youth for HIV prevention: a social network analysis of the acceptability of a face-to-face and online social networking intervention. Health Educ. Res. 27(2), 226 (2012)CrossRefGoogle Scholar
  22. 22.
    Ross, S., Pineau, J., Paquet, S., Chaib-Draa, B.: Online planning algorithms for POMDPs. J. Artif. Intell. Res. 32, 663–704 (2008)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Schneider, J.A., Zhou, A.N., Laumann, E.O.: A new HIV prevention network approach: sociometric peer change agent selection. Soc. Sci. Med. 125, 192–202 (2015)CrossRefGoogle Scholar
  24. 24.
    Seegebarth, B., Müller, F., Schattenberg, B., Biundo, S.: Making hybrid plans more clear to human users-a formal approach for generating sound explanations. In: Twenty-Second International Conference on Automated Planning and Scheduling (2012)Google Scholar
  25. 25.
    Silver, D., Veness, J.: Monte-Carlo planning in large POMDPs. In: Advances in Neural Information Processing Systems, pp. 2164–2172 (2010)Google Scholar
  26. 26.
    Spaan, M.T., Vlassis, N.: Perseus: randomized point-based value iteration for POMDPs. J. Artif. Intell. Res. 24, 195–220 (2005)zbMATHGoogle Scholar
  27. 27.
    Tang, Y., Xiao, X., Shi, Y.: Influence maximization: near-optimal time complexity meets practical efficiency. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 75–86. ACM (2014)Google Scholar
  28. 28.
    Valente, T.W., Pumpuang, P.: Identifying opinion leaders to promote behavior change. Health Educ. Behav. 3, 881–896 (2007)Google Scholar
  29. 29.
    Yadav, A., Chan, H., Jiang, A., Xu, H., Rice, E., Tambe, M.: Using social networks to aid homeless shelters: dynamic influence maximization under uncertainty - an extended version. In: Proceedings of the Fifteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016) (2016)Google Scholar
  30. 30.
    Yadav, A., Marcolino, L., Rice, E., Petering, R., Winetrobe, H., Rhoades, H., Tambe, M., Carmichael, H.: Preventing HIV spread in homeless populations using PSINET. In: Proceedings of the Twenty-Seventh Conference on Innovative Applications of Artificial Intelligence (IAAI-15) (2015)Google Scholar
  31. 31.
    Yan, Q., Guo, S., Yang, D.: Influence maximizing and local influenced community detection based on multiple spread model. In: Tang, J., King, I., Chen, L., Wang, J. (eds.) ADMA 2011. LNCS (LNAI), vol. 7121, pp. 82–95. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25856-5_7 CrossRefGoogle Scholar
  32. 32.
    Young, S.D., Rice, E.: Online social networking technologies, HIV knowledge, and sexual risk and testing behaviors among homeless youth. AIDS Behav. 15(2), 253–260 (2011)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Amulya Yadav
    • 1
    Email author
  • Hau Chan
    • 2
  • Albert Jiang
    • 2
  • Eric Rice
    • 1
  • Ece Kamar
    • 3
  • Barbara Grosz
    • 4
  • Milind Tambe
    • 1
  1. 1.University of Southern CaliforniaLos AngelesUSA
  2. 2.Trinity UniversitySan AntonioUSA
  3. 3.Microsoft ResearchRedmondUSA
  4. 4.Harvard UniversityCambridgeUSA

Personalised recommendations