The More the Merrier - Federated Learning from Local Sphere Recommendations

  • Bernd Malle
  • Nicola Giuliani
  • Peter Kieseberg
  • Andreas Holzinger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10410)

Abstract

With Google’s Federated Learning & Facebook’s introduction of client-side NLP into their chat service, the era of client-side Machine Learning is upon us. While interesting ML approaches beyond the realm of toy examples were hitherto confined to large data-centers and powerful GPU’s, exponential trends in technology and the introduction of billions of smartphones enable sophisticated processing swarms of even hand-held devices. Such approaches hold several promises: 1. Without the need for powerful server infrastructures, even small companies could be scalable to millions of users easily and cost-efficiently; 2. Since data only used in the learning process never need to leave the client, personal information can be used free of privacy and data security concerns; 3. Since privacy is preserved automatically, the full range of personal information on the client device can be utilized for learning; and 4. without round-trips to the server, results like recommendations can be made available to users much faster, resulting in enhanced user experience. In this paper we propose an architecture for federated learning from personalized, graph based recommendations computed on client devices, collectively creating & enhancing a global knowledge graph. In this network, individual users will ‘train’ their local recommender engines, while a server-based voting mechanism aggregates the developing client-side models, preventing over-fitting on highly subjective data from tarnishing the global model.

Keywords

Machine Learning Federated ML Interactive ML The local sphere Graph based personalized recommenders Distributed bagging 

References

  1. 1.
    Leskovec, J., Singh, A., Kleinberg, J.: Patterns of influence in a recommendation network. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS, vol. 3918, pp. 380–389. Springer, Heidelberg (2006). doi: 10.1007/11731139_44 CrossRefGoogle Scholar
  2. 2.
    Leskovec, J., McGlohon, M., Faloutsos, C., Glance, N., Hurst, M.: Patterns of cascading behavior in large blog graphs. In: Proceedings of the 2007 SIAM International Conference on Data Mining, pp. 551–556. SIAM (2007)Google Scholar
  3. 3.
    Leskovec, J., Faloutsos, C.: Sampling from large graphs. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 631–636. ACM (2006)Google Scholar
  4. 4.
    Al-Fares, M., Loukissas, A., Vahdat, A.: A scalable, commodity data center network architecture. In: ACM SIGCOMM Computer Communication Review, vol. 38, pp. 63–74. ACM (2008)Google Scholar
  5. 5.
    Holzinger, A.: Introduction to machine learning & knowledge extraction (make). Mach. Learn. Knowl. Extr. 1(1), 1–20 (2017)CrossRefGoogle Scholar
  6. 6.
    McMahan, H.B., Moore, E., Ramage, D., Hampson, S. et al.: Communication-efficient learning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629 (2016)
  7. 7.
    Konečnỳ, J., McMahan, H.B., Yu, F.X., Richtárik, P., Suresh, A.T., Bacon, D.: Federated learning: strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016)
  8. 8.
    Konečnỳ, J., McMahan, H.B., Ramage, D., Richtárik, P.: Federated optimization: distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 (2016)
  9. 9.
    Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H.B., Patel, S., Ramage, D., Segal, A., Seth, K.: Practical secure aggregation for privacy preserving machine learning. Cryptology ePrint Archive, Report 2017/281 (2017). http://eprint.iacr.org/2017/281
  10. 10.
    Wainwright, M.J., Jordan, M.I., Duchi, J.C.: Privacy Aware Learning. In: Advances in Neural Information Processing Systems, pp. 1430–1438 (2012)Google Scholar
  11. 11.
    Malle, B., Kieseberg, P., Weippl, E., Holzinger, A.: The right to be forgotten: towards machine learning on perturbed knowledge bases. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 251–266. Springer, Cham (2016). doi: 10.1007/978-3-319-45507-5_17 CrossRefGoogle Scholar
  12. 12.
    Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Springer Brain Inform. (BRIN) 3(2), 119–131 (2016)CrossRefGoogle Scholar
  13. 13.
    Kieseberg, P., Malle, B., Frhwirt, P., Weippl, E., Holzinger, A.: A tamper-proof audit and control system for the doctor in the loop. Brain Inform. 3(4), 1–11 (2016)CrossRefGoogle Scholar
  14. 14.
    Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive Machine Learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-45507-5_6 CrossRefGoogle Scholar
  15. 15.
    Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)MATHGoogle Scholar
  16. 16.
    Zhu, N.Q.: Data visualization with D3. Js cookbook. Packt Publishing Ltd, UK (2013)Google Scholar
  17. 17.
    Alani, H., Kim, S., Millard, D.E., Weal, M.J., Hall, W., Lewis, P.H., Shadbolt, N.R.: Automatic ontology-based knowledge extraction from web documents. IEEE Intell. Syst. 18(1), 14–21 (2003)CrossRefGoogle Scholar
  18. 18.
    Holzinger, A., Plass, M., Holzinger, K., Crisan, G.C., Pintea, C.-M., Palade, V.: A glass-box interactive machine learning approach for solving NP-hard problems with the human-in-the-loop. arXiv preprint arXiv:1708.01104 (2017)

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Bernd Malle
    • 1
    • 2
  • Nicola Giuliani
    • 1
  • Peter Kieseberg
    • 1
    • 2
  • Andreas Holzinger
    • 1
  1. 1.Holzinger Group HCI-KDD Institute for Medical Informatics, Statistics and DocumentationMedical University GrazGrazAustria
  2. 2.SBA Research GmbHWienAustria

Personalised recommendations