Distributed regression estimation with incomplete data in multi-agent networks

Science China Information Sciences volume 61, Article number: 092202 (2018) Cite this article

Abstract

In this paper, distributed regression estimation problem with incomplete data in a time-varying multi-agent network is investigated. Regression estimation is carried out based on local agent information with incomplete in the non-ignorable mechanism. By virtue of gradient-based design and adaptive filter, a distributed algorithm is proposed to deal with a regression estimation problem with incomplete data. With the help of convex analysis and stochastic approximation techniques, the exact convergence is obtained for the proposed algorithm with incomplete data and a jointly-connected multi-agent topology. Moreover, online regret analysis is also given for real-time learning. Then, simulations for the proposed algorithm are also given to demonstrate how it can solve the estimation problem in a distributed way, even when the network configuration is time-varying.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

US$ 39.95

Tax calculation will be finalised during checkout.

References

  1. 1

    Nedić A, Ozdaglar A. Distributed subgradient methods for multi-agent optimization. IEEE Trans Automat Control, 2009, 54: 48–61

    MathSciNet  Article  MATH  Google Scholar 

  2. 2

    Shi G, Johansson K. Robust consensus for continuous-time multiagent dynamics. SIAM J Control Optim, 2013, 51: 3673–3691

    MathSciNet  Article  MATH  Google Scholar 

  3. 3

    Zhang Y Q, Lou Y C, Hong Y G, et al. Distributed projection-based algorithms for source localization in wireless sensor networks. IEEE Trans Wirel Commun, 2015, 43: 3131–3142

    Article  Google Scholar 

  4. 4

    Feng H, Jiang Z D, Hu B, et al. The incremental subgradient methods on distributed estimations in-network. Sci China Inf Sci, 2014, 57: 092103

    Google Scholar 

  5. 5

    Lou Y C, Hong Y G, Wang S Y. Distributed continuous-time approximate projection protocols for shortest distance optimization problems. Automatica, 2016, 69: 289–297

    MathSciNet  Article  MATH  Google Scholar 

  6. 6

    Yi P, Hong Y G, Liu F. Initialization-free distributed algorithms for optimal resource allocation with feasibility con-straints and application to economic dispatch of power systems. Automatica, 2016, 74: 259–269

    MathSciNet  Article  MATH  Google Scholar 

  7. 7

    Kokaram A C. On missing data treatment for degraded video and film archives: a survey and a new Bayesian approach. IEEE Trans Image Process, 2004, 13: 397–415

    Article  Google Scholar 

  8. 8

    Molenberghs G, Kenward M G. Missing Data in Clinical Studies. New York: Wiley, 2007

    Google Scholar 

  9. 9

    Ibrahim J G, Chen M H, Lipsitz S R, et al. Missing data methods for generalized linear models: a comparative review. J Am Stat Assoc, 2005, 100: 332–346

    MathSciNet  Article  MATH  Google Scholar 

  10. 10

    Gholami M R, Jansson M, Strom E G, et al. Diffusion estimation over cooperative multi-agent networks with missing data. IEEE Trans Signal Inf Process Netw, 2016, 2: 276–289

    MathSciNet  Article  Google Scholar 

  11. 11

    Davey A, Savla J. Statistical Power Analysis with Missing Data: A Structural Equation Modeling Approach. Oxford, UK: Routledge Academic, 2009

    Google Scholar 

  12. 12

    Ram S S, Nedić A, Veeravalli V V. Distributed stochastic subgradient projection algorithms for convex optimization. J Optim Theory Appl, 2010, 147: 516–545

    MathSciNet  Article  MATH  Google Scholar 

  13. 13

    Graybill F, Iyer H K. Regression Analysis: Concepts and Applications. California: Duxbury Press Belmont, 1994

    Google Scholar 

  14. 14

    Feng Y, Sundaram S, Vishwanathan S V N, et al. Distributed autonomous online learning: regrets and intrinsic privacy-preserving properties. IEEE Trans Knowl Data Eng, 2013, 25: 2483–2493

    Article  Google Scholar 

  15. 15

    Hazan E, Kale S. Beyond the regret minimization barrier: optimal algorithms for stochastic strongly-convex optimiza-tion. J Mach Learn Res, 2014, 15: 2489–2512

    MathSciNet  MATH  Google Scholar 

  16. 16

    Shamir O, Zhang T. Stochastic gradient descent for non-smooth optimization: convergence results and optimal aver-aging schemes. In: Proceedings of International Conference on Machine Learning, Edinburgh, 2012. 71–79

    Google Scholar 

  17. 17

    Towfic Z J, Chen J S, Sayed A H. On distributed online classification in the midst of concept drifts. Neurocomputing, 2013, 112: 138–152

    Article  Google Scholar 

  18. 18

    Widrow B, Stearns S D. Adaptive Signal Processing. Cliffs: Prentice-Hall, 1985. 1–32

    Google Scholar 

  19. 19

    Sayed A H. Adaptation, learning, and optimization over networks. Found Trends Mach Learn, 2014, 7: 311–801

    Article  MATH  Google Scholar 

  20. 20

    Sayed A H, Tu S Y, Chen J S, et al. Diffusion strategies for adaptation and learning. IEEE Signal Proc Mag, 2013, 30: 155–171

    Article  Google Scholar 

  21. 21

    Polyak B T. Introduction to Optimization. New York: Optimization Software Inc., 1983. 2–8

    Google Scholar 

  22. 22

    Godsil C, Royle G. Algebraic Graph Theory. New York: Springer-Verlag, 2001. 1–18

    Google Scholar 

  23. 23

    Ferguson T S. A Course in Large Sample Theory. London: Chapman and Hall Ltd., 1996. 3–4

    Google Scholar 

  24. 24

    Durrett R. Probability Theory and Examples. Camberidge, UK: Camberidge Press, 2010. 328–347

    Google Scholar 

  25. 25

    Enders C K. Applied Missing Data Analysis. New York: The Guilford Press, 2010

    Google Scholar 

  26. 26

    Kushner H J, Yin G. Stochastic Approximation and Recursive Algorithms and Applications. New York: Springer-Verlag, 1997. 117–157

    Google Scholar 

  27. 27

    Widrow B, Mccool J, Larimore M G, et al. Stationary and nonstationary learning characteristics of the LMS adaptive filter. Proc IEEE, 1976, 64: 1151–1162

    MathSciNet  Article  Google Scholar 

  28. 28

    Yi P, Hong Y G. Stochastic sub-gradient algorithm for distributed optimization with random sleep scheme. Control Theory Technol, 2015, 13: 333–347

    MathSciNet  Article  MATH  Google Scholar 

  29. 29

    Larsen R J, Max M L. An Introduction to Mathematical Statistics and Its Applications. 4th ed. New York: Pearson, 2006. 221–280

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Key Research and Development Program of China (Grant No. 2016YFB0901902) and National Natural Science Foundation of China (Grant Nos. 61573344, 61333001, 61374168).

Author information

Affiliations

  1. University of Chinese Academy of Sciences, Beijing, 100049, China

    Yinghui Wang & Peng Lin

  2. Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, 100190, China

    Yinghui Wang, Peng Lin & Yiguang Hong

Authors
  1. Yinghui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yiguang Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yinghui Wang.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Lin, P. & Hong, Y. Distributed regression estimation with incomplete data in multi-agent networks. Sci. China Inf. Sci. 61, 092202 (2018). https://doi.org/10.1007/s11432-016-9173-8

Download citation

Keywords

  • multi-agent systems
  • time-varying network
  • estimation with incomplete data
  • online learning
  • stochastic approximation