Convergence Acceleration for Multiobjective Sparse Reconstruction via Knowledge Transfer

  • Bai YanEmail author
  • Qi Zhao
  • J. Andrew Zhang
  • Yonghui Li
  • Zhihai Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11411)


Multiobjective sparse reconstruction (MOSR) methods can potentially obtain superior reconstruction performance. However, they suffer from high computational cost, especially in high-dimensional reconstruction. Furthermore, they are generally implemented independently without reusing prior knowledge from past experiences, leading to unnecessary computational consumption due to the re-exploration of similar search spaces. To address these problems, we propose a sparse-constraint knowledge transfer operator to accelerate the convergence of MOSR solvers by reusing the knowledge from past problem-solving experiences. Firstly, we introduce the deep nonlinear feature coding method to extract the feature mapping between the search of the current problem and a previously solved MOSR problem. Through this mapping, we learn a set of knowledge-induced solutions which contain the search experience of the past problem. Thereafter, we develop and apply a sparse-constraint strategy to refine these learned solutions to guarantee their sparse characteristics. Finally, we inject the refined solutions into the iteration of the current problem to facilitate the convergence. To validate the efficiency of the proposed operator, comprehensive studies on extensive simulated signal reconstruction are conducted.


Sparse reconstruction Multiobjective evolutionary algorithm Learning Knowledge transfer 



This work was supported by the China Scholarship Council under Grant 201706540025.


  1. 1.
    Bader, J., Zitzler, E.: HypE: an algorithm for fast hypervolume-based many-objective optimization. Evol. Comput. 19(1), 45–76 (2011)CrossRefGoogle Scholar
  2. 2.
    Candès, E.J., Wakin, M.B.: An introduction to compressive sampling. IEEE Signal Process. Mag. 25(2), 21–30 (2008)CrossRefGoogle Scholar
  3. 3.
    Chen, M., Xu, Z., Weinberger, K., Sha, F.: Marginalized denoising autoencoders for domain adaptation. In: Proceedings of the 29th International Conference on Machine Learning (2012)Google Scholar
  4. 4.
    Combettes, P.L., Wajs, V.R.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4(4), 1168–1200 (2005)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  6. 6.
    Dong, Z., Zhu, W.: Homotopy methods based on \(l_0\)-norm for compressed sensing. IEEE Trans. Neural Netw. Learn. Syst. 29(4), 1132–1146 (2018)CrossRefGoogle Scholar
  7. 7.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Feng, L., Ong, Y.S., Jiang, S., Gupta, A.: Autoencoding evolutionary search with learning across heterogeneous problems. IEEE Trans. Evol. Comput. 21(5), 760–772 (2017)CrossRefGoogle Scholar
  9. 9.
    Gretton, A., Borgwardt, K.M., Rasch, M., Schölkopf, B., Smola, A.J.: A kernel method for the two-sample-problem. In: Proceedings of the Conference on Neural Information Processing System, pp. 513–520 (2007)Google Scholar
  10. 10.
    Gupta, A., Ong, Y.S., Feng, L.: Insights on transfer optimization: because experience is the best teacher. IEEE Trans. Emerg. Topics Comput. Intell. 2(1), 51–64 (2018)CrossRefGoogle Scholar
  11. 11.
    Jiang, M., Huang, Z., Liming, Q., Huang, W., et al.: Transfer learning based dynamic multiobjective optimization algorithms. IEEE Trans. Evol. Comput. 22(4), 501–514 (2017)CrossRefGoogle Scholar
  12. 12.
    Jiao, Y., Jin, B., Lu, X.: Iterative soft/hard thresholding with homotopy continuation for sparse recovery. IEEE Signal Process. Lett. 24(6), 784–788 (2017)CrossRefGoogle Scholar
  13. 13.
    Li, H., Zhang, Q., Deng, J., Xu, Z.B.: A preference-based multiobjective evolutionary approach for sparse optimization. IEEE Trans. Neural Netw. Learn. Syst. 29(5), 1716–1731 (2018)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Li, L., Yao, X., Stolkin, R., Gong, M., He, S.: An evolutionary multiobjective approach to sparse reconstruction. IEEE Trans. Evol. Comput. 18(6), 827–845 (2014)CrossRefGoogle Scholar
  15. 15.
    Liu, C., Zhao, Q., Yan, B., Elsayed, S., Ray, T., Sarker, R.: Adaptive sorting-based evolutionary algorithm for many-objective optimization. IEEE Trans. Evol. Comput. (in press).
  16. 16.
    Pan, S.J., Tsang, I.W., Kwok, J.T., Yang, Q.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)CrossRefGoogle Scholar
  17. 17.
    Sentelle, C.G., Anagnostopoulos, G.C., Georgiopoulos, M.: A simple method for solving the SVM regularization path for semidefinite kernels. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 709–722 (2016)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Steinwart, I.: On the influence of the kernel on the consistency of support vector machines. J. Mach. Learn. Res. 2(Nov), 67–93 (2001)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Wei, P., Ke, Y., Goh, C.K.: Deep nonlinear feature coding for unsupervised domain adaptation. In: IJCAI, pp. 2189–2195 (2016)Google Scholar
  20. 20.
    Yan, B., Zhao, Q., Wang, Z., Zhang, J.A.: Adaptive decomposition-based evolutionary approach for multiobjective sparse reconstruction. Inf. Sci. 462, 141–159 (2018)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Yan, B., Zhao, Q., Wang, Z., Zhao, X.: A hybrid evolutionary algorithm for multiobjective sparse reconstruction. Signal Image Video P. 11, 993–1000 (2017)CrossRefGoogle Scholar
  22. 22.
    Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)CrossRefGoogle Scholar
  23. 23.
    Zhou, Y., Kwong, S., Guo, H., Zhang, X., Zhang, Q.: A two-phase evolutionary approach for compressive sensing reconstruction. IEEE Trans. Cybern. 47(9), 2651–2663 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Bai Yan
    • 1
    Email author
  • Qi Zhao
    • 2
  • J. Andrew Zhang
    • 3
  • Yonghui Li
    • 4
  • Zhihai Wang
    • 5
  1. 1.Institute of Laser EngineeringBeijing University of TechnologyBeijingChina
  2. 2.College of Economics and ManagementBeijing University of TechnologyBeijingChina
  3. 3.Global Big Data Technologies CentreUniversity of Technology SydneySydneyAustralia
  4. 4.School of Electrical and Information EngineeringUniversity of SydneySydneyAustralia
  5. 5.Key Laboratory of Optoelectronics Technology, Ministry of EducationBeijing University of TechnologyBeijingChina

Personalised recommendations