Skip to main content
SpringerLink
Log in

Incorporating prior knowledge and multi-kernel into linear programming support vector regression

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

This paper proposes a multi-kernel linear program support vector regression with prior knowledge in order to obtain an accurate data-driven model in the case of an insufficient amount of measured data. In the algorithm, multiple feature spaces have been utilized to incorporate multi-kernel functions into the framework of linear programming support vector regression (LPSVR), and then the prior knowledge which may be exact or biased from a calibrated physical simulator has also been incorporated into LPSVR by modifying optimization formulations. Moreover, a strategy of parameter selections for the proposed algorithm has been presented to facilitate practical applications. Some experiments from a synthetic example, a microstrip antenna and six-pole microwave filter have been carried out, and the experimental results show that the proposed algorithm can obtain a satisfactory data-based model in the case of the scarcity of measured data. The proposed algorithm shows great potentialities in some applications where the experimental data are insufficient for an accurate data-driven model and the prior knowledge from a calibrated physical simulator of practical applications is available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Bach FR, Lanckriet GRG, Jordan MI Multiple kernel learning, conic duality, and the SMO algorithm. In: Banff, Alta, Canada, 2004. Proceedings, Twenty-First International conference on machine learning, ICML 2004. Association for Computing Machinery, pp 41–48

  • Bloch G, Lauer F, Colin G, Chamaillard Y (2008) Support vector regression from simulation data and few experimental samples. Inf Sci 178(20):3813–3827

    Article  Google Scholar 

  • Cherkassky V, Yunqian M (2004) Practical selection of SVM parameters and noise estimation for SVM regression. Neural Netw 17:113–126

    Article  MATH  Google Scholar 

  • Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. Trans ASME J Mech Design 127(6):1077–1087

    Article  Google Scholar 

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge

  • Huang CF (2012) A hybrid stock selection model using genetic algorithms and support vector regression. Appl Soft Comput 12(2):807–818

    Article  Google Scholar 

  • Lanckriet GRG, Cristianini N, Bartlett P, El Ghaoui L, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72

  • Lauer F, Bloch G (2008a) Incorporating prior knowledge in support vector machines for classification: a review. Neurocomputing 71(7–9):1578–1594

    Article  Google Scholar 

  • Lauer F, Bloch G (2008b) Incorporating prior knowledge in support vector regression. Mach Learn 70(1):89–118

    Article  Google Scholar 

  • Lu Z, Sun J (2009) Non-Mercer hybrid kernel for linear programming support vector regression in nonlinear systems identification. Appl Soft Comput 9(1):94–99

    Article  Google Scholar 

  • Lu Z, Sun J, Butts KR (2009) Linear programming support vector regression with wavelet kernel: a new approach to nonlinear dynamical systems identification. Math Comput Simul 79(7):2051–2063

    Article  MATH  MathSciNet  Google Scholar 

  • Mangasarian OL, Wild EW (2007) Nonlinear knowledge in kernel approximation. IEEE Trans Neural Netw 18(1):300–306

    Article  Google Scholar 

  • Mingqing Hu, Chen Yiqiang (2009) Building sparse multiple-kernel SVM classifiers. IEEE Trans Neural Netw 20(5):827–839

    Article  Google Scholar 

  • Muller KR, Mika S, Ratsch G, Tsuda K, Scholkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181–201

    Article  Google Scholar 

  • Nguyen C-V, Tay DBH (2008) Regression using multikernel and semiparametric support vector algorithms. IEEE Signal Process Lett 15:481–484

    Article  Google Scholar 

  • Mangasarian Olvi L, Shavlik Jude W (2004) Knowledge-based kernel approximation. J Mach Learn Res 5:1127–1141

    MATH  Google Scholar 

  • Pasolli L, Notarnicola C, Bruzzone L (2012) Multi-objective parameter optimization in support vector regression: general formulation and application to the retrieval of soil moisture from remote sensing data. IEEE J Select Topics Appl Earth Observ Remote Sens 5(5):1495–1508

    Article  Google Scholar 

  • Phienthrakul T, Kijsirikul B (2010) Evolutionary strategies for hyperparameters of support vector machines based on multi-scale radial basis function kernels. Soft Comput 14(7):681–699

    Article  Google Scholar 

  • Pozdnoukhov A, Kanevski M (2008) Multi-scale support vector algorithms for hot spot detection and modelling. Stoch Environ Res Risk Assess 22(5):647–660

    Article  MathSciNet  Google Scholar 

  • Qiu S, Lane T (2009) A framework for multiple kernel support vector regression and its applications to siRNA efficacy prediction. IEEE/ACM Trans Comput Biol Bioinform 6(2):190–199

    Article  Google Scholar 

  • Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521

    MATH  MathSciNet  Google Scholar 

  • Sagiroglu S, Guney K, Erler M (1999) Calculation of bandwidth for electrically thin and thick rectangular microstrip antennas with the use of multilayered perceptrons. Int J RF Microwave Comput Aided Eng 9(3):277–286

    Article  Google Scholar 

  • Sanchez AVD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20

    Article  Google Scholar 

  • Smola AJ, Sch\(\ddot{\rm o}\)lkopf B (2004) A tutorial on support vector regression. Stat Comput 14:199–220

  • Smola A, Schoelkopf B, Raetsch G (1999) Linear programs for automatic accuracy control in regression. In: Proceedings of the Ninth International Conference on Artificial Neural Networks, Edinburgh, UK. IEE Conference Publication. IEE, pp 575–580

  • Smola A, Scholkopf B (2002) Learning with Kernels. MIT Press, Cambridge

    Google Scholar 

  • Sonnenburg S, Ratsch G, Schafer C, Scholkopf B (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531–1565

    MATH  MathSciNet  Google Scholar 

  • Subrahmanya N, Shin YC (2010) Sparse multiple kernel learning for signal processing applications. IEEE Trans Pattern Anal Mach Intell 32(5):788–798

  • Trnka P, Havlena V (2009) Subspace like identification incorporating prior information. Automatica 45(4):1086–1091

    Article  MATH  MathSciNet  Google Scholar 

  • Vapnik VN (1995) The nature of statistical learning theory. Springer, Berlin

    Book  MATH  Google Scholar 

  • Varma M, Babu BR (2009) More generality in efficient multiple kernel learning. In, Montreal, QC, Canada, 2009. Proceedings of the 26th International Conference on machine learning, ICML. Omnipress, pp 1065–1072

  • Zhao YP, Sun JG (2011) Multikernel semiparametric linear programming support vector regression. Expert Systems Appl 38:1611–1618

  • Yu Y, Qian F (2008) Multi-scale linear programming support vector regression for ethylene distillation modeling. In: Chongqing, China. Proceedings of the World Congress on Intelligent Control and Automation (WCICA). Institute of Electrical and Electronics Engineers Inc., pp 1548–1552

  • Zheng D, Wang J, Zhao Y (2006) Non-flat function estimation with a multi-scale support vector regression. Neurocomputing 70(1–3): 420–429

  • Zhou J, Huang J, Xue X (2011) Modeling and optimization for electromechanical coupling of cavity filters based on support vector regression. (Dianzi Yu Xinxi Xuebao) J Electron Inf Technol 33(11):2780–2784

  • Zhou J, Duan B, Huang J (2010) Influence and tuning of tunable screws for microwave filters using least squares support vector regression. Int J RF Microwave Comput Aided Eng 20:422–429

  • Zhou J, Huang J (2010) Incorporating priori knowledge into linear programming support vector regression. In: 2010 IEEE International Conference on Intelligent Computing and Integrated Systems, ICISS2010, October 22, 2010–October 24, 2010, Guilin, China. IEEE Computer Society, pp 591–595

  • Zhou J, Huang J (2013) Intelligent tuning for microwave filters based on multi-kernel machine learning model. In: 5th IEEE International Symposium on Microwave, Antenna, Propagation and EMC Technologies for Wireless Communications, MAPE 2013, October 29, 2013–October 31, 2013. Chengdu, China, pp 259–266

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (Grant No. 51305323, 51305322 and 51035006) and the Fundamental Research Funds for the Central Universities (Grant No JB140404).

Author information

Authors and Affiliations

  1. Key Laboratory of Electronic Equipment Structure Design of Ministry of Education, Xidian University, Xi’an, Shaanxi, People’s Republic of China

    Jinzhu Zhou, Baoyan Duan, Jin Huang & Na Li

Authors
  1. Jinzhu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baoyan Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Na Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinzhu Zhou.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, J., Duan, B., Huang, J. et al. Incorporating prior knowledge and multi-kernel into linear programming support vector regression. Soft Comput 19, 2047–2061 (2015). https://doi.org/10.1007/s00500-014-1390-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-014-1390-x

Keywords

Search

Navigation