Frontiers of Chemical Science and Engineering

, Volume 13, Issue 3, pp 599–607 | Cite as

Modeling of oil near-infrared spectroscopy based on similarity and transfer learning algorithm

  • Yifei Wang
  • Kai Wang
  • Zhao ZhouEmail author
  • Wenli DuEmail author
Research Article


Near-infrared spectroscopy mainly reflects the frequency-doubled and total-frequency absorption information of hydrogen-containing groups (O‒H, C‒H, N‒H, S‒H) in organic molecules for near-infrared lights with different wavelengths, so it is applicable to testing of most raw materials and products in the field of petrochemicals. However, the modeling process needs to collect a large number of laboratory analysis data. There are many oil sources in China, and oil properties change frequently. Modeling of each raw material is not only unfeasible but also will affect its engineering application efficiency. In order to achieve rapid modeling of near-infrared spectroscopy and based on historical data of different crude oils under different detection conditions, this paper discusses about the feasibility of the application of transfer learning algorithm and makes it possible that transfer learning can assist in rapid modeling using certain historical data under similar distributions under a small quantity of new data. In consideration of the requirement of transfer learning for certain similarity of different datasets, a transfer learning method based on local similarity feature selection is proposed. The simulation verification of spectral data of 13 crude oils measured by three different probe detection methods is performed. The effectiveness and application scope of the transfer modeling method under different similarity conditions are analyzed.


near-infrared spectroscopy transfer learning similarity modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The financial supports from the National Natural Science Foundation of China (Grant No. 61590923), National Science Fund for Distinguished Young Scholars (No. 61725301), the Fundamental Research Funds for the Central Universities and the Programme of Introducing Talents of Discipline to Universities (the 111 Project) under Grant B17017 are gratefully acknowledged.


  1. 1.
    Yan Y L. The Basis and Application of Near Infrared Spectroscopy. Beijing: China Light Industry Press, 2005, 286–564 (in Chinese)Google Scholar
  2. 2.
    Lu W Z. Modern Near Infrared Spectroscopy Analysis Technology. Beijing: China Petrochemical Press, 2007, 14–26 (in Chinese)Google Scholar
  3. 3.
    Workman J Jr. A brief review of near infrared in petroleum product analysis. Journal of Near Infrared Spectroscopy, 1996, 4(1): 69CrossRefGoogle Scholar
  4. 4.
    Oja H. Multivariate Linear Regression. New York: Springer, 2010, 183–200Google Scholar
  5. 5.
    Tormod N, Harald M. Principal component regression in NIR analysis: Viewpoint, background details and selection of components. Journal of Chemometrics, 1988, 2(2): 155–167CrossRefGoogle Scholar
  6. 6.
    Geladi P, Kowalski B R. Partial least-squares regression: A tutorial. Analytica Chimica Acta, 1985, 185(86): 1–17Google Scholar
  7. 7.
    He Y, Li X, Deng X. Discrimination of varieties of tea using near infrared spectroscopy by principal component analysis and BP model. Journal of Food Engineering, 2007, 79(4): 1238–1242CrossRefGoogle Scholar
  8. 8.
    Shimodaira H. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 2000, 90(2): 227–244CrossRefGoogle Scholar
  9. 9.
    He Y. Modelling of near-infrared spectroscopy based on semisupervised learning and transfer learning. Dissertation for the Doctoral Degree. Shandong: Ocean University of China, 2012Google Scholar
  10. 10.
    Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359CrossRefGoogle Scholar
  11. 11.
    Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning. Journal of Big Data, 2016, 3(1): 9CrossRefGoogle Scholar
  12. 12.
    Tan B, Song Y, Zhong E, Yang Q. Transitive transfer learning. Acm Sigkdd International Conference on Knowledge Discovery & Data Mining, 2015, 1155–1164CrossRefGoogle Scholar
  13. 13.
    Tan B, Zhang Y, Pan S J, Yang Q. Distant domain transfer learning. Association for the Advance of Artificial Intelligence, 2017, 2604–2610Google Scholar
  14. 14.
    Gao J. The application of near infrared spectroscopy in oil quality analysis. Dissertation for the Master Degree. Jiangsu: Nanjing Tech University, 2005, 11–12Google Scholar
  15. 15.
    Karstang T V, Valheim K. Multivariate prediction and background correction using local modeling and derivative spectroscopy. Analytical Chemistry, 1996, 63(8): 767–772CrossRefGoogle Scholar
  16. 16.
    Zhao C H, Tian M H, Li J W. Research progress on spectral similarity metrics. Journal of Harbin Engineering University, 2017, 38(8): 1179–1189 (in Chinese)Google Scholar
  17. 17.
    Wang C, Gong M, Zhang M, Chan Y. Unsupervised hyperspectral image band selection via column subset selection. IEEE Geoscience and Remote Sensing Letters, 2015, 12(7): 1411–1415CrossRefGoogle Scholar
  18. 18.
    Schlamm A, Messinger D. Improved detection clustering of hyperspectral image date by preprocessing with a euclidean distance transformation. WHISPERS, 2011, 1(2): 1–4Google Scholar
  19. 19.
    Zhong Y, Lin X, Zhang L. A support vector conditional random fields classifier with a Mahalanobis distance boundary constraint for high spatial resolution remote sensing imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(4): 1314–1330CrossRefGoogle Scholar
  20. 20.
    Kruse F A, Lefkoff A B, Boardman J W, Heidebrecht K B, Shapiro A T, Barloon P J. The spectral image processing systems (SIPS)-interactive visualization and analysis of imaging spectrometer data. Aip Conference, 1993, 283(1): 192–201CrossRefGoogle Scholar
  21. 21.
    Chang C I. Spectral information divergence for hyperspectral image analysis. IEEE International Geoscience & Remote Sensing Symposium, 1999, 509–511Google Scholar
  22. 22.
    Pan S J, Kwok J T, Yang Q, Pan J J. Adaptive localization in a dynamic WiFi environment through multi-view learning. Association for the Advance of Artificial Intelligence, 2007, 1108–1113Google Scholar
  23. 23.
    Granahan J C, Sweet J N. An evaluation of atmospheric correction techniques using the spectral similarity scale. IEEE International Geoscience & Remote Sensing Symposium, 2001, 2022–2024Google Scholar
  24. 24.
    Pan S J, Tsang I W, Kwok J T, Yang Q. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 2011, 22(2): 199–210CrossRefGoogle Scholar
  25. 25.
    Breiman L. Bagging predictors. Machine Learning, 1996, 24(2): 123–140Google Scholar
  26. 26.
    Dai W Y, Yang Q, Xue G R, Yu Y. Boosting for transfer learning. International Conference on Machine Learning, Corvalis, 2007, 238 (6): 193–200CrossRefGoogle Scholar
  27. 27.
    Freund Y, Schapire R E. A decision-theoretic generalization of online learning and an application to boosting. Journal of Computer and System Sciences, 1997, 55(1): 119–139CrossRefGoogle Scholar
  28. 28.
    Zhou S H, Du W L. Modeling of ethylene cracking furnace yields based on transfer learning. CIESC Journal, 2014, 65(12): 4921–4928Google Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Key Laboratory of Advanced Control and Optimization for Chemical Processes (Ministry of Education)East China University of Science and TechnologyShanghaiChina
  2. 2.School of information science and engineeringEast China University of Science and TechnologyShanghaiChina

Personalised recommendations