Skip to main content
SpringerLink
Log in

Comprehensive Review of Orthogonal Regression and Its Applications in Different Domains

  • Review article
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

Orthogonal regression is one of the prominent approaches for linear regression used to adjust the estimate of predictor errors. It can be considered as a least square regression with orthogonal constraints. It can maintain more discriminative constraints in the projection subspace than the least square regression and can avoid trivial solutions. In contrast to basic linear regression, orthogonal regression involves a computation error in both the answer and the predictor. Only the response variable contains the estimated error in simple regression. Orthogonal regression has also been utilized as the variable error occurs. Based on the data properties, specific models of orthogonal regression can be selected depending on whether there are calculation errors and/or equation errors. This article presents a comprehensive review of various variants of orthogonal regressions. The comparisons are drawn among the various variants of orthogonal regressions by considering various characteristics. The use of orthogonal regressions in various domains is also studied. Finally, various future directions are also presented.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Lin B, Xu B (2020) Investigating drivers of CO2 emission in China’s heavy industry: a quantile regression analysis. Energy 206:118–159

    Article  Google Scholar 

  2. Ding P, Li X (2020) Rerandomization and regression adjustment. J R Statl Soc Stat Methodol 82:241–268

    Article  MathSciNet  MATH  Google Scholar 

  3. Mulund D, Abdulazeez AM (2020) A review on linear regression comprehensive in machine learning. J Appl Sci Technol Trends 1:140–147

    Article  Google Scholar 

  4. Shrestha N (2020) Detecting multicollinearity in regression analysis. Am J Appl Math Stat 8:39–42

    Article  Google Scholar 

  5. Derwin JM, Thomas VA, Wynne RH, Coulston JW, Liknes GC, Bender S, Blinn CE, Brooks EB (2020) Estimating tree canopy cover using harmonic regression coefficients derived from multitemporal Landsat data. Int J Appl Earth Obs Geoinf 86:101

    Google Scholar 

  6. Van G, Joost R (2020) Standardized regression coefficients and newly proposed estimators for R2 in multiply imputed data. Psychometric Soc 85:185–205

    Article  MATH  Google Scholar 

  7. Alotaibi, Saud S (2021) Regression coefficients as triad scale for malware detection. Comput Electr Eng 90:106

  8. Chachi J, Taheri SM, D’Urso P (2021) Fuzzy regression analysis based on M-estimates. Expert Syst Appl 115

  9. Ye T, Liu B (2021) Uncertain hypothesis test with application to uncertain regression analysis. Fuzzy Optim Decis Mak 1–18

  10. Wu X, Xu X, Liu J, Wang H, Hu B, Nie F (2021) Supervised feature selection with orthogonal regression and feature weighting. IEEE Trans Neural Netw Learn Syst 32:1831–1838

    Article  MathSciNet  Google Scholar 

  11. Heydari MR, Salehkaleybar S, Zhang K (2021) Adverserial orthogonal regression: two non-linear regressions for casual inference. Neural Netw 143:66–73

    Article  Google Scholar 

  12. Bhat KA (2021) Study on efficiency improvement for ordinary least square and orthogonal regression: an application in chemical engineering. Adv Asp Eng Res 5:49–54

    Article  Google Scholar 

  13. Izquierdo KA, Alexander N (2020) Spatial regression and spillover effects in cluster randomized trials with count outcomes. Biometrics 77:490–505

    Article  MathSciNet  Google Scholar 

  14. Pandit P, Krishnamurthy KN, Murthy MN (2020) Algebraic and geometric basis of principal components: an overview. J Reliab Stat Stud 13:73–86

    Article  Google Scholar 

  15. Chang J, Rong Y, Magistrale H, Chen G, Xu X (2020) Earthquake rupture scaling relations for mainland China. Seismol Res Lett 91:248–261

    Article  Google Scholar 

  16. Abdalzaher MS (2020) Seismic hazard maps of Egypt based on spatially smoothed seismicity model and recent seismotectonic models. J Afr Earth Sc 170:103

    Google Scholar 

  17. Fang X, Zeng W, Zhou Y, Wang B (2020) On the total least median of squares adjustment for the pattern recognition in point clouds. Measurement 160:107

    Article  Google Scholar 

  18. Chen Z, Wang C, Zhang Y (2020) Multiple regression prediction model for cutting forces and surface roughness in micro-milling of TA2. Procedia CIRP 89:233–238

    Article  Google Scholar 

  19. Zhou T, He L, Zou Z, Du F, Wu J, Tian P (2020) Three-dimensional turning force prediction based on hybrid finite element and predictive machining theory considering edge radius and nose radius. J Manuf Process 58:1304–1317

    Article  Google Scholar 

  20. Boogaart KGVD, Filzmoser P, Hron K, Templ M, Tolosana-Delgado R (2021) Classical and robust regression analysis with compositional data. Math Geosci 53:823–858

    Article  MathSciNet  MATH  Google Scholar 

  21. Castellaro S, Mulargia F, Kagan YY (2006) Regression problems for magnitudes. Geophys J Int 913–930:165

    Google Scholar 

  22. Lolli B, Gasperini P (2012) A comparison among general orthogonal regression methods applied to earthquake magnitude conversions. Geophys J Int 1135–1151:190

    Google Scholar 

  23. Wason HR, Das R, Sharma ML (2012) Magnitude conversion problem using general orthogonal regression. Geophys J Int 190:1091–1096

    Article  Google Scholar 

  24. Das R, Wason HR, Sharma ML (2012) Magnitude conversion to unified moment magnitude using orthogonal regression relation. J Asian Earth Sci 50:44–51

    Article  Google Scholar 

  25. Das R, Wason HR, Sharma ML (2013) General orthogonal regression relations between body-wave and moment magnitudes. Seismol Res Lett 84:219–224

    Article  Google Scholar 

  26. Sharma ML, Wason HR, Das R (2016) Probabilistic seismic hazard assessment for Northeast India region. Pure Appl Geophys 173:2653–2670

    Article  Google Scholar 

  27. Nath SK, Mandal S, Adhikari MD, Maiti SK (2016) A unified earthquake catalogue for South Asia covering the period 1900–2014. Nat Hazards 85:1787–1810

    Article  Google Scholar 

  28. Wason HR, Das R, Gonzalez G, Sharma ML, Choudhary D, Lindholm C, Roy N, Salazar P (2018) Earthquake magnitude conversion problem. Bull Seismol Soc Am 108:1995–2007

    Article  Google Scholar 

  29. Wang JP, Huang D (2018) Deterministic seismic hazard assessments for Taiwan considering non-controlling seismic sources. Bull Eng Geol Env 73:635–641

    Article  Google Scholar 

  30. Sharma ML, Wason HR, Das R (2011) Homogenization of earthquake catalog for Northeast India and adjoining region. Pure Appl Geophys 169:725–731

    Google Scholar 

  31. Vankeerberghen P, Hartmann C, Smeyers-Verbeke J, Massart DL (1997) Robust orthogonal regression for the outlier detection when comparing two series of measurement results. Anal Chim Acta 344:17–28

    Article  Google Scholar 

  32. Calafiore, Crio G (2000) Outliers robustness in multivariate orthogonal regression. IEEE Trans Syst Man Cybern-Part A: Syst Hum 30:674–679

  33. Gazenb R, Fekria M (2004) Robust weighted orthogonal regression in the errors-in-variables model. J Multivar Anal 88:89–108

    Article  MathSciNet  Google Scholar 

  34. Zhang T, Leng L, Kleinman L, Zhu W (2007) Ordinary least square regression, orthogonal regression, geometric mean regression and their applications in aerosol science. J Phys: Conf Ser 78:012

    Google Scholar 

  35. Kim, Sik C (2007) Bayesian Orthogonal Least Squares (BOLS) algorithm for reverse engineering of gene regulatory networks. BMC Bioinform 8:1–15

  36. Billings SA, Wei H-L (2007) Sparse model identification using a forward orthogonal regression algorithm aided by mutual information. IEEE Trans Neural Netw 18:306–310

    Article  Google Scholar 

  37. Han M, Li D-C (2010) Orthogonal least squares based on singular value decomposition for spare basis selection. International Symposium on Neural Networks, pp 413–421

  38. Carr JR (2010) Orthogonal regression: a teaching perspective. Int J Math Educ Sci Technol 43:134–143

    Article  MathSciNet  MATH  Google Scholar 

  39. Jin Y, Guo Liang D, Chu B, Zhu CA (2013) Comparison of orthogonal regression and least squares in measurement error modeling for prediction of material property. Adv Mater Res 661:166–170

  40. Nei F, Li X, Rui Z (2017) Feature selection under regularized orthogonal least square regression with optimal scaling. Neurocomput Sci Direct 273:547–553

    Google Scholar 

  41. Keles T (2018) Comparison of classical least squares and orthogonal regression in measurement error models. Int Online J Educ Sci 10:200–214

    Google Scholar 

  42. Carroll RJ, Ruppert D (1996) The use and misuse of orthogonal regression in linear errors-in-variables models. Am Stat Assoc 50:1–6

    Google Scholar 

  43. Van Aelst S, Wangb X, Zamar RH, Zhu R (2004) Linear grouping using orthogonal regression. Comput Stat Data Anal, Sci Direct 50:1287–1312

    Article  MathSciNet  MATH  Google Scholar 

  44. Faxekas I, Kukush A, Zwanzig S (2004) Correction of nonlinear orthogonal regression estimator. Ukr Math J 56:1308–1330

    Article  MathSciNet  Google Scholar 

  45. Firat M, Mete HO (2010) Stamping process design using FEA in conjuction with orthogonal regression. Finite Elem Anal Des, Elsevier 46:992–1000

    Article  Google Scholar 

  46. Boone EL, Brooks JP (2011) Outlier-resistant L1 orthogonal regression via the reformulation—linearization technique. Adv Oper Res 1–18:2011

    Google Scholar 

  47. Haeckel R, Wosniok W, Klauke R (2013) Comparison of ordinary linear regression, orthogonal regression, standardized principal component analysis, Deming and Passing-Bablok approach for method validation in laboratory medicine. Laboratoriumsmedizin 37:147–163

    Article  Google Scholar 

  48. Souza RC, Leite SC, Borges CC, Neto RF (2013) Online algorithm based on support vectors for orthogonal regression. Pattern Recogn Lett 34:1394–1404

    Article  Google Scholar 

  49. Tang X, Zhang L (2018) Stability orthogonal regression for system identification. Pattern Recogn Lett 34:1394–1404

    Google Scholar 

  50. Boggs PT, Spiegelman CH, Donaldson JR, Schnabel RB (1988) A computational examination of orthogonal. J Econom 38:169–201

    Article  Google Scholar 

  51. Ness LA, Van Ness J (1988) A routine for converting regression algorithms into corresponding orthogonal regression algorithms. ACM Trans Math Softw 14:76–87

    Article  MathSciNet  MATH  Google Scholar 

  52. Ammann L, Van Ness J (1989) Standard and robust orthogonal regression. Commun Stat—Simul Comput 18:145–162

    Article  MATH  Google Scholar 

  53. Tiyanyong JBC (1991) Estimating the residual variance in orthogonal regression with variable selection. J R Stat Soc: Ser D (The Statistician) 40:51–59

    Google Scholar 

  54. Zamar HR (1992) Bias robust estimation in orthogonal regression. Ann Stat 1875–1888

  55. McQuarrie A, Tsai C-L (1999) Model selection in orthogonal regression. Statist Probab Lett 45:341–349

    Article  MathSciNet  MATH  Google Scholar 

  56. Maronna R (2005) Principal components and orthogonal regression based on robust scales. Am Stat Assoc Am Soc Qual 47:264–273

    MathSciNet  Google Scholar 

  57. Wellmann R, Muller HC (2010) Depth notions for orthogonal regression. J Multivar Anal 101:2358–2371

    Article  MathSciNet  MATH  Google Scholar 

  58. Todorovb V, Hron K, Filzmoser P (2016) Classical and robust orthogonal regression between parts of compositional data. Statistics 50:1261–1275

    Article  MathSciNet  MATH  Google Scholar 

  59. Li GY, Tan MJ, Liew KJ (1999) Springback analysis for sheet forming processes by explicit finite element method in conjunction with orthogonal regression analysis. Int J Solids Struct 36:4653–4668

    Article  MATH  Google Scholar 

  60. Jiang D, Hu G, Qi G, Mazur N (2021) A fully convolutional neural network-based regression approach for effective chemical composition analysis using near-infrared spectroscopy in cloud. J Artif Intell Technol 1(1):74–82

    Article  Google Scholar 

  61. Basavegowda HS, Dagnew G (2020) Deep learning approach for microarray cancer data classification. CAAI Trans Intell Technol 5(1):22–33

    Article  Google Scholar 

  62. Besalu E, Jesus V, Julian-Ortiz D, Pogliani L (2011) Ordinary and orthogonal regressions in QSAR/QSPR and chemistry-related studies. Match Commun Math Comput Chem 573–583

  63. Xu Y, Qiu TT (2021) Human activity recognition and embedded application based on convolutional neural network. J Artif Intell Technol 1(1):51–60

    Article  Google Scholar 

  64. Ghosh S, Shivakumara P, Roy P, Pal U, Lu T (2020) Graphology based handwritten character analysis for human behaviour identification. CAAI Trans Intell Technol 5(1):55–65

    Article  Google Scholar 

  65. Ba KHT, Everett TA, Ito T, Higgins DA (2011) Trajectory angle determination in one dimensional single molecule tracking data by orthogonal regression analysis. Phys Chem Chem Phys 13:1827–1835

    Article  Google Scholar 

  66. Hu G, Kay Chen S-H, Mazur N (2021) Deep neural network-based speaker-aware information logging for augmentative and alternative communication. J Artif Intell Technol 1(2):138–143

  67. Gupta B, Tiwari M, Lamba S (2019) Visibility improvement and mass segmentation of mammogram images using quantile separated histogram equalisation with local contrast enhancement. CAAI Trans Intell Technol 4(2):73–79

    Article  Google Scholar 

  68. Qi’an C, Dandan Z, Songhua L, Liqun Z (2011) The proposition and application of the orthogonal regression experiment design in the ergonomics study of the interface design. IEEE Int Symp IT Med Educ 2:165–169

    Google Scholar 

  69. Jiao J, Zhen W, Zhu W, Wang G (2021) Quality-related root cause diagnosis based on orthogonal kernel principal component regression and transfer entropy. IEEE Trans Industr Inf 17(9):6347–6356

    Article  Google Scholar 

  70. Wu X, Xu X, Liu J, Wang H, Hu B, Nie F (2021) Supervised feature selection with orthogonal regression and feature weighting. IEEE Trans Neural Netw Learn Syst 32(5):1831–1838

    Article  MathSciNet  Google Scholar 

  71. Zhang R, Li X, Wu T, Zhao Y (2021) Data clustering via uncorrelated ridge regression. IEEE Trans Neural Netw Learn Syst 32(1):450–456

    Article  MathSciNet  Google Scholar 

  72. Wibowo A et al (2022) Earthquake early warning system using ncheck and hard-shared orthogonal multitarget regression on deep learning. In: IEEE Geoscience and Remote Sensing Letters, vol 19, pp 1–5, Art 7502605

  73. Mo D, Lai Z (2019) Robust jointly sparse regression with generalized orthogonal learning for image feature selection. Pattern Recogn 93:164–178

    Article  Google Scholar 

  74. Guo W, Shi Y, Wang S (2021) A unified scheme for distance metric learning and clustering via rank-reduced regression. IEEE Trans Syst Man Cybern: Syst 51(8):5218–5229

    Article  Google Scholar 

  75. Lu J, Lai Z, Wang H, Chen Y, Zhou J, Shen L (2022) Generalized embedding regression: a framework for supervised feature extraction. IEEE Trans Neural Netw Learn Syst 33(1):185–199

    Article  MathSciNet  Google Scholar 

  76. Richard L, Branham J (1995) Multivariate orthogonal regression in astronomy. Celest Mech Dyn Astron 61:239–251

    Article  MATH  Google Scholar 

  77. Fang R, Ji A, Zhao Z, Long D, Chen C (2020) A regression orthogonal biomechanical analysis of internal fixation for femoral shaft fracture. Biocybern Biomed Eng 40:1277–1290

    Article  Google Scholar 

  78. Yu Y, Gao G, Yang M, Chang H, Huang P, Yue D (2020) Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression. Inform Sci 506:19–36

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean government (MSIP) (NRF-2021R1A2B5B03002118) and This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2021-0-01835) supervised by the IITP (Institute of Information & Communications Technology Planning & Evaluation).

The authors declare no conflict of interest regarding the publication of this paper.

Author information

Authors and Affiliations

  1. School of Computing and IT, Manipal University Jaipur, Jaipur, India

    Pallavi & Sandeep Joshi

  2. School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju, South Korea

    Dilbag Singh, Manjit Kaur & Heung-No Lee

Authors
  1. Pallavi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandeep Joshi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dilbag Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manjit Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Heung-No Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heung-No Lee.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pallavi, Joshi, S., Singh, D. et al. Comprehensive Review of Orthogonal Regression and Its Applications in Different Domains. Arch Computat Methods Eng 29, 4027–4047 (2022). https://doi.org/10.1007/s11831-022-09728-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-022-09728-5

Search

Navigation