Skip to main content

Non-negative Sparse Principal Component Analysis for Multidimensional Constrained Optimization

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNAI,volume 5351)

Abstract

One classic problem in air traffic management (ATM) has been the problem of detection and resolution of conflicts between aircraft. Traditionally, a conflict between two aircraft is detected whenever the two protective cylinders surrounding the aircraft intersect. In Trajectory-based Air Traffic Management, a baseline for the next generation of air traffic management system, we suggest that these protective cylinders be deformable volumes induced by variations in weather information such as wind speed and directions subjected to uncertainties of future states of trajectory controls. Using contact constraints on deforming parametric surfaces of these protective volumes, a constrained minimization algorithm is proposed to compute collision between two deformable bodies, and a differential optimization scheme is proposed to resolve detected conflicts. Given the covariance matrix representing the state of aircraft trajectory and its control and objective functions, we consider the problem of maximizing the variance explained by a particular linear combination of the input variables where the coefficients in this combination are required to be non-negative, and the number of non-zero coefficients is constrained (e.g. state of trajectory and estimated time of arrival over one change point). Using convex relaxation and re-weighted l 1 technique, we reduce the problem to solving some semi-definite programming ones, and reinforce the non-negative principal components that satisfy the sparsity constraints. Numerical results show that the method presented in this paper is efficient and reliable in practice. Since the proposed method can be applied to a wide range of dynamic modeling problems such as collision avoidance in dynamic autonomous robots environments, dynamic interactions with 4D computer animation scenes, financial asset trading, or autonomous intelligent vehicles, we also attempt to keep all descriptions as general as possible.

Keywords

  • principal component analysis
  • semi-definite relaxation
  • semi-definite programming
  • l1-minimization
  • iterative reweighting

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-540-89197-0_13
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-3-540-89197-0
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.00
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alizadeh, F.: Interior point methods in semi-definite programming with applications to combinatorial optimization. SIAM J. Optim. 5, 13–51 (1995)

    MathSciNet  CrossRef  MATH  Google Scholar 

  2. Badea, L., Tilivea, D.: Sparse factorizations of gene expression guided by binding data. In: Pacific Symposium on Biocomputing (2005)

    Google Scholar 

  3. Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press, Cambridge (2004)

    CrossRef  MATH  Google Scholar 

  4. Cadima, J., Jolliffe, I.T.: Loadings and correlations in the interpretation of principal components. J. Appl. Statist. 22, 203–214 (1995)

    MathSciNet  CrossRef  Google Scholar 

  5. Candes, E.J., Wakin, M.B., Boyd, S.: Enhancing sparsity by re-weighted l 1 minimization (preprint)

    Google Scholar 

  6. D’Aspremont, A., El Ghaoui, L., Jordan, M.I., Lanckriet, G.R.G.: A direct formulation for sparse PCA using semi-definite programming. SIAM Rev. 49, 434–448 (2007)

    MathSciNet  CrossRef  MATH  Google Scholar 

  7. Duong, V.: Dynamic models for airborne air traffic management capability: State-of-the-art analysis (Internal report). Eurocontrol Experimental Centre, Bretigny (1996)

    Google Scholar 

  8. Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference, Arlington, VA., vol. 6, pp. 4734–4739 (2001)

    Google Scholar 

  9. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417–441 (1933)

    CrossRef  MATH  Google Scholar 

  10. Jagannathan, R., Ma, T.: Risk reduction in large portfolios: Why imposing the wrong constraints helps. Journal of Finance 58, 1651–1684 (2003)

    CrossRef  Google Scholar 

  11. Jeffers, J.: Two case studies in the application of principal components. Appl. Statist. 16, 225–236 (1967)

    CrossRef  Google Scholar 

  12. Jolliffe, I.T.: Rotation of principal components: Choice of normalization constraints. J. Appl. Statist. 22, 29–35 (1995)

    MathSciNet  CrossRef  Google Scholar 

  13. Jolliffe, I.T., Trendafilov, N.T., Uddin, M.: A modified principal component technique based on the LASSO. J.Comput. Graphical Statist. 12, 531–547 (2003)

    MathSciNet  CrossRef  Google Scholar 

  14. Jolliffe, I.T.: Principal component analysis. Springer, New York (2002)

    MATH  Google Scholar 

  15. Lemarechal, C., Oustry, F.: Semi-definite relaxations and lagrangian duality with application to combinatorial optimization. Rapport de recherche 3710, INRIA, France (1999)

    Google Scholar 

  16. Lovasz, L., Schrijver, A.: Cones of matrices and set-functions and 0-1 optimization. SIAM J. Optim. 1, 166–190 (1991)

    MathSciNet  CrossRef  MATH  Google Scholar 

  17. Moghaddam, B., Weiss, Y., Avidan, S.: Spectral Bounds for Sparse PCA: Exact & Greedy Algorithms. In: Advances in Neural Information Processing Systems, vol. 18, pp. 915–922. MIT Press, Cambridge (2006)

    Google Scholar 

  18. Nesterov, Y.: Smoothing technique and its application in semi-definite optimization. Math. Program. 110, 245–259 (2007)

    MathSciNet  CrossRef  MATH  Google Scholar 

  19. Pearson, K.: On lines and planes of closest fit to systems of points in space. Phil. Mag. 2, 559–572 (1901)

    CrossRef  MATH  Google Scholar 

  20. Sturm, J.: Using SEDUMI 1.0x, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11, 625–653 (1999)

    MathSciNet  CrossRef  MATH  Google Scholar 

  21. Toh, K.C., Todd, M.J., Tutuncu, R.H.: SDPT3 - a MA TLAB software package for semi-definite programming. Optim. Methods Softw. 11, 545–581 (1999)

    MathSciNet  CrossRef  MATH  Google Scholar 

  22. Vines, S.: Simple principal components. Appl. Statist. 49, 441–451 (2000)

    MathSciNet  MATH  Google Scholar 

  23. Zass, R., Shashua, A.: Non-negative Sparse PCA. In: Advances In Neural Information Processing Systems, vol. 19, pp. 1561–1568 (2007)

    Google Scholar 

  24. Zhang, Z., Zha, H., Simon, H.: Low-rank approximations with sparse factors I: Basic algorithms and error analysis. SIAM J. Matrix Anal. Appl. 23, 706–727 (2002)

    MathSciNet  CrossRef  MATH  Google Scholar 

  25. Zhang, Z., Zha, H., Simon, H.: Low-rank approximations with sparse factors II: Penalized methods with discrete Newton-like iterations. SIAM J. Matrix Anal. Appl. 25, 901–920 (2004)

    MathSciNet  CrossRef  MATH  Google Scholar 

  26. Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graphical Statist. 15, 265–286 (2006)

    MathSciNet  CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Duong, T.D.X., Duong, V.N. (2008). Non-negative Sparse Principal Component Analysis for Multidimensional Constrained Optimization. In: Ho, TB., Zhou, ZH. (eds) PRICAI 2008: Trends in Artificial Intelligence. PRICAI 2008. Lecture Notes in Computer Science(), vol 5351. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89197-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-89197-0_13

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-89196-3

  • Online ISBN: 978-3-540-89197-0

  • eBook Packages: Computer ScienceComputer Science (R0)