Non-negative Sparse Principal Component Analysis for Multidimensional Constrained Optimization

  • Thanh D. X. Duong
  • Vu N. Duong
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5351)


One classic problem in air traffic management (ATM) has been the problem of detection and resolution of conflicts between aircraft. Traditionally, a conflict between two aircraft is detected whenever the two protective cylinders surrounding the aircraft intersect. In Trajectory-based Air Traffic Management, a baseline for the next generation of air traffic management system, we suggest that these protective cylinders be deformable volumes induced by variations in weather information such as wind speed and directions subjected to uncertainties of future states of trajectory controls. Using contact constraints on deforming parametric surfaces of these protective volumes, a constrained minimization algorithm is proposed to compute collision between two deformable bodies, and a differential optimization scheme is proposed to resolve detected conflicts. Given the covariance matrix representing the state of aircraft trajectory and its control and objective functions, we consider the problem of maximizing the variance explained by a particular linear combination of the input variables where the coefficients in this combination are required to be non-negative, and the number of non-zero coefficients is constrained (e.g. state of trajectory and estimated time of arrival over one change point). Using convex relaxation and re-weighted l 1 technique, we reduce the problem to solving some semi-definite programming ones, and reinforce the non-negative principal components that satisfy the sparsity constraints. Numerical results show that the method presented in this paper is efficient and reliable in practice. Since the proposed method can be applied to a wide range of dynamic modeling problems such as collision avoidance in dynamic autonomous robots environments, dynamic interactions with 4D computer animation scenes, financial asset trading, or autonomous intelligent vehicles, we also attempt to keep all descriptions as general as possible.


principal component analysis semi-definite relaxation semi-definite programming l1-minimization iterative reweighting 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alizadeh, F.: Interior point methods in semi-definite programming with applications to combinatorial optimization. SIAM J. Optim. 5, 13–51 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Badea, L., Tilivea, D.: Sparse factorizations of gene expression guided by binding data. In: Pacific Symposium on Biocomputing (2005)Google Scholar
  3. 3.
    Boyd, S., Vandenberghe, L.: Convex optimization. Cambridge University Press, Cambridge (2004)CrossRefzbMATHGoogle Scholar
  4. 4.
    Cadima, J., Jolliffe, I.T.: Loadings and correlations in the interpretation of principal components. J. Appl. Statist. 22, 203–214 (1995)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Candes, E.J., Wakin, M.B., Boyd, S.: Enhancing sparsity by re-weighted l 1 minimization (preprint)Google Scholar
  6. 6.
    D’Aspremont, A., El Ghaoui, L., Jordan, M.I., Lanckriet, G.R.G.: A direct formulation for sparse PCA using semi-definite programming. SIAM Rev. 49, 434–448 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Duong, V.: Dynamic models for airborne air traffic management capability: State-of-the-art analysis (Internal report). Eurocontrol Experimental Centre, Bretigny (1996)Google Scholar
  8. 8.
    Fazel, M., Hindi, H., Boyd, S.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the American Control Conference, Arlington, VA., vol. 6, pp. 4734–4739 (2001)Google Scholar
  9. 9.
    Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24, 417–441 (1933)CrossRefzbMATHGoogle Scholar
  10. 10.
    Jagannathan, R., Ma, T.: Risk reduction in large portfolios: Why imposing the wrong constraints helps. Journal of Finance 58, 1651–1684 (2003)CrossRefGoogle Scholar
  11. 11.
    Jeffers, J.: Two case studies in the application of principal components. Appl. Statist. 16, 225–236 (1967)CrossRefGoogle Scholar
  12. 12.
    Jolliffe, I.T.: Rotation of principal components: Choice of normalization constraints. J. Appl. Statist. 22, 29–35 (1995)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Jolliffe, I.T., Trendafilov, N.T., Uddin, M.: A modified principal component technique based on the LASSO. J.Comput. Graphical Statist. 12, 531–547 (2003)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Jolliffe, I.T.: Principal component analysis. Springer, New York (2002)zbMATHGoogle Scholar
  15. 15.
    Lemarechal, C., Oustry, F.: Semi-definite relaxations and lagrangian duality with application to combinatorial optimization. Rapport de recherche 3710, INRIA, France (1999)Google Scholar
  16. 16.
    Lovasz, L., Schrijver, A.: Cones of matrices and set-functions and 0-1 optimization. SIAM J. Optim. 1, 166–190 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Moghaddam, B., Weiss, Y., Avidan, S.: Spectral Bounds for Sparse PCA: Exact & Greedy Algorithms. In: Advances in Neural Information Processing Systems, vol. 18, pp. 915–922. MIT Press, Cambridge (2006)Google Scholar
  18. 18.
    Nesterov, Y.: Smoothing technique and its application in semi-definite optimization. Math. Program. 110, 245–259 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Pearson, K.: On lines and planes of closest fit to systems of points in space. Phil. Mag. 2, 559–572 (1901)CrossRefzbMATHGoogle Scholar
  20. 20.
    Sturm, J.: Using SEDUMI 1.0x, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11, 625–653 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Toh, K.C., Todd, M.J., Tutuncu, R.H.: SDPT3 - a MA TLAB software package for semi-definite programming. Optim. Methods Softw. 11, 545–581 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Vines, S.: Simple principal components. Appl. Statist. 49, 441–451 (2000)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Zass, R., Shashua, A.: Non-negative Sparse PCA. In: Advances In Neural Information Processing Systems, vol. 19, pp. 1561–1568 (2007)Google Scholar
  24. 24.
    Zhang, Z., Zha, H., Simon, H.: Low-rank approximations with sparse factors I: Basic algorithms and error analysis. SIAM J. Matrix Anal. Appl. 23, 706–727 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Zhang, Z., Zha, H., Simon, H.: Low-rank approximations with sparse factors II: Penalized methods with discrete Newton-like iterations. SIAM J. Matrix Anal. Appl. 25, 901–920 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Zou, H., Hastie, T., Tibshirani, R.: Sparse principal component analysis. J. Comput. Graphical Statist. 15, 265–286 (2006)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Thanh D. X. Duong
    • 1
    • 2
  • Vu N. Duong
    • 2
    • 3
  1. 1.Ton Duc Thang UniversityVietnam
  2. 2.University of Science, VNU-HCMVietnam
  3. 3.EUROCONTROL Experimental CenterFrance

Personalised recommendations