The Role of Dynamics in Extracting Information Sparsely Encoded in High Dimensional Data Streams

  • Mario Sznaier
  • Octavia Camps
  • Necmiye Ozay
  • Tao Ding
  • Gilead Tadmor
  • Dana Brooks
Part of the Springer Optimization and Its Applications book series (SOIA, volume 40)


A major roadblock in taking full advantage of the recent exponential growth in data collection and actuation capabilities stems from the curse of dimensionality. Simply put, existing techniques are ill-equipped to deal with the resulting overwhelming volume of data. The goal of this chapter is to show how the use of simple dynamical systems concepts can lead to tractable, computationally efficient algorithms for extracting information sparsely encoded in multimodal, extremely large data sets. In addition, as shown here, this approach leads to nonentropic information measures, better suited than the classical, entropy-based information theoretic measure, to problems where the information is by nature dynamic and changes as it propagates through a network where the nodes themselves are dynamical systems.


Video Sequence Data Stream Locally Linear Embedding Hankel Matrix Dynamic Texture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zaslaver, A., Bren, A., Ronen, M., Itzkovitz, S., Kikoin, I., Shavit, S., Liebermeister, W., Surette, M.G., Alon, U.: A comprehensive library of fluorescent transcriptional reporters for Escherichia coli. Nat. Methods 3, 623–628 (2006) CrossRefGoogle Scholar
  2. 2.
    Lee, J.A., Verleysen, M.: Nonlinear Dimensionality Reduction. Springer, Berlin (2007) MATHCrossRefGoogle Scholar
  3. 3.
    Costeira, J., Kanade, T.: A multibody factorization method for independently moving objects. Int. J. Comput. Vis. 29, 159–179 (1998) CrossRefGoogle Scholar
  4. 4.
    Vidal, R., Ma, Y., Soatto, S., Sastry, S.: Two–view multibody structure from motion. Int. J. Comput. Vis. 68, 7–25 (2006) CrossRefGoogle Scholar
  5. 5.
    Zelnik-Manor, L., Irani, M.: Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In: Proc. of the 2003 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 287–293 (2003) Google Scholar
  6. 6.
    Ma, Y., Vidal, R.: A closed form solution to the identification of hybrid arx models via the identification of algebraic varieties. In: Hybrid Systems Computation and Control, pp. 449–465 (2005) Google Scholar
  7. 7.
    Bemporad, A., Garulli, A., Paoletti, S., Vicino, A.: A bounded-error approach to piecewise affine system identification. IEEE Trans. Autom. Control 50, 1567–1580 (2005) CrossRefMathSciNetGoogle Scholar
  8. 8.
    Saul, L.K., Roweis, S.T.: Think globally, fit locally: unsupervised learning of low dimensional manifolds. J. Mach. Learning Res. 4, 119–155 (2003) CrossRefMathSciNetGoogle Scholar
  9. 9.
    Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319–2323 (2000) CrossRefGoogle Scholar
  10. 10.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003) MATHCrossRefGoogle Scholar
  11. 11.
    Donoho, D.L., Grimes, C.E.: Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. In: Proceedings of the National Academy of Arts and Sciences, vol. 100, pp. 5591–5596 (2003) Google Scholar
  12. 12.
    Weinberger, K.Q., Sha, F., Saul, L.K.: Learning a kernel matrix for nonlinear dimensionality reduction. In: Proc. of the 2004 International Conference on Machine Learning. ACM, New York (2004) Google Scholar
  13. 13.
    Weinberger, K.Q., Saul, L.K.: Unsupervised learning of image manifolds by semidefinite programming. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 988–995 (2004) Google Scholar
  14. 14.
    Ozay, N., Sznaier, M., Lagoa, C., Camps, O.: A sparsification approach to set membership identification of a class of affine hybrid system. In: Proc. 47th IEEE Conf. Dec. Control (CDC), pp. 123–130 (2008) Google Scholar
  15. 15.
    Sznaier, M., Lagoa, C., Mazzaro, M.C.: An algorithm for sampling balls in ℋ with applications to risk–adjusted performance analysis and model (in)validation. IEEE Trans. Autom. Control 50, 410–416 (2005) CrossRefMathSciNetGoogle Scholar
  16. 16.
    Sznaier, M., Lagoa, C., Ma, W.: Risk adjusted identification of a class of nonlinear systems. In: Proc. 46 IEEE Conf. Dec. Control., pp. 5117–5122 (2007) Google Scholar
  17. 17.
    Sánchez Peña, R., Sznaier, M.: Robust Systems Theory and Applications. Wiley, New York (1998) Google Scholar
  18. 18.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9, 137–154 (1992) CrossRefGoogle Scholar
  19. 19.
    Xiao, J., Chai, J., Kanade, T.: A closed-form solution to non-rigid shape and motion recovery. In: Proc. of the 8th European Conference on Computer Vision (ECCV 2004) (2004) Google Scholar
  20. 20.
    Vidal, R., Hartley, R.: Motion segmentation with missing data using powerfactorization and gpca. In: Proc. of the 2004 IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, pp. 310–316 (2004) Google Scholar
  21. 21.
    Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American Control Conf. (2001), vol. 6, pp. 4734–4739. AACC, Washington (2001) Google Scholar
  22. 22.
    Lobo, M., Fazel, M., Boyd, S.: Portfolio optimization with linear and fixed transaction costs. Ann. Oper. Res. 152, 376–394 (2007) CrossRefMathSciNetGoogle Scholar
  23. 23.
    Gargi, U., Kasturi, R., Strayer, S.H.: Performance characterization of video-shot-change detection methods. IEEE Trans. Circuits Syst. Video Technol. 10, 1–13 (2000) CrossRefGoogle Scholar
  24. 24.
    Yuan, J., Wang, H., Xiao, L., Zheng, W., Li, J., Lin, F., Zhang, B.: A formal study of shot boundary detection. IEEE Trans. Circuits Syst. Video Technol. 17, 168–186 (2007) CrossRefGoogle Scholar
  25. 25.
    Osian, M., Van Gool, L.: Video shot characterization. Mach. Vis. Appl. 15, 172–177 (2004) CrossRefGoogle Scholar
  26. 26.
    Petrovic, N., Ivanovic, A., Jojic, N.: Recursive estimation of generative models of video. In: Proc. of the 2006 IEEE Conf. on Computer Vision and Pattern Recognition, pp. 79–86 (2006) Google Scholar
  27. 27.
    Lu, L., Vidal, R.: Combined central and subspace clustering for computer vision applications. In: Proc. of the 2006 International Conference on Machine Learning, pp. 593–600 (2006) Google Scholar
  28. 28.
    Rand, W.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971) CrossRefGoogle Scholar
  29. 29.
    Doretto, G., Chiuso, A., Wu, Y., Soatto, S.: Dynamic textures. Int. J. Comput. Vis. 51, 91–109 (2003) MATHCrossRefGoogle Scholar
  30. 30.
    Chan, A.B., Vasconcelos, N.: Mixtures of dynamic textures. In: Proc. of the 2005 International Conference on Computer Vision (ICCV), vol. 1, pp. 641–647 (2005) Google Scholar
  31. 31.
    Cooper, L., Liu, J., Huang, K.: Spatial segmentation of temporal texture using mixture linear models. In: Proceedings of the Dynamical Vision Workshop in the International Conference of Computer Vision, pp. 142–150 (2005) Google Scholar
  32. 32.
    Ghoreyshi, A., Vidal, R.: Segmenting dynamic textures with using descriptors, arx models and level sets. In: Proceedings of the Dynamical Vision Workshop in the European Conference of Computer Vision, pp. 127–141 (2006) Google Scholar
  33. 33.
    Vandenberghe, L., Boyd, S.P.: Semidefinite programming. SIAM Rev. 38(1), 49–95 (1996) MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Fazel, M., Hindi, H., Boyd, S.P.: Log-det heuristic for matrix rank minimization with applications to Hankel and euclidean distance matrices. In: Proceedings of the 2003 American Control Conf., vol. 3, pp. 2156–2162. AACC, Washington (2003) CrossRefGoogle Scholar
  35. 35.
    Sznaier, M., Mazzaro, C., Camps, O.I.: Open-loop worst-case identification of nonschur plants. Automatica 39, 1019–1025 (2003) MATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    Chen, J., Gu, G.: Control Oriented System Identification, an ℋ Approach. Wiley, New York (2000) Google Scholar
  37. 37.
    Donoho, D.: High-dimensional data analysis: The curses and blessings of dimensionality. In: AMS Conf. on Math Challenges of the 21st Century (2000). Plenary presentation Google Scholar
  38. 38.
    Candes, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59, 1207–1223 (2006) MATHCrossRefMathSciNetGoogle Scholar
  39. 39.
    Donoho, D.: Compressed sensing. IEEE Trans. Information Theory 52, 1289–13006 (2006) CrossRefMathSciNetGoogle Scholar
  40. 40.
    Carin, L., Liu, D., Guo, B.: In situ compressive sensing. Inverse Probl. 24, 015023 (2008) CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Mario Sznaier
    • 1
  • Octavia Camps
    • 1
  • Necmiye Ozay
    • 1
  • Tao Ding
    • 2
  • Gilead Tadmor
    • 1
  • Dana Brooks
    • 1
  1. 1.ECE DepartmentNortheastern UniversityBostonUSA
  2. 2.Department of Electrical EngineeringPenn State UniversityUniversity ParkUSA

Personalised recommendations