Optimization-Based Visualization

  • Gintautas Dzemyda
  • Olga Kurasova
  • Julius Žilinskas
Part of the Springer Optimization and Its Applications book series (SOIA, volume 75)


In this chapter, we consider one of themost popular approaches of multidimensional data visualization, known as multidimensional scaling (MDS) [14, 31, 127, 139, 150, 191, 202]. The essential part of this technique is optimization of a function possessing many optimization adverse properties [231]. By means of MDS, a set of objects can be represented as a set of points in a low-dimensional space and exposed in this way to a human expert for a heuristic analysis. The data for MDS is a pairwise similarity/dissimilarity between the objects—it is not necessary to have multidimensional points as data. Application areas of MDS vary from psychometrics [197] and market analysis [39, 165] to mobile communications [75] and pharmacology [232].


  1. 3.
    Arabie, P.: Was Euclid an unnecessarily sophisticated psychologist? Psychometrika 56(4), 567–587 (1991). DOI 10.1007/BF02294491CrossRefMATHMathSciNetGoogle Scholar
  2. 14.
    Borg, I., Groenen, P.J.F.: Modern Multidimensional Scaling: Theory and Applications, 2nd edn. Springer, New York (2005)Google Scholar
  3. 15.
    Bortz, J.: Kritische bemerkungen über den einsatz nicht-euklidischer metriken im rahmen der multidimensionalen skalierung. Archiv für Psychologie 126(2–4), 196–212 (1974)Google Scholar
  4. 18.
    Brusco, M., Stahl, S.: Using quadratic assignment methods to generate initial permutations for least-squares unidimensional scaling of symmetric proximity matrices. J. Classif. 17(2), 197–223 (2000). DOI 10.1007/s003570000019CrossRefMATHMathSciNetGoogle Scholar
  5. 19.
    Brusco, M.J.: A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices. J. Classif. 18(1), 3–33 (2001). DOI 10.1007/s00357-001-0003-4MATHMathSciNetGoogle Scholar
  6. 20.
    Brusco, M.J.: Integer programming methods for seriation and unidimensional scaling of proximity matrices: A review and some extensions. J. Classif. 19(1), 45–67 (2002). DOI 10.1007/ s00357-001-0032-zCrossRefMATHMathSciNetGoogle Scholar
  7. 21.
    Brusco, M.J.: On the performance of simulated annealing for large-scale L2 unidimensional scaling. J. Classif. 23(2), 255–268 (2006). DOI 10. 1007/s00357-006-0015-1CrossRefMathSciNetGoogle Scholar
  8. 22.
    Brusco, M.J., Stahl, S.: Branch-and-Bound Applications in Combinatorial Data Analysis. Springer, New York (2005)MATHGoogle Scholar
  9. 23.
    Brusco, M.J., Stahl, S.: Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. Psychometrika 70(2), 253–270 (2005). DOI 10.1007/ s11336-002-1032-6CrossRefMathSciNetGoogle Scholar
  10. 25.
    Cantú-Paz, E.: Efficient and Accurate Parallel Genetic Algorithms. Kluwer Academic, Boston, MA (2000)MATHGoogle Scholar
  11. 30.
    Corne, D., Dorigo, M., Glover, F. (eds.): New Ideas in Optimization. McGraw-Hill, Maidenhead (1999)Google Scholar
  12. 31.
    Cox, T.F., Cox, M.A.A.: Multidimensional Scaling, 2nd edn. Chapman & Hall/CRC, Boca Raton (2001)MATHGoogle Scholar
  13. 35.
    Defays, D.: A short note on a method of seriation. Br. J. Math. Stat. Psychol. 31, 49–53 (1978)CrossRefGoogle Scholar
  14. 39.
    DeSarbo, W.S., Kim, Y., Wedel, M., Fong, D.K.H.: A bayesian approach to the spatial representation of market structure from consumer choice data. Eur. J. Oper. Res. 111(2), 285–305 (1998). DOI 10.1016/ S0377-2217(98)00150-7CrossRefMATHGoogle Scholar
  15. 60.
    Everett, J.E.: Algorithms for multidimensional scaling. In: Chambers, L.D. (ed.) The Practical Handbook of Genetic Algorithms, 2nd edn., pp. 203–233. Chapman & Hall/CRC, Boca Raton (2001)Google Scholar
  16. 62.
    Festa, P., Pardalos, P.M., Resende, M.G.C., Ribeiro, C.C.: Randomized heuristics for the max-cut problem. Optim. Method Softw. 17(6), 1033–1058 (2002). DOI  10.1080/1055678021000090033 CrossRefMATHMathSciNetGoogle Scholar
  17. 65.
    Floudas, C.A.: Deterministic Global Optimization: Theory, Methods and Applications, Nonconvex Optimization and its Applications, vol. 37. Kluwer Academic, Boston, MA (2000)Google Scholar
  18. 70.
    Gendron, B., Crainic, T.G.: Parallel branch-and-bound algorithms: Survey and synthesis. Oper. Res. 42(6), 1042–1066 (1994)CrossRefMATHMathSciNetGoogle Scholar
  19. 72.
    Green, P., Carmone, F., Smith, S.: Multidimensional Scaling: Concepts and Applications. Allyn and Bacon, Boston (1989)Google Scholar
  20. 75.
    Groenen, P., Mathar, R., Trejos, J.: Global optimization methods for multidimensional scaling applied to mobile communication. In: Gaul, W., Opitz, O., Schander, M. (eds.) Data Analysis: Scientific Modeling and Practical Applications, pp. 459–475. Springer, Berlin (2000)CrossRefGoogle Scholar
  21. 76.
    Groenen, P.J.F.: The Majorization Approach to Multidimentional Scaling: Some Problems and Extensions. DSWO, Leiden (1993)Google Scholar
  22. 77.
    Groenen, P.J.F., Heiser, W.J.: The tunneling method for global optimization in multidimensional scaling. Psychometrika 61(3), 529–550 (1996). DOI 10.1007/BF02294553CrossRefMATHGoogle Scholar
  23. 78.
    Groenen, P.J.F., Heiser, W.J., Meulman, J.J.: City-block scaling: smoothing strategies for avoiding local minima. In: Balderjahn, I., Mathar, R., Schader, M. (eds.) Classification, Data Analysis, and Data Highways, pp. 46–53. Springer, Berlin (1998)CrossRefGoogle Scholar
  24. 79.
    Groenen, P.J.F., Heiser, W.J., Meulman, J.J.: Global optimization in least-squares multidimensional scaling by distance smoothing. J. Classif. 16(2), 225–254 (1999). DOI 10.1007/s003579900055CrossRefMATHMathSciNetGoogle Scholar
  25. 80.
    Groenen, P.J.F., Mathar, R., Heiser, W.J.: The majorization approach to multidimensional scaling for Minkowski distances. J. Classif. 12(1), 3–19 (1995). DOI 10.1007/BF01202265CrossRefMATHMathSciNetGoogle Scholar
  26. 96.
    Hooker, J.N.: Testing heuristics: We have it all wrong. J. Heuristics 1(1), 33–42 (1995). DOI 10.1007/BF02430364CrossRefMATHMathSciNetGoogle Scholar
  27. 97.
    Horst, R., Pardalos, P.M., Thoai, N.V.: Introduction to Global Optimization, Nonconvex Optimization and Its Applications, vol. 48, 2nd edn. Kluwer Academic, Boston, MA (2001)Google Scholar
  28. 98.
    Hubert, L., Arabie, P., Hesson-Mcinnis, M.: Multidimensional scaling in the city-block metric: A combinatorial approach. J. Classif. 9(2), 211–236 (1992). DOI 10.1007/BF02621407CrossRefGoogle Scholar
  29. 99.
    Hubert, L.J., Arabie, P., Meulman, J.J.: Linear unidimensional scaling in the l 2-norm: basic optimization methods using matlab. J. Classif. 19(2), 303–328 (2002). DOI 10.1007/s00357-001-0047-5CrossRefMATHMathSciNetGoogle Scholar
  30. 100.
    Hubert, L.J., Golledge, R.G.: Matrix reorganization and dynamic programming: Applications to paired comparisons and unidimensional seriation. Psychometrika 46(4), 429–441 (1981). DOI 10.1007/ BF02293800CrossRefMATHGoogle Scholar
  31. 116.
    Kearsley, A.J., Tapia, R.A., Trosset, M.W.: The solution of the metric STRESS and SSTRESS problems in multidimensional scaling using Newton’s method. Comput. Stat. 13(3), 369–396 (1998)MATHGoogle Scholar
  32. 119.
    Klock, H., Buhmann, J.M.: Data visualization by multidimensional scaling: a deterministic annealing approach. Pattern Recog. 33(4), 651–669 (2000). DOI 10.1016/S0031-3203(99)00078-3CrossRefGoogle Scholar
  33. 127.
    Kruskal, J.B., Wish, M.: Multidimensional Scaling. Bell Laboratories, Murray Hill (1978)Google Scholar
  34. 133.
    Lau, K., Leung, P.L., Tse, K.: A nonlinear programming approach to metric unidimensional scaling. J. Classif. 15(1), 3–14 (1998). DOI 10. 1007/s003579900017CrossRefMATHGoogle Scholar
  35. 136.
    de Leeuw, J.: Applications of convex analysis to multidimensional scaling. In: Barra, J., Brodeau, F., Romier, G., van Cutsem, B. (eds.) Recent Developments in Statistics, pp. 133–145. North-Holland, Amsterdam (1977)Google Scholar
  36. 137.
    de Leeuw, J.: Differentiability of Kruskal’s stress at a local minimum. Psychometrika 49(1), 111–113 (1984). DOI 10.1007/BF02294209CrossRefMathSciNetGoogle Scholar
  37. 138.
    de Leeuw, J.: Convergence of the majorization method for multidimensional scaling. J. Classif. 5(2), 163–180 (1988). DOI 10. 1007/BF01897162CrossRefMATHGoogle Scholar
  38. 139.
    de Leeuw, J., Heiser, W.: Theory of multidimensional scaling. In: Krishnaiah, P.R., Kanal, L.N. (eds.) Classification Pattern Recognition and Reduction of Dimensionality, Handbook of Statistics, vol. 2, pp. 285–316. Elsevier, Amsterdam (1982). DOI 10.1016/S0169-7161(82) 02016-1Google Scholar
  39. 140.
    Leung, P.L., Lau, K.: Estimating the city-block two-dimensional scaling model with simulated annealing. Eur. J. Oper. Res. 158(2), 518–524 (2004). DOI 10.1016/S0377-2217(03)00357-6CrossRefMATHGoogle Scholar
  40. 149.
    Mathar, R.: A hybrid global optimization algorithm for multidimensional scaling. In: Klar, R., Opitz, O. (eds.) Classification and Knowledge Organization, pp. 63–71. Springer, Berlin (1997)CrossRefGoogle Scholar
  41. 150.
    Mathar, R.: Multidimensionale Skalierung, Mathematische Grundlagen und Algorithmische Konzepte. Teubner Verlag, Leipzig (1997)MATHGoogle Scholar
  42. 151.
    Mathar, R., Žilinskas, A.: On global optimization in two-dimensional scaling. Acta Applicandae Mathematicae 33(1), 109–118 (1993). DOI 10.1007/BF00995497CrossRefMATHMathSciNetGoogle Scholar
  43. 153.
    McIver, J.P., Carmines, E.G.: Unidimensional Scaling. Sage Publications, Newbury Park (1981)Google Scholar
  44. 159.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs, 3rd edn. Springer, Berlin (1996)CrossRefMATHGoogle Scholar
  45. 160.
    Miyano, H., Inukai, Y.: Sequential estimation in multidimensional scaling. Psychometrika 47(3), 321–336 (1982). DOI 10.1007/BF02294163CrossRefMathSciNetGoogle Scholar
  46. 161.
    Murillo, A., Vera, J.F., Heiser, W.J.: A permutation-translation simulated annealing algorithm for l 1 and l 2 unidimensional scaling. J. Classif. 22(1), 119–138 (2005). DOI 10.1007/s00357-005-0008-5CrossRefMATHMathSciNetGoogle Scholar
  47. 165.
    Nelson, T.R., Rabianski, J.: Consumer preferences in housing market analysis: An application of multidimensional scaling techniques. R. Estate Econ. 16(2), 138–159 (1988). DOI 10.1111/1540-6229.00451CrossRefGoogle Scholar
  48. 173.
    Pliner, V.: Metric unidimensional scaling and global optimization. J. Classif. 13(1), 3–18 (1996). DOI 10.1007/BF01202579CrossRefMATHMathSciNetGoogle Scholar
  49. 174.
    Poole, K.T.: Least squares metric, unidimensional scaling of multivariate linear models. Psychometrika 55(1), 123–149 (1990). DOI 10.1007/BF02294747CrossRefMATHMathSciNetGoogle Scholar
  50. 175.
    Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes in C\(++\): The Art of Scientific Computing, 2nd edn. Cambridge University Press, Cambridge (2002)Google Scholar
  51. 178.
    Rayward-Smith, V.J., Rush, S.A., McKeown, G.P.: Efficiency considerations in the implementation of parallel branch-and-bound. Ann. Oper. Res. 43(2), 123–145 (1993). DOI 10.1007/BF02024489CrossRefMATHMathSciNetGoogle Scholar
  52. 186.
    Ruuskanen, J.O., Laurila, J., Xhaard, H., Rantanen, V.V., Vuoriluoto, K., Wurster, S., Marjamäki, A., Vainio, M., Johnson, M.S., Scheinin, M.: Conserved structural, pharmacological and functional properties among the three human and five zebrafish α2-adrenoceptors. Br. J. Pharmacol. 144(2), 165–177 (2005). DOI 10.1038/sj.bjp.0706057CrossRefGoogle Scholar
  53. 191.
    Schiffman, S.S., Reynolds, M.L., Young, F.W.: Introduction to Multidimensional Scaling: Theory, Methods, and Applications. Academic, London (1981)MATHGoogle Scholar
  54. 194.
    Simantiraki, E.: Unidimensional scaling: A linear programming approach minimizing absolute deviations. J. Classif. 13(1), 19–25 (1996). DOI 10.1007/BF01202580CrossRefMATHGoogle Scholar
  55. 197.
    Takane, Y.: Applications of multidimensional scaling in psychometrics. In: Rao, C.R., Sinharay, S. (eds.) Psychometrics, Handbook of Statistics, vol. 26, pp. 359–400. Elsevier, Amsterdam (2006). DOI 10.1016/ S0169-7161(06)26011-5Google Scholar
  56. 202.
    Torgerson, W.S.: Theory and Methods of Scaling. Wiley, New York (1958)Google Scholar
  57. 203.
    Törn, A., Žilinskas, A.: Global optimization. Lect. Notes Comput. Sci. 350, 1–252 (1989). DOI 10.1007/3-540-50871-6CrossRefGoogle Scholar
  58. 210.
    Varoneckas, A., Žilinskas, A., Žilinskas, J.: Multidimensional scaling using parallel genetic algorithm. In: Bogle, I.D.L., Žilinskas, J. (eds.) Computer Aided Methods in Optimal Design and Operations, Series on Computers and Operations Research, vol. 7, pp. 129–138. World Scientific, Singapore (2006). DOI 10.1142/9789812772954_0014Google Scholar
  59. 211.
    Varoneckas, A., Žilinskas, A., Žilinskas, J.: Parallel multidimensional scaling using grid computing: assessment of performance. Inform. Tech. Contr. 37(1), 52–56 (2008)Google Scholar
  60. 212.
    Vera, J.F., Heiser, W.J., Murillo, A.: Global optimization in any Minkowski metric: a permutation-translation simulated annealing algorithm for multidimensional scaling. J. Classif. 24(2), 277–301 (2007). DOI 10.1007/s00357-007-0020-1CrossRefMATHMathSciNetGoogle Scholar
  61. 221.
    Žilinskas, A.: On the distribution of the distance between two points in a cube. Random Operat. Stoch. Equat. 11, 21–24 (2003)CrossRefMATHGoogle Scholar
  62. 223.
    Žilinskas, A., Žilinskas, J.: On multidimensional scaling with Euclidean and city block metrics. Technol. Econ. Dev. Econ. 12(1), 69–75 (2006). DOI 10.1080/13928619.2006.9637725Google Scholar
  63. 224.
    Žilinskas, A., Žilinskas, J.: On visualization of multidimensional data using three-dimensional embedding space. Technol. Econ. Dev. Econ. 12(4), 353–359 (2006). DOI 10.1080/13928619.2006.9637758Google Scholar
  64. 225.
    Žilinskas, A., Žilinskas, J.: Parallel hybrid algorithm for global optimization of problems occurring in MDS-based visualization. Comput. Math. Appl. 52(1–2), 211–224 (2006). DOI 10.1016/j.camwa. 2006.08.016CrossRefMATHMathSciNetGoogle Scholar
  65. 226.
    Žilinskas, A., Žilinskas, J.: Parallel genetic algorithm: assessment of performance in multidimensional scaling. In: GECCO ’07: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 1492–1501. ACM, New York (2007). DOI 10.1145/1276958.1277229Google Scholar
  66. 227.
    Žilinskas, A., Žilinskas, J.: Two level minimization in multidimensional scaling. J. Global Optim. 38(4), 581–596 (2007). DOI 10.1007/s10898-006-9097-xCrossRefMATHMathSciNetGoogle Scholar
  67. 228.
    Žilinskas, A., Žilinskas, J.: A hybrid method for multidimensional scaling using city-block distances. Math. Method Oper. Res. 68(3), 429–443 (2008). DOI 10.1007/s00186-008-0238-5CrossRefMATHGoogle Scholar
  68. 229.
    Žilinskas, A., Žilinskas, J.: Three-dimensional visualization by means of multidimensional scaling. In: Sakalauskas, L., Weber, G.W., Zavadskas, E.K. (eds.) The 20th International Conference EURO Mini Conference Continuous Optimization and Knowledge-Based Technologies (EurOPT-2008), May 20–23, 2008, Neringa, Lithuania, pp. 71–76 (2008)Google Scholar
  69. 230.
    Žilinskas, A., Žilinskas, J.: Branch and bound algorithm for multidimensional scaling with city-block metric. J. Global Optim. 43(2–3), 357–372 (2009). DOI 10.1007/s10898-008-9306-xCrossRefMATHMathSciNetGoogle Scholar
  70. 231.
    Žilinskas, A., Žilinskas, J.: Optimization-based visualization. In: Floudas, C.A., Pardalos, P.M. (eds.) Encyclopedia of Optimization, 2nd edn., pp. 2785–2791. Springer, New York (2009). DOI 10.1007/ 978-0-387-74759-0_478Google Scholar
  71. 232.
    Žilinskas, J.: Multidimensional scaling in protein and pharmacological sciences. In: Bogle, I.D.L., Žilinskas, J. (eds.) Computer Aided Methods in Optimal Design and Operations, Series on Computers and Operations Research, vol. 7, pp. 139–148. World Scientific, Singapore (2006). DOI 10.1142/9789812772954_0015Google Scholar
  72. 233.
    Žilinskas, J.: Reducing of search space of multidimensional scaling problems with data exposing symmetries. Inform. Tech. Contr. 36(4), 377–382 (2007)Google Scholar
  73. 234.
    Žilinskas, J.: On dimensionality of embedding space in multidimensional scaling. Informatica 19(3), 447–460 (2008)MATHGoogle Scholar
  74. 235.
    Žilinskas, J.: Multidimensional scaling with city-block distances based on combinatorial optimization and systems of linear equations. Math. Model. Anal. 14(2), 259–270 (2009). DOI 10.3846/1392-6292.2009.14. 259-270CrossRefMATHMathSciNetGoogle Scholar
  75. 236.
    Žilinskas, J.: Parallel global optimization in multidimensional scaling. In: Čiegis, R., Henty, D., Kågström, B., Žilinskas, J. (eds.) Parallel Scientific Computing and Optimization, Springer Optimization and Its Applications, vol. 27, pp. 69–82. Springer, New York (2009). DOI 10. 1007/978-0-387-09707-7_6Google Scholar
  76. 237.
    Žilinskas, J.: Parallel branch and bound for multidimensional scaling with city-block distances. J. Global Optim. (2012). DOI 10.1007/ s10898-010-9624-7Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2013

Authors and Affiliations

  • Gintautas Dzemyda
    • 1
  • Olga Kurasova
    • 1
  • Julius Žilinskas
    • 1
  1. 1.Institute of Mathematics and InformaticsVilnius UniversityVilniusLithuania

Personalised recommendations