A Conceptual Framework for Visualizing Composite Indicators

Abstract

Composite indicators (CIs) are common measurements and benchmarking tools used to measure multidimensional concepts such as well-being, education and more. Indicators and sub-indicators are selected and combined to reflect a measured phenomenon. Measurement iterations produce a series of time-oriented data, which stakeholders, as well as the general public, might be interested in interpreting. Visualization of a CI is highly recommended, in order to facilitate interpretation and enhance understanding of indicator components and their evolution over time. In recent years, a variety of CI visualizations have been published including various visualization techniques. Indeed, visualizing a CI is a complex and challenging issue, involving many design choices. However, there is a lack of guidelines and methodological approaches for CI visualization design. We suggest a framework that provides a systematic way of thinking of CI visualizations. The framework is intended for two uses: as a design tool when constructing a new CI visualization, and as an analytic tool for systematically describing, comparing and evaluating CI visualizations. The suggested framework is the outcome of both a top-down process, based on CI construction and information visualization literature, and a bottom-up process, in which 35 existing visualization applications of popular CIs were analyzed. We use Munzner’s visualization analysis and design framework (Munzner in Visualization analysis and design, CRC Press, Boca Raton, 2014) in an adaptive way, considering the specific challenges and characteristics of CI visualizations, in order to develop and discuss a systematic view of the data, tasks and methods for visualizing CIs. We demonstrate the use of the framework with a case study analyzing the popular OECD Better Life Index visualization tool.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Source: http://www.oecd.org/std/47917288.pdf. (Color figure online)

Fig. 6

Source: NRI, World Economic Forum, Switzerland, 2015—http://reports.weforum.org/globalinformation-technology-report-2015/report-highlights/. (Color figure online)

Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Notes

  1. 1.

    http://oe.cd/bli.

References

  1. Aigner, W., Miksch, S., Schumann, H., & Tominski, C. (2011). Visualization of time-oriented data. Berlin: Springer Science & Business Media.

    Google Scholar 

  2. Albo, Y., Lanir, J., Bak, P., & Rafaeli, S. (2016a). Off the radar: Comparative evaluation of radial visualization solutions for composite indicators. IEEE Transactions on Visualization and Computer Graphics, 22(1), 569–578.

    Article  Google Scholar 

  3. Albo, Y., Lanir, J., Bak, P., & Rafaeli, S. (2016b). Static vs. dynamic time mapping in radial composite indicator visualization. In Proceedings of the International Working Conference on Advanced Visual Interfaces (pp. 264–271). ACM.

  4. Andrienko, N., & Andrienko, G. (2006). Exploratory analysis of spatial and temporal data: A systematic approach. Berlin: Springer Science & Business Media.

    Google Scholar 

  5. Bandura, R. (2008). A survey of composite indices measuring country performance: Update. Office of Development Studies, New York: United Nations Development Programme.

  6. Barzilai-Nahon, K. (2006). Gaps and bits: Conceptualizing measurements for digital divide/s. The Information Society, 22(5), 269–278.

    Article  Google Scholar 

  7. Barzilai-Nahon, K., Rafaeli, S., & Ahituv, N. (2004). Measuring gaps in cyberspace: Constructing a comprehensive digital divide index. In Workshop on Measuring the Information Society, the conference of Internet Research (Vol. 5).

  8. Bertin, J. (1981). Graphics and graphic information processing. Berlin: Walter de Gruyter.

    Google Scholar 

  9. Bertini, E. (2011). Review: OECD’s Better Life Index. Retrieved September 23, 2015 from http://fellinlovewithdata.com/reviews/review-better-life-index.

  10. Borkin, M. A., Bylinskii, Z., Kim, N. W., Bainbridge, C. M., Yeh, C. S., Borkin, D., et al. (2016). Beyond memorability: Visualization recognition and recall. IEEE Transactions on Visualization and Computer Graphics, 22(1), 519–528.

    Article  Google Scholar 

  11. Brehmer, M., & Munzner, T. (2013). A multi-level typology of abstract visualization tasks. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2376–2385.

    Article  Google Scholar 

  12. Burch, M., & Weiskopf, D. (2014). On the benefits and drawbacks of radial diagrams. In W. Huang (Ed.), Handbook of human centric visualization (pp. 429–451). New York: Springer.

  13. Cherchye, L., Moesen, W., Rogge, N., & Van Puyenbroeck, T. (2007). An introduction to ‘benefit of the doubt’ composite indicators. Social Indicators Research, 82(1), 111–145.

    Article  Google Scholar 

  14. Colecchia, A., Pattinson, B., & Atrostic, B. (2000). Defining and measuring electronic commerce. Document de discussion de la DSTI/OCDE.

  15. Cukier, J. (2011). Can data visualization help build democracy? XRDS: Crossroads, ACM Magazine for Students, 18(2), 26–30.

    Article  Google Scholar 

  16. Dickson, G. W., Senn, J. A., & Chervany, N. L. (1977). Research in management information systems: The Minnesota experiments. Management Science, 23(9), 913–923.

    Article  Google Scholar 

  17. Diehl, S., Beck, F., & Burch, M. (2010). Uncovering strengths and weaknesses of radial visualizations—an empirical approach. IEEE Transactions on Visualization and Computer Graphics, 16(6), 935–942.

    Article  Google Scholar 

  18. Dutta, S, & Bilbao-Osorio, B. (2012). The Global information technology report 2012: Living in a hyperconnected world. World Economic Forum.

  19. Elting, L. S., Martin, C. G., Cantor, S. B., & Rubenstein, E. B. (1999). Influence of data display formats on physician investigators’ decisions to stop clinical trials: Prospective trial with repeated measures. BMJ, 318(7197), 1527–1531.

    Article  Google Scholar 

  20. Few, S. (2011). Data blooms in beauty and truth. Retrieved September 23, 2015 from http://www.perceptualedge.com/blog/?p=1044.

  21. Fuchs, J., et al. (2013). Evaluation of alternative glyph designs for time series data in a small multiple setting. In Proceedings of the SIGCHI conference on human factors in computing systems.

  22. Google Public Data Explorer. Retrieved January 15, 2016: https://support.google.com/publicdata/answer/1100640?hl=en.

  23. Gnaldi, M., & Ranalli, M. G. (2015). Measuring University performance by means of composite indicators: A robustness analysis of the composite measure used for the benchmark of Italian Universities. Social Indicators Research, 129(2), 1–17.

    Google Scholar 

  24. Goldberg, J., & Helfman, J. (2011). Eye tracking for visualization evaluation: Reading values on linear versus radial graphs. Information Visualization. https://doi.org/10.1177/1473871611406623.

    Article  Google Scholar 

  25. Gratzl, S., Lex, A., Gehlenborg, N., Pfister, H., & Streit, M. (2013). Lineup: Visual analysis of multi-attribute rankings. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2277–2286.

    Article  Google Scholar 

  26. Hoskins, B. L., & Mascherini, M. (2009). Measuring active citizenship through the development of a composite indicator. Social Indicators Research, 90(3), 459–488.

    Article  Google Scholar 

  27. Index, L. P. (2014). The 2014 Legatum Prosperity Index.

  28. ITU. ICT-Eye. Retrieved September 23, 2015 from http://www.itu.int/net4/itu-d/icteye/.

  29. Joint Research Centre-European Commission. (2008). Handbook on constructing composite indicators: Methodology and User guide. OECD publishing. http://www.keepeek.com/Digital-Asset-Management/oecd/economics/handbook-on-constructing-composite-indicators-methodology-and-user-guide_9789264043466-en#page42.

  30. Keim, D. A., Schneidewind, J., & Sips, M. (2004). CircleView: A new approach for visualizing time-related multidimensional data sets. In Proceedings of the working conference on advanced visual interfaces (pp. 179–182). ACM.

  31. Kelvin, W. T. (1883). Electrical units of measurement. Popular Lectures and Addresses (1889), 1, 80–81.

    Google Scholar 

  32. Mackinlay, J. (1986). Automating the design of graphical presentations of relational information. Acm Transactions on Graphics (Tog), 5(2), 110–141.

    Article  Google Scholar 

  33. Maheshwari, D., & Janssen, M. (2013). Measurement and benchmarking foundations: Providing support to organizations in their development and growth using dashboards. Government Information Quarterly, 30, 83–93.

    Article  Google Scholar 

  34. Meyer, J., Shinar, D., & Leiser, D. (1997). Multiple factors that determine performance with tables and graphs. Human factors, 39(2), 268–286.

    Article  Google Scholar 

  35. Munzner, T. (2014). Visualization analysis and design. Boca Raton: CRC Press.

    Google Scholar 

  36. Nardo, M., Saisana, M., Saltelli, A., & Tarantola, S. (2005a). Tools for composite indicators building. European Commission Joint Research Centre. Institute for the Protection and the Security of the Citizen, Econometrics and Statistical Support to Antifraud Unit, I-21020 Ispra (VA) Italy, Report number: EUR, 21682.

  37. Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., & Giovannini, E. (2005b). Handbook on constructing composite indicators. Berlin: Springer.

    Google Scholar 

  38. Perin, C., Vuillemot, R., & Fekete, J. D. (2014). A table!: Improving temporal navigation in soccer ranking tables. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 887–896). ACM.

  39. Petrović, M., Bojković, N., Anić, I., & Petrović, D. (2012). Benchmarking the digital divide using a multi-level outranking framework: Evidence from EBRD countries of operation. Government Information Quarterly, 29(4), 597–607.

    Article  Google Scholar 

  40. Plaisant, C. (2005). Information visualization and the challenge of universal usability. In J. Dykes, A. MacEachren & M. J. Kraak (Eds.), Exploring geovisualization (pp. 53–82) Oxford: Elsevier.

  41. OECD Better Life Index. Retrieved April 29, 2016 from http://www.oecdbetterlifeindex.org/.

  42. Robertson, G., Fernandez, R., Fisher, D., Lee, B., & Stasko, J. (2008). Effectiveness of animation in trend visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1325–1332.

    Article  Google Scholar 

  43. Saisana, M. & Tarantola, S., (2002). State-of-the-art report on current methodologies and practices for composite indicator development. Citeseer.

  44. Saltelli, A. (2007). Composite indicators between analysis and advocacy. Social Indicators Research, 81(1), 65–77.

    Article  Google Scholar 

  45. Sciadas, G. (2004). International benchmarking for the information society. In ITU-KADO digital bridges symposium.

  46. Shneiderman, B. (1996). The eyes have it: A task by data type taxonomy for information visualizations. In IEEE symposium on visual languages, 1996. Proceedings (pp. 336–343). IEEE.

  47. Stapleton, L. M., & Garrod, G. D. (2008). Policy preceding possibility? Examining headline composite sustainability indicators in the United Kingdom. Social Indicators Research, 87(3), 495–502.

    Article  Google Scholar 

  48. Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127–145.

    Article  Google Scholar 

  49. Tversky, B., Morrison, J., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human–Computer Studies, 57(4), 247–262.

    Article  Google Scholar 

  50. UNDP (2015). Human Development Index (HDI). Google Public Data Explorer. Retrieved September 23, 2015 from http://hdr.undp.org/en/data-explorer.

  51. Van Wijk, J. J. (2006). Bridging the gaps. IEEE Computer Graphics and Applications, 26(6), 6–9.

    Article  Google Scholar 

  52. Wehrend, S., & Lewis, C. (1990). A problem-oriented classification of visualization techniques. In Proceedings of the 1st conference on Visualization’90  (pp. 139–143). IEEE Computer Society Press.

  53. World Wide Web Foundation. Web Index. 2014. Retrieved September 23, 2015, from http://thewebindex.org/data/?indicator=INDEX&country=ALL.

  54. Yi, J. S., ah Kang, Y., Stasko, J. T., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1224–1231.

    Article  Google Scholar 

Download references

Acknowledgements

This research was done as part of the National Israel ICT project by The Center of Internet Research (http://infosoc.haifa.ac.il) supported by the Israel Internet Association-ISOC-IK; the I-CORE Program of the Planning and Budgeting Committee and The Israel Science Foundation (1716/12); and the Samuel Neaman Institute for National Policy Studies.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yael Albo.

Appendices

Appendix 1: List of Analyzed CIs Visualization, Retrieved on 7/2015–3/2016

No. CI name Views Category Organization
1 Better Life Index 9 General OECD
2 The Good Country Index 3 General Developed and funded by Simon Anholt
3 Global Competitiveness Report 2 Economy World Economic Forum
4 Index of Economic Freedom 5 Economy The Heritage Foundation
5 Bloomberg Innovation Index 1 Economy Bloomberg company
6 PISA-Program for Int. Student Assess 4 Education OECD
7 EPI—Environmental Performance Index 6 Environment The Yale Center for Environmental Law & Policy
8 Sustainable Society Index (SSI) 5 Environment the Sustainable Society Foundation
9 Environmental Vulnerability Index (EVI) 2 Environment The South Pacific Applied Geoscience Commission (SOPAC), the United Nations Environment Programme (UNEP) and their partners
10 Euro health consumer index (EHCI) 3 Health Health Consumer Powerhouse
11 Global Hunger Index (GHI) 3 Health The International Food Policy Research Institute
12 Global Corruption Barometer and Corruption Perceptions Index 2 Politics Transparency International
13 Worldwide Press Freedom Index 3 Politics Reporters Without Borders
14 Worldwide Governance Indicators (WGI) 5 Politics The World Bank Group
15 Global Militarization Index 2 Politics Bonn International Center for Conversion
16 World Justice Project Rule of Law Index 4 Politics The World Justice Project (WJP)
17 The Global Peace Index 3 Politics The Institute for Economics and Peace (IEP)
18 Human Development Index 10 Society United Nations Development Programme
19 Quality of Life Index 5 Society Numbeo
20 SIGI-social inst. and gender index 2 Society OECD
21 Save the Children 1 Society Save the Children International
22 Global Slavery Index 4 Society the Walk Free Foundation
23 KOF Globalization Index 3 Society KOF, ETH Zürich
24 Global Gender Gap Report 4 Society World Economic Forum
25 Legatum Prosperity Index 12 Society Legatum Institute
26 Social Progress Index 7 Society The Social Progress Imperative
27 World Giving Index 4 Society Charities Aid Foundation
28 World Happiness Report 3 Society the United Nations Sustainable Development Solutions Network
29 IDI- ICT Development Index 5 Technology The United Nations International Telecommunication Union
30 UN e-Government 6 Technology United Nations Department of Economic and Social Affairs (UNDESA)
31 The Web Index 8 Technology the World Wide Web Foundation
32 the global innovation index 5 Technology Cornell University, INSEAD, and the World Intellectual Property Organization
33 Networked Readiness Index-NRI 16 Technology World Economic Forum
34 Academic Ranking of World Universities 3 Education Shanghai Ranking Consultancy
35 QS University rankings 4 Education QS Quacquarelli Symonds
  Total 164   

Appendix 2: CI Visualization Framework Components—A Short Description

Component Section Short description
What 3.1 CI Data types
 Items 3.1.1 The measured organizations (e.g. cities, countries, universities etc.)
 Indices 3.1.2 The different measurements that reflect the multidimensional phenomena
 Values 3.1.3 Item’s performance in a measurement, expressed by ranks, scores, or raw data.
 Time 3.1.4 Performance evaluation timing, usually at regular intervals (e.g. once a year)
Why 3.2 CI domain questions. The goals of the CI visualization use
 High-level tasks 3.2.1 High-level goals for interacting with a CI visualization tool
 Consume 3.2.1.1 Consuming existing CI information tasks: Present, Discover and Enjoy
 Produce 3.2.1.2 Using a CI visualization to generate new material: Derive and Record
 Low-level tasks 3.2.2 Low-level tasks described by items, indices and time
 Items 3.2.2.1 Sub-tasks distinguished by the number of items in focus: Ranking, Overview, Profile, Compare and Correlate
 Indices 3.2.2.2 Single vs. multiple indices in focus of relevant sub-tasks
 Time 3.2.2.3 Existence of tasks’ focus in trend, i.e. change in values over time
How 3.3 Design choices for constructing a CI visualization
 Encode 3.3.1 Visual mapping and visualization methods for CI data types
 Mapping 3.3.1.1 Matching of CI data to appropriate marks and visual channels
 Methods 3.3.1.2 Visualization techniques suitable for CIs
 Manipulate 3.3.2 User interactions that change the view over time: Filter, Select, Navigate, Sort and Self-Encode
 Facet 3.3.3 Splitting the display into multiple static CI views

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Albo, Y., Lanir, J. & Rafaeli, S. A Conceptual Framework for Visualizing Composite Indicators. Soc Indic Res 141, 1–30 (2019). https://doi.org/10.1007/s11205-017-1804-0

Download citation

Keywords

  • Framework
  • Visualization
  • Composite indicator