Abstract
Composite indicators (CIs) are common measurements and benchmarking tools used to measure multidimensional concepts such as well-being, education and more. Indicators and sub-indicators are selected and combined to reflect a measured phenomenon. Measurement iterations produce a series of time-oriented data, which stakeholders, as well as the general public, might be interested in interpreting. Visualization of a CI is highly recommended, in order to facilitate interpretation and enhance understanding of indicator components and their evolution over time. In recent years, a variety of CI visualizations have been published including various visualization techniques. Indeed, visualizing a CI is a complex and challenging issue, involving many design choices. However, there is a lack of guidelines and methodological approaches for CI visualization design. We suggest a framework that provides a systematic way of thinking of CI visualizations. The framework is intended for two uses: as a design tool when constructing a new CI visualization, and as an analytic tool for systematically describing, comparing and evaluating CI visualizations. The suggested framework is the outcome of both a top-down process, based on CI construction and information visualization literature, and a bottom-up process, in which 35 existing visualization applications of popular CIs were analyzed. We use Munzner’s visualization analysis and design framework (Munzner in Visualization analysis and design, CRC Press, Boca Raton, 2014) in an adaptive way, considering the specific challenges and characteristics of CI visualizations, in order to develop and discuss a systematic view of the data, tasks and methods for visualizing CIs. We demonstrate the use of the framework with a case study analyzing the popular OECD Better Life Index visualization tool.
This is a preview of subscription content, log in via an institution to check access.
Source: http://www.oecd.org/std/47917288.pdf. (Color figure online)
Source: NRI, World Economic Forum, Switzerland, 2015—http://reports.weforum.org/globalinformation-technology-report-2015/report-highlights/. (Color figure online)
Similar content being viewed by others
Notes
References
Aigner, W., Miksch, S., Schumann, H., & Tominski, C. (2011). Visualization of time-oriented data. Berlin: Springer Science & Business Media.
Albo, Y., Lanir, J., Bak, P., & Rafaeli, S. (2016a). Off the radar: Comparative evaluation of radial visualization solutions for composite indicators. IEEE Transactions on Visualization and Computer Graphics, 22(1), 569–578.
Albo, Y., Lanir, J., Bak, P., & Rafaeli, S. (2016b). Static vs. dynamic time mapping in radial composite indicator visualization. In Proceedings of the International Working Conference on Advanced Visual Interfaces (pp. 264–271). ACM.
Andrienko, N., & Andrienko, G. (2006). Exploratory analysis of spatial and temporal data: A systematic approach. Berlin: Springer Science & Business Media.
Bandura, R. (2008). A survey of composite indices measuring country performance: Update. Office of Development Studies, New York: United Nations Development Programme.
Barzilai-Nahon, K. (2006). Gaps and bits: Conceptualizing measurements for digital divide/s. The Information Society, 22(5), 269–278.
Barzilai-Nahon, K., Rafaeli, S., & Ahituv, N. (2004). Measuring gaps in cyberspace: Constructing a comprehensive digital divide index. In Workshop on Measuring the Information Society, the conference of Internet Research (Vol. 5).
Bertin, J. (1981). Graphics and graphic information processing. Berlin: Walter de Gruyter.
Bertini, E. (2011). Review: OECD’s Better Life Index. Retrieved September 23, 2015 from http://fellinlovewithdata.com/reviews/review-better-life-index.
Borkin, M. A., Bylinskii, Z., Kim, N. W., Bainbridge, C. M., Yeh, C. S., Borkin, D., et al. (2016). Beyond memorability: Visualization recognition and recall. IEEE Transactions on Visualization and Computer Graphics, 22(1), 519–528.
Brehmer, M., & Munzner, T. (2013). A multi-level typology of abstract visualization tasks. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2376–2385.
Burch, M., & Weiskopf, D. (2014). On the benefits and drawbacks of radial diagrams. In W. Huang (Ed.), Handbook of human centric visualization (pp. 429–451). New York: Springer.
Cherchye, L., Moesen, W., Rogge, N., & Van Puyenbroeck, T. (2007). An introduction to ‘benefit of the doubt’ composite indicators. Social Indicators Research, 82(1), 111–145.
Colecchia, A., Pattinson, B., & Atrostic, B. (2000). Defining and measuring electronic commerce. Document de discussion de la DSTI/OCDE.
Cukier, J. (2011). Can data visualization help build democracy? XRDS: Crossroads, ACM Magazine for Students, 18(2), 26–30.
Dickson, G. W., Senn, J. A., & Chervany, N. L. (1977). Research in management information systems: The Minnesota experiments. Management Science, 23(9), 913–923.
Diehl, S., Beck, F., & Burch, M. (2010). Uncovering strengths and weaknesses of radial visualizations—an empirical approach. IEEE Transactions on Visualization and Computer Graphics, 16(6), 935–942.
Dutta, S, & Bilbao-Osorio, B. (2012). The Global information technology report 2012: Living in a hyperconnected world. World Economic Forum.
Elting, L. S., Martin, C. G., Cantor, S. B., & Rubenstein, E. B. (1999). Influence of data display formats on physician investigators’ decisions to stop clinical trials: Prospective trial with repeated measures. BMJ, 318(7197), 1527–1531.
Few, S. (2011). Data blooms in beauty and truth. Retrieved September 23, 2015 from http://www.perceptualedge.com/blog/?p=1044.
Fuchs, J., et al. (2013). Evaluation of alternative glyph designs for time series data in a small multiple setting. In Proceedings of the SIGCHI conference on human factors in computing systems.
Google Public Data Explorer. Retrieved January 15, 2016: https://support.google.com/publicdata/answer/1100640?hl=en.
Gnaldi, M., & Ranalli, M. G. (2015). Measuring University performance by means of composite indicators: A robustness analysis of the composite measure used for the benchmark of Italian Universities. Social Indicators Research, 129(2), 1–17.
Goldberg, J., & Helfman, J. (2011). Eye tracking for visualization evaluation: Reading values on linear versus radial graphs. Information Visualization. https://doi.org/10.1177/1473871611406623.
Gratzl, S., Lex, A., Gehlenborg, N., Pfister, H., & Streit, M. (2013). Lineup: Visual analysis of multi-attribute rankings. IEEE Transactions on Visualization and Computer Graphics, 19(12), 2277–2286.
Hoskins, B. L., & Mascherini, M. (2009). Measuring active citizenship through the development of a composite indicator. Social Indicators Research, 90(3), 459–488.
Index, L. P. (2014). The 2014 Legatum Prosperity Index.
ITU. ICT-Eye. Retrieved September 23, 2015 from http://www.itu.int/net4/itu-d/icteye/.
Joint Research Centre-European Commission. (2008). Handbook on constructing composite indicators: Methodology and User guide. OECD publishing. http://www.keepeek.com/Digital-Asset-Management/oecd/economics/handbook-on-constructing-composite-indicators-methodology-and-user-guide_9789264043466-en#page42.
Keim, D. A., Schneidewind, J., & Sips, M. (2004). CircleView: A new approach for visualizing time-related multidimensional data sets. In Proceedings of the working conference on advanced visual interfaces (pp. 179–182). ACM.
Kelvin, W. T. (1883). Electrical units of measurement. Popular Lectures and Addresses (1889), 1, 80–81.
Mackinlay, J. (1986). Automating the design of graphical presentations of relational information. Acm Transactions on Graphics (Tog), 5(2), 110–141.
Maheshwari, D., & Janssen, M. (2013). Measurement and benchmarking foundations: Providing support to organizations in their development and growth using dashboards. Government Information Quarterly, 30, 83–93.
Meyer, J., Shinar, D., & Leiser, D. (1997). Multiple factors that determine performance with tables and graphs. Human factors, 39(2), 268–286.
Munzner, T. (2014). Visualization analysis and design. Boca Raton: CRC Press.
Nardo, M., Saisana, M., Saltelli, A., & Tarantola, S. (2005a). Tools for composite indicators building. European Commission Joint Research Centre. Institute for the Protection and the Security of the Citizen, Econometrics and Statistical Support to Antifraud Unit, I-21020 Ispra (VA) Italy, Report number: EUR, 21682.
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., & Giovannini, E. (2005b). Handbook on constructing composite indicators. Berlin: Springer.
Perin, C., Vuillemot, R., & Fekete, J. D. (2014). A table!: Improving temporal navigation in soccer ranking tables. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 887–896). ACM.
Petrović, M., Bojković, N., Anić, I., & Petrović, D. (2012). Benchmarking the digital divide using a multi-level outranking framework: Evidence from EBRD countries of operation. Government Information Quarterly, 29(4), 597–607.
Plaisant, C. (2005). Information visualization and the challenge of universal usability. In J. Dykes, A. MacEachren & M. J. Kraak (Eds.), Exploring geovisualization (pp. 53–82) Oxford: Elsevier.
OECD Better Life Index. Retrieved April 29, 2016 from http://www.oecdbetterlifeindex.org/.
Robertson, G., Fernandez, R., Fisher, D., Lee, B., & Stasko, J. (2008). Effectiveness of animation in trend visualization. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1325–1332.
Saisana, M. & Tarantola, S., (2002). State-of-the-art report on current methodologies and practices for composite indicator development. Citeseer.
Saltelli, A. (2007). Composite indicators between analysis and advocacy. Social Indicators Research, 81(1), 65–77.
Sciadas, G. (2004). International benchmarking for the information society. In ITU-KADO digital bridges symposium.
Shneiderman, B. (1996). The eyes have it: A task by data type taxonomy for information visualizations. In IEEE symposium on visual languages, 1996. Proceedings (pp. 336–343). IEEE.
Stapleton, L. M., & Garrod, G. D. (2008). Policy preceding possibility? Examining headline composite sustainability indicators in the United Kingdom. Social Indicators Research, 87(3), 495–502.
Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127–145.
Tversky, B., Morrison, J., & Betrancourt, M. (2002). Animation: Can it facilitate? International Journal of Human–Computer Studies, 57(4), 247–262.
UNDP (2015). Human Development Index (HDI). Google Public Data Explorer. Retrieved September 23, 2015 from http://hdr.undp.org/en/data-explorer.
Van Wijk, J. J. (2006). Bridging the gaps. IEEE Computer Graphics and Applications, 26(6), 6–9.
Wehrend, S., & Lewis, C. (1990). A problem-oriented classification of visualization techniques. In Proceedings of the 1st conference on Visualization’90 (pp. 139–143). IEEE Computer Society Press.
World Wide Web Foundation. Web Index. 2014. Retrieved September 23, 2015, from http://thewebindex.org/data/?indicator=INDEX&country=ALL.
Yi, J. S., ah Kang, Y., Stasko, J. T., & Jacko, J. A. (2007). Toward a deeper understanding of the role of interaction in information visualization. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1224–1231.
Acknowledgements
This research was done as part of the National Israel ICT project by The Center of Internet Research (http://infosoc.haifa.ac.il) supported by the Israel Internet Association-ISOC-IK; the I-CORE Program of the Planning and Budgeting Committee and The Israel Science Foundation (1716/12); and the Samuel Neaman Institute for National Policy Studies.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: List of Analyzed CIs Visualization, Retrieved on 7/2015–3/2016
No. | CI name | Views | Category | Organization |
|---|---|---|---|---|
1 | Better Life Index | 9 | General | OECD |
2 | The Good Country Index | 3 | General | Developed and funded by Simon Anholt |
3 | Global Competitiveness Report | 2 | Economy | World Economic Forum |
4 | Index of Economic Freedom | 5 | Economy | The Heritage Foundation |
5 | Bloomberg Innovation Index | 1 | Economy | Bloomberg company |
6 | PISA-Program for Int. Student Assess | 4 | Education | OECD |
7 | EPI—Environmental Performance Index | 6 | Environment | The Yale Center for Environmental Law & Policy |
8 | Sustainable Society Index (SSI) | 5 | Environment | the Sustainable Society Foundation |
9 | Environmental Vulnerability Index (EVI) | 2 | Environment | The South Pacific Applied Geoscience Commission (SOPAC), the United Nations Environment Programme (UNEP) and their partners |
10 | Euro health consumer index (EHCI) | 3 | Health | Health Consumer Powerhouse |
11 | Global Hunger Index (GHI) | 3 | Health | The International Food Policy Research Institute |
12 | Global Corruption Barometer and Corruption Perceptions Index | 2 | Politics | Transparency International |
13 | Worldwide Press Freedom Index | 3 | Politics | Reporters Without Borders |
14 | Worldwide Governance Indicators (WGI) | 5 | Politics | The World Bank Group |
15 | Global Militarization Index | 2 | Politics | Bonn International Center for Conversion |
16 | World Justice Project Rule of Law Index | 4 | Politics | The World Justice Project (WJP) |
17 | The Global Peace Index | 3 | Politics | The Institute for Economics and Peace (IEP) |
18 | Human Development Index | 10 | Society | United Nations Development Programme |
19 | Quality of Life Index | 5 | Society | Numbeo |
20 | SIGI-social inst. and gender index | 2 | Society | OECD |
21 | Save the Children | 1 | Society | Save the Children International |
22 | Global Slavery Index | 4 | Society | the Walk Free Foundation |
23 | KOF Globalization Index | 3 | Society | KOF, ETH Zürich |
24 | Global Gender Gap Report | 4 | Society | World Economic Forum |
25 | Legatum Prosperity Index | 12 | Society | Legatum Institute |
26 | Social Progress Index | 7 | Society | The Social Progress Imperative |
27 | World Giving Index | 4 | Society | Charities Aid Foundation |
28 | World Happiness Report | 3 | Society | the United Nations Sustainable Development Solutions Network |
29 | IDI- ICT Development Index | 5 | Technology | The United Nations International Telecommunication Union |
30 | UN e-Government | 6 | Technology | United Nations Department of Economic and Social Affairs (UNDESA) |
31 | The Web Index | 8 | Technology | the World Wide Web Foundation |
32 | the global innovation index | 5 | Technology | Cornell University, INSEAD, and the World Intellectual Property Organization |
33 | Networked Readiness Index-NRI | 16 | Technology | World Economic Forum |
34 | Academic Ranking of World Universities | 3 | Education | Shanghai Ranking Consultancy |
35 | QS University rankings | 4 | Education | QS Quacquarelli Symonds |
Total | 164 |
Appendix 2: CI Visualization Framework Components—A Short Description
Component | Section | Short description |
|---|---|---|
What | 3.1 | CI Data types |
Items | 3.1.1 | The measured organizations (e.g. cities, countries, universities etc.) |
Indices | 3.1.2 | The different measurements that reflect the multidimensional phenomena |
Values | 3.1.3 | Item’s performance in a measurement, expressed by ranks, scores, or raw data. |
Time | 3.1.4 | Performance evaluation timing, usually at regular intervals (e.g. once a year) |
Why | 3.2 | CI domain questions. The goals of the CI visualization use |
High-level tasks | 3.2.1 | High-level goals for interacting with a CI visualization tool |
Consume | 3.2.1.1 | Consuming existing CI information tasks: Present, Discover and Enjoy |
Produce | 3.2.1.2 | Using a CI visualization to generate new material: Derive and Record |
Low-level tasks | 3.2.2 | Low-level tasks described by items, indices and time |
Items | 3.2.2.1 | Sub-tasks distinguished by the number of items in focus: Ranking, Overview, Profile, Compare and Correlate |
Indices | 3.2.2.2 | Single vs. multiple indices in focus of relevant sub-tasks |
Time | 3.2.2.3 | Existence of tasks’ focus in trend, i.e. change in values over time |
How | 3.3 | Design choices for constructing a CI visualization |
Encode | 3.3.1 | Visual mapping and visualization methods for CI data types |
Mapping | 3.3.1.1 | Matching of CI data to appropriate marks and visual channels |
Methods | 3.3.1.2 | Visualization techniques suitable for CIs |
Manipulate | 3.3.2 | User interactions that change the view over time: Filter, Select, Navigate, Sort and Self-Encode |
Facet | 3.3.3 | Splitting the display into multiple static CI views |
Rights and permissions
About this article
Cite this article
Albo, Y., Lanir, J. & Rafaeli, S. A Conceptual Framework for Visualizing Composite Indicators. Soc Indic Res 141, 1–30 (2019). https://doi.org/10.1007/s11205-017-1804-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11205-017-1804-0