Health Care Management Science

, Volume 8, Issue 2, pp 157–166 | Cite as

Ranking the ‘Balance’ of State Long-Term Care Systems: A Comparative Exposition of the SMARTER and CaRBS Techniques

  • Malcolm Beynon
  • Martin KitchenerEmail author


The use of attribute sets to rank units of health provision (e.g., states, organizations) against policy goals is an essential task within decision-making and analysis. This paper elucidates and compares two techniques, SMARTER (Simple Multiattribute Rating Technique Exploiting Ranks) and CaRBS (Classification and Ranking Belief Simplex), within an expositional ranking of US states’ long-term care (LTC) systems against the policy goal of providing a balance between (traditionally dominant) institutional care, and alternative home and community-based services (HCBS). While the (more established) SMARTER technique is used primarily for comparative purposes, greater emphasis is placed on elucidating CaRBS which is based on the Dempster-Shafer theory of evidence. It is shown that CaRBS offers four appealing features for health policy analysis: (1) the capacity to rank using either of two confidence measures (DST-related belief and plausibility values), (2) a systematic approach to managing missing data, (3) the production of stable rankings, and (4) the simplex plot method of data representation. In addition to discussing the LTC policy implications of the study findings, the issues of rank order stability and the management of missing data are discussed with respect to the two techniques employed.


ranking CaRBS SMARTER state long-term care systems rebalancing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. J.S. Dyer, P.C. Fishburn, R.E. Steuer, J. Wallenius and S. Zionts, Multiple criteria decision making, multiattribute utility theory: The next ten years, Management Science 38(5) (1992) 645–654.Google Scholar
  2. D.L. Olson, Review of empirical studies in multiobjective mathematical programming: Subject learning and response to nonlinear utility, Decision Sciences 23(1) (1992) 1–20.Google Scholar
  3. D.L. Olson, Comparison of three multicriteria methods to predict known outcomes, European Journal of Operational Research 130 (2001) 576–587.CrossRefGoogle Scholar
  4. H.M. Moshkovich, R.E. Schellenberger and D.L. Olson, Data influences the result more than preferences: Some lessons from implementation of multiattribute techniques in a real decision task, Decision Support Systems 22 (1998) 73–84.Google Scholar
  5. G. Gerig, D. Welti, C.R.G. Guttman, A.C.F. Colchester and G. Szekely, Exploring the discrimination power of the time domain for segmentation and characterisation of active lesions in serial MR data, Medical Image Analysis 4 (2000) 31–42.CrossRefPubMedGoogle Scholar
  6. National Governors Association, Rebalancing Long Term Care Toward Quality Community Living and Healthy Aging Accessed 8/19/04.Google Scholar
  7. R.L. Kane, R.A. Kane, R.C. Ladd and W. Nielson, Variation in state spending for long term care: Factors associated with more balanced systems, Journal of Health Politics, Policy, and Law 23 (1998) 363–390.Google Scholar
  8. M. Kitchener, M. Beynon and C. Harrington, QCA and public services research: Lessons from an early application, Public Management Review 4(4) (2002) 485–504.Google Scholar
  9. M. Kitchener, M. Beynon and C. Harrington, Explaining the diffusion of Medicaid home care waiver programs using VPRS decision rules, Healthcare Management Science 7 (2004) 237–244.PubMedGoogle Scholar
  10. C. Harrington, S. Anzaldo, A. Burdin, M. Kitchener and N. Miller, Trends in state certificate of need and moratoria programs for long term care providers, Journal of Health and Social Policy 19(10) forthcoming (2005).Google Scholar
  11. A. Stewart, M. Cox, J. Teitlebaum and S. Rosenbaum, Beyond Olmstead and Toward Community Integration: Measuring Progress and Change (Washington DC: Center for Health Strategies, 2003).Google Scholar
  12. J. Von Neumann and N.J. Morgenstern, Theory of Games and Economic Behavior (Princeton University Press, Princeton, 1947).Google Scholar
  13. O.I. Larichev, H.M. Moshkovich, A.I. Mechitov and D.L. Olson, Experimental comparing qualitative approaches to rank ordering of multiattribute alternatives, Journal of Multi-Criteria Decision Analysis 2(1) (1993) 5–26.Google Scholar
  14. R. Keeney and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value Tradeoffs (New York: Wiley, 1976).Google Scholar
  15. G.W. Fischer, Range sensitivity of attribute weights in multiattribute value models, Organizational Behavior and Human Decision Processes 62 (1995) 252–266.Google Scholar
  16. W. Edwards and F.H. Barron, SMARTS and SMARTER: Improved simple methods for multiattribute utility measurement, Organisational Behavior and Human Decision Processes 60 (1994) 306–325.Google Scholar
  17. M.J. Beynon, A novel technique of object ranking and classification under ignorance: An application to the corporate failure risk problem, European Journal of Operational Research 167 (2005) 493-517.Google Scholar
  18. M.J. Beynon and K. Buchanan: Object classification under ignorance using CaRBS: The case of the European barn swallow, Expert Systems with Applications 27(3) (2004) 403–415.Google Scholar
  19. A.P. Dempster, A generalization of Bayesian inference (with discussion), Journal of Royal Statistical Society Series B 30 (1968) 205–247.Google Scholar
  20. G.A. Shafer, Mathematical theory of Evidence (Princeton: Princeton University Press, 1976).Google Scholar
  21. W. Edwards, How to use multiattribute utility measurements for social decision making, IEEE Transactions on Systems Man and Cybernetics SMC-7 (1977) 326–340.Google Scholar
  22. P. Domingos, The Role of Occam’s Razor in Knowledge Discovery, Data Mining and Knowledge Discovery 3(4) (1999) 409–425.Google Scholar
  23. J.L. Schafer and J.W. Graham, Missing data: Our view of the state of the art, Psychological Methods 7(2) (2002) 147–177.PubMedGoogle Scholar
  24. X. Huang and Q. Zhu, A pseudo-nearest-neighbour approach for missing data on Gaussian random data sets, Pattern Recognition Letters 23 (2002) 1613–1622.Google Scholar
  25. P.D. Allison, Missing Data, Sage University Paper Series, Quantitative Applications in the Social Sciences, 136 (Sage, Thousand Oaks, 2000).Google Scholar
  26. S.G. West, New approaches to missing data in psychological research: Introduction to the special section, Psychological Methods 6(4) (2001) 315–316.PubMedGoogle Scholar
  27. K.C. Carriere, Methods for repeated measures data analysis with missing values, Journal of Statistical Planning and Inference 77 (1999) 221–236.Google Scholar
  28. W.T.M. Wolters and B. Mareschal, Novel types of sensitivity analysis for additive MCDM models, European Journal of Operational Research 81 (1995) 281–290.Google Scholar
  29. E. Triantaphyllou and B. Shu, On the maximum number of feasible ranking sequences in multi-criteria decision making problem, European Journal of Operational Research 130 (2001) 665–678.MathSciNetGoogle Scholar
  30. M. Pöyhönen and R.P. Hämäläinen, On the convergence of multiattribute weighting methods, European Journal of Operational Research 129 (2001) 569–585.Google Scholar
  31. T. Solymosi and J. Dombi, A method for determining weights of criteria: The centralized weights, European Journal of Operational Research 26 (1986) 35–41.Google Scholar
  32. F.H. Barron and B.E. Barrett, Decision quality using ranked attribute weights, Management Science 42(11) (1996) 1515–1523.Google Scholar
  33. J.R. Doyle, R.H. Green and P.A. Bottomley, Judging relative importance: Direct rating and point allocation are not equivalent, Organisational Behavior and Human Decision Processes 70(1) (1997) 65-72.Google Scholar
  34. R.J. Safranek, S. Gottschlich and A.C. Kak, Evidence accumulation using binary frames of discernment for verification vision, IEEE Transactions on Robotics and Automation 6 (1990) 405–417.Google Scholar
  35. M. Kitchener and C. Harrington, The U.S. long-term care field: A dialectic analysis of institutional dynamics, Journal of Health and Social Behavior 45 (2004) 87–101.PubMedGoogle Scholar
  36. M. Kitchener, T. Ng and C. Harrington, Medicaid home-care expenditure and participation trends during fiscal crisis, Health Affairs 24(1) (2005) 1–7.Google Scholar
  37. M. Kitchener, T. Ng and C. Harrington, Medicaid Home and Community-Based Services: Program Data, 1992–2001 (The Kaiser Commission on Medicaid and the Uninsured, Washington DC, 2003).Google Scholar
  38. M. Kitchener, M. Willmott and C. Harrington, Federal Systems Change Grants to States and Territories: 2001–2004 (University of California, Center for Personal Assistance Services, 2004).Google Scholar
  39. A. Olinsky, S. Chen and L. Harlow, The comparative efficacy of imputation methods for missing data in structural equation modelling, European Journal of Operational Research 151 (2003) 53–79.MathSciNetGoogle Scholar
  40. G. King, J. Honaker, A. Jospeh and K. Scheve, Analyzing incomplete political science data: An alternative algorithm for multiple imputation, American Political Science Review 95 (2001) 49–69.Google Scholar
  41. M.J. Beynon, The MAUT based CARBS System using Belief Functions: Ranking US National League Baseball Teams, The 17th International Conference on Multiple Criteria Decision Analysis, MCDM2004 (Whistler, Canada, 2004).Google Scholar
  42. M. Silver, Business Statistics (McGraw Hill, London, 1992).Google Scholar

Copyright information

© Springer Science + Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Cardiff Business SchoolCardiff UniversityCardiffUK

Personalised recommendations