Quality & Quantity

, Volume 48, Issue 3, pp 1455–1472 | Cite as

Combining strengths of methods of party positioning to counter their weaknesses: the development of a new methodology to calibrate parties on issues and ideological dimensions



In this paper we review various methods of party positioning on issues and abstract political dimensions and describe the method that has been developed specifically for the Dutch voting advice application Kieskompas (Election Compass). Basically there are five methods for positioning political parties on political issues and on deeper-lying conflict dimensions: (a) expert surveys (b) voter surveys, (c) party elite surveys, (d) on the basis of roll-call behaviour (e) placements based on manifesto coding (either hand coded or by computer-based calibration). After reviewing the strengths and weaknesses of the various methods, we argue that combining expert placements with text-based calibration (using party manifestos) results in an improved methodology to position political parties on issues and bipolar political dimensions. This new methodology, that was developed for the Voting Advice Application Kieskompas, combines a text-based expert judgement, party-manifesto coding (including computer-based search techniques) and a self-placement by political parties. This process increases the validity and reliability of estimating party positions on political issues.


Methods of party positioning Party manifestos Expert placements  Content analysis Voting advice applications 


  1. Adcock, R., Collier, D.: Measurement validity: a shared standard for qualitative and quantitative research. Am. Polit. Sci. Rev. 95(3), 529–546 (2001)CrossRefGoogle Scholar
  2. Albright, J.J., Mair, P.: Does the number of parties to place affect the placement of parties? Results from an expert survey experiment. Elect. Stud. 30(4), 858–864 (2011)CrossRefGoogle Scholar
  3. Alexa, M., Zuell, C.: Text analysis software: commonalities, differences and limitations: the results of a review. Qual. Quant. 34(3), 299–321 (2000)CrossRefGoogle Scholar
  4. Andersson, H.E., Granberg, D.: On the validity and reliability of self-reported vote: Validity without reliability? Qual. Quant. 31, 127–140 (1997)CrossRefGoogle Scholar
  5. Barone, C., Luccini, M., Sarti, S.: Class and political preferences in Europe: a multi-level analysis of trends over time. Eur. Soc. Rev. 23(3), 373–392 (2007)CrossRefGoogle Scholar
  6. Benoit, K., Laver, M.: Estimating party policy positions: comparing expert surveys and hand-coded content analysis. Elect. Stud. 26(1), 90–107 (2007)CrossRefGoogle Scholar
  7. Benoit, K., Laver, M.: Compared to what? A comment on “A robust transformation procedure for interpreting political text” by Martin and Vanberg. Polit. Anal. 16(1), 101–111 (2008)CrossRefGoogle Scholar
  8. Benoit, K., Laver, M., Mikhaylov, S.: Treating words as data with error, uncertainty in text statements of policy positions. Am. J. Polit. Sci. 53(3), 495–513 (2009)CrossRefGoogle Scholar
  9. Brier, A., Hopp, B.: Computer-assisted text analysis in the social sciences. Qual. Quant. 45(1), 103–128 (2012)CrossRefGoogle Scholar
  10. Budge, I.: Expert judgements of party policy positions: uses and limitations in political research. Eur. J. Polit. Res. 37(1), 103–113 (2000)Google Scholar
  11. Budge, I.: Validating the manifesto research group approach: Theoretical assumptions and empirical confirmations. In: Laver, M. (ed.) Estimating the Policy Positions of Political Actors, pp. 50–65. Routledge, London (2001)Google Scholar
  12. Budge, I., Farlie, D.: Party competition—selective emphasis or direct confrontation? An alternative view with data. In: Daalder, H., Mair, P. (eds.) Western European Party Systems. Continuity & Change, pp. 267–307. SAGE Publications Ltd, London (1983)Google Scholar
  13. Budge, I., Klingemann, H., Volkens, A., Bara, J., Tanenbaum, E.: Mapping Policy Preferences: Estimates for Parties, Electors, and Governments 1945–1998. Oxford University Press, New York (2001)Google Scholar
  14. Budge, I., Pennings, P.: Do they Work? Validating computerised word frequency estimates against policy series. Elect. Stud. 26(1), 136–141 (2007)CrossRefGoogle Scholar
  15. Carmines, E.G., Zeller, R.A.: Reliability and Validity Assessment. Sage University Paper, Beverly Hills (1979)Google Scholar
  16. Carruba, C.J., Gabel, M., Murrah, L., Montgomery, R.C., Schambach, R.: Off the record: Unrecorded legislative votes, selection bias and roll-call vote analysis. Br. Polit. Sci. 36(4), 691–704 (2006)CrossRefGoogle Scholar
  17. Clinton, J., Jackman, S., Rivers, D.: The statistical analysis of roll-call data. Am. Polit. Sci. Rev. 98(2), 355–370 (2004)CrossRefGoogle Scholar
  18. Converse, P.E.: The nature of belief systems in mass publics. In: Apter, D.E. (ed.) Ideology and Discontent, pp. 206–261. Free Press, New York (1964)Google Scholar
  19. Coppedge, M.: A classification of Latin American Political Parties. Kellogg Institute Working Paper No. 244, University of Notre Dame (1997)Google Scholar
  20. DeWinter, L.: Parties and policy in Belgium. Eur. J. Polit. Res. 17(6), 707–730 (1989)CrossRefGoogle Scholar
  21. Dorussen, H., Lenz, H., Blavoukos, S.: Assessing the reliability and validity of expert interviews. Eur. Union Polit. 6(3), 315–337 (2005)CrossRefGoogle Scholar
  22. Downs, A.: An Economic Theory of Democracy. Harper & Row Publishers Incorporated, New York (1957)Google Scholar
  23. Eisinga, R., Franses, PhH: Testing for convergence in left-right ideological positions. Qual. Quant. 30(4), 345–359 (1996)CrossRefGoogle Scholar
  24. Evans, G., Heath, A., Lalljee, M.: Measuring left-right and libertarian-authoritarian values in the British electorate. Br. J. Sociol. 47(1), 93–112 (1996)CrossRefGoogle Scholar
  25. Gemenis, K.: Proxy documents as a source of measurement error in the comparative manifesto project. Elect. Stud. 31(3), 597–604 (2012)CrossRefGoogle Scholar
  26. Gemenis, K.: What to do (and not to do) with the comparative manifestos project data. Polit. Stud. 61, 3–23 (2013)Google Scholar
  27. Groot, L.F.M.: De kwaliteit van stemprogramma’s en de Stemvork. SISWO, Amsterdam (2003a)Google Scholar
  28. Groot, L.F.M.: Criteria to evaluate voting indicators and a recipe for a new one. SISWO, Amsterdam (2003b)Google Scholar
  29. Groot, L.F.M.: Een kritische evaluatie van de StemWijzer 2002. Beleid en Maatsch 30(1), 20–30 (2003c)CrossRefGoogle Scholar
  30. Groot, L.F.M.: Het verrassingseffect van de StemWijzer is niet verrassend. Beleid en Maatsch 30(3), 201–203 (2003d)CrossRefGoogle Scholar
  31. Hansen, E.H.: Back to the archives? A critique of the Danish part of the manifesto dataset. Scand. Polit. Stud. 31(2), 201–216 (2008)CrossRefGoogle Scholar
  32. Helbling, M., Tresch, A.: Measuring party positions and issue salience from media coverage: discussing and cross-validating new indicators. Elect. Stud. 30, 1–10 (2010)Google Scholar
  33. Hooghe, L., Bakker, R., Brigevich, A., de Vries, C., Edwards, E., Marks, G., Rovny, J., Steenbergen, M.: Reliability and validity of the 2002 and 2006 Chapel Hill expert surveys on party positioning. Eur. J. Polit. Res. 49(5), 684–703 (2010)CrossRefGoogle Scholar
  34. Hug, S., Schultz, T.: Using mass survey data to infer political positions. Eur. Union Polit. 6(3), 339–352 (2005)CrossRefGoogle Scholar
  35. Hug, S.: Selection effects in roll call votes. Br. J. Polit. Sci. 40(1), 225–235 (2010)CrossRefGoogle Scholar
  36. Jijkoun, V., Marx, M., De Rijke, M., Van Waveren, F.: Support for decision making: Electoral search, ISLA, Amsterdam. http://staff.science.uva.nl/~mdr/Publications/Files/dir2007-vk.pdf (2007). Accessed 6 December 2012
  37. Keman, H.: Experts and manifestos: different sources—same results for comparative research? Elect. Stud. 26(1), 76–89 (2007)CrossRefGoogle Scholar
  38. Kleinnijenhuis, J., Pennings, P.: Measurement of party positions on the basis of party programmes, media coverage and voter perceptions. In: Laver, M. (ed.) Estimating the Policy Positions of Political Actors, pp. 162–182. Routledge, London (2001)Google Scholar
  39. Kleinnijenhuis, J., Scholten, O. (eds): Nederland vijfstromenland. De rol van de media en stemwijzers bij de verkiezingen van 2006. Bert Bakker, Amsterdam (2007)Google Scholar
  40. Kleinnijenhuis, J., Krouwel, A.P.M.: The nature and influence of party profiling websites. Paper Presented at Politicologenetmaal, Antwerp (2007)Google Scholar
  41. Kleinnijenhuis, J., Krouwel, A.P.M.: Simulation of decision rules for party advice websites. In: Welsch, F., Carrasquero, J.V., Oropeza, A., Chen, C.B. (eds.) Proceedings of the 2nd International Multi-Conference on Society, Cybernetics and Informatics, pp. 138–145. IIIS, Orlando (2008)Google Scholar
  42. Klingemann, H., Hofferbert, R.I., Budge, I.: Parties, policies, and democracy. Westview Press, Oxford (1994)Google Scholar
  43. Klingemann, H., Volkens, A., Bara, J., Budge, I., McDonald, M.: Mapping Policy Preferences II: Estimates for Parties, Electors, and Governments in Eastern Europe, European Union and OECD 1990–2003. Oxford University Press, Oxford (2006)Google Scholar
  44. Klüver, H.: Measuring interest group influence using quantitative text analysis. Eur. Union Polit. 10(4), 535–549 (2009)CrossRefGoogle Scholar
  45. Krippendorff, K.: Content Analysis. An Introduction to its Methodology. Sage Publications, Thousand Oaks (2004)Google Scholar
  46. Krouwel, A., Fiers, S.: From text to position. Text-extraction and authorisation of party positions along dimensions of political competition. In: Paper Presented at Conference Voting Advice Applications (VAAs): Between Charlatism and Political Science. Antwerp (2008)Google Scholar
  47. Krouwel, A.: Party Transformations in European Democracies. SUNY Press, Albany (2012)Google Scholar
  48. Krouwel, A.P.M., Wall, M.: Vitiello: The practicalities of issuing vote advice: a new methodology for profiling and matching. Int. J. Electron. Gov. 5(3/4), 223–243 (2012)CrossRefGoogle Scholar
  49. Krouwel, A.P.M., Wall, M.: From text to the Construction of political party landscapes: A hybrid methodology. In: Kaal, B, Maks, E., Van Elfrinkhof, A. (eds.) From Text to Political Positions: Converging approaches to estimating party positions. Benjamins, Amsterdam (forthcoming)Google Scholar
  50. Latcheva, R.: Cognitive interviewing and factor-analytic techniques: a mixed method approach to validity of survey items measuring national identity. Qual. Quant. 45(4), 1175–1199 (2011)CrossRefGoogle Scholar
  51. Laver, M.: Position and salience in the policies of political actors. In: Laver, M. (ed.) Estimating the policy positions of political actors, pp. 66–75. Routledge, London (2001)Google Scholar
  52. Laver, M., Garry, J.: Estimating policy positions from political texts. Am. J. Polit. Sci. 44(3), 619–634 (2000)CrossRefGoogle Scholar
  53. Laver, M., Benoit, K., Garry, J.: Extracting policy positions from political texts using words as data. Am. Polit. Sci. Rev. 97(2), 311–331 (2003)CrossRefGoogle Scholar
  54. Mair, P.: Searching for the positions of political actors: a review of approaches and a critical evaluation of expert surveys. In: Laver, M. (ed.) Estimating the Policy Positions of Political Actors, pp. 10–30. Routledge, New York (2001)Google Scholar
  55. Marks, G., Hooghe, L., Nelson, M., Edwards, E.: Party competition and European integration in the East and West. Comp. Polit. Stud. 39(2), 155–175 (2006)CrossRefGoogle Scholar
  56. Marks, G., Hooghe, L., Steenbergen, M.R., Bakker, R.: Crossvalidating data on party positioning on European integration. Elect. Stud. 26(1), 23–38 (2007)CrossRefGoogle Scholar
  57. Martin, L.W., Vanberg, G.: A robust transformation procedure for interpreting political text. Polit. Anal. 16(1), 93–100 (2007)CrossRefGoogle Scholar
  58. McDonald, M.D., Mendes, S.M.: The policy space of party manifestos. In: Laver, M. (ed.) Estimating the Policy Positions of Political Actors, pp. 90–114. Routledge, London (2001)Google Scholar
  59. McDonald, M.D., Mendes, S.M., Kim, M.: Cross-temporal and cross-national comparisons of party left-right positions. Elect. Stud. 26(1), 62–75 (2007)CrossRefGoogle Scholar
  60. Mikhaylov, S., Laver, M., Benoit, M.: Coder Reliability and Misclassification in the human coding of party manifestos. Polit. Anal. 20(1), 78–91 (2012)CrossRefGoogle Scholar
  61. Netjes, C.E., Binnema, H.A.: The salience of the European integration issue: three data sources compared. Elect. Stud. 26(1), 39–49 (2007)CrossRefGoogle Scholar
  62. Neurendorf, K.A.: The Content Analysis Guidebook. Sage Publications, Thousand Oaks (2002)Google Scholar
  63. Pelizzo, R.: Party positions or party direction? An analysis of party manifesto data. West Eur. Polit. 26(2), 67–89 (2003)CrossRefGoogle Scholar
  64. Pennings, P.: Assessing the ’Gold Standard’ of party policy placements: is computerized replication possible? Elect. Stud. 30(3), 561–570 (2011)CrossRefGoogle Scholar
  65. Pennings, P., Keman, H.: Towards a new methodology of estimating party policy positions. Qual. Quant. 36(1), 55–79 (2002)CrossRefGoogle Scholar
  66. Poole, K.T., Rosenthal, H.: D-Nominate after 10 years: a comparative update to Congress: a political-economic history of roll call voting. Legislat. Stud. Q. 26(1), 5–29 (2001)CrossRefGoogle Scholar
  67. Poole, K.T.: Changing minds? Not in congress!. Public Choice 131(3–4), 435–451 (2007)CrossRefGoogle Scholar
  68. Power, T.J., Zucco, C.: Estimating ideology of Brazilian legislative parties, 1990–2005: a research communication. Lat. Am. Res. Rev. 44(1), 218–247 (2008)CrossRefGoogle Scholar
  69. Ray, L.: Measuring party orientations toward European integration: results from an expert survey. Eur. J. Polit. Res. 36(2), 283–306 (1999)Google Scholar
  70. Ray, L.: A natural sentences approach to the computer coding of party manifestos. In: Laver, M. (ed.) Estimating the Policy Positions of Political Actors, pp. 149–161. Routledge, London (2001)Google Scholar
  71. Roberts, J.M.: The statistical analysis of roll-call data: a cautionary tale. Legislat. Stud. Q. 32(3), 341–360 (2007)CrossRefGoogle Scholar
  72. Rovny, J.: Who emphasizes and who blurs? Party strategies in multidimensional competition. Eur. Union Polit. 13(2), 269–292 (2012)CrossRefGoogle Scholar
  73. Shively, W.Ph.: The craft of political research. Prentice Hall, Upper Saddle River (1998)Google Scholar
  74. Slapin, J.B., Proksch, S.: A scaling model for estimating time-series party positions from texts. Am. J. Polit. Sci. 52(3), 705–722 (2008)CrossRefGoogle Scholar
  75. Steenbergen, M.R., Marks, G.: Evaluating expert judgments. Eur. J. Polit. Res. 46(3), 347–366 (2007)CrossRefGoogle Scholar
  76. VandenBerg, H., Mehciz, M., Houtkoop-Steenstra, H., Holleman, B.: Opinie-maken of opinie-meten. De rol van stellingvragen in markt- en opinieonderzoek. Stichting voor Culturele Studies, Amsterdam (2002)Google Scholar
  77. VandenBrug, W., VanSpanje, J.: Immigration, Europe and the ’new’ cultural dimensions. Eur. J. Polit. Res. 48(3), 309–334 (2009)CrossRefGoogle Scholar
  78. VanPraag, P.: De stemwijzer: hulpmiddel voor de kiezers of instrument van manipulatie? Lecture Amsterdamse Academische Club, 24–05-2007Google Scholar
  79. Wagner, M., Ruusuvirta, O.: Matching voters to parties: voting advice applications and models of party choice. Acta politica advance online publication (2011). doi:10.1057/ap.2011.29 Google Scholar
  80. Walgrave, S., Nuytemans, M., Pepermans, K.: Voting advice applications and the effect of statement selection. West Eur. Polit. 32(6), 1161–1180 (2009)CrossRefGoogle Scholar
  81. Wiesehomeier, N., Benoit, K.: Presidents, parties and policy competition. J. Polit. 71(4), 1435–1447 (2009)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Department of Political ScienceVU University AmsterdamAmsterdamThe Netherlands

Personalised recommendations