, Volume 34, Issue 3-4, pp 315-326

Three dilemmas in the integrated assessment of climate change

Purchase on Springer.com

$39.95 / €34.95 / £29.95*

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access

Conclusions

These three dilemmas embody the hardest, most important, and most enduring problems of doing assessment well. None admits simple, obvious solutions. Each can be managed better or worse for any particular assessment endeavor, but doing better requires clear understanding of the purpose of the endeavor. What ways of combining different pieces of disciplinary knowledge, of making projections, and of pursuing policy relevance are more or less appropriate will differ, depending on whether a project seeks to characterize uncertainties and gaps in knowledge; to advise a particular policy choice; to support dialog among policy actors; or to facilitate inquiry into relevant values or goals. Evaluation of the relative emphasis, the methods, and the process of an assessment can only be done relative to some such purpose.

Of course, some pitfalls may be so serious as to thwart any purpose, as Risbey et al.'s discussion of the global modeling movement reminds us. The global models' most obvious pitfalls — inadequate treatment of uncertainty, neglect of economic adjustment, excessive confidence in predictions — have largely been seen and avoided by the current assessment community (though there may be more to be learned even here). But on the subtler questions of how assessment or modeling can contribute most usefully to policy, little progress has been made since the 1970s. Consequently, though assessment has advanced in many ways since then, IA remains at risk of suffering the same fate as the global models: a cycle of early enthusiasm, followed by a reaction of frustration and excessive, undeserved rejection.

Current endeavors in IA have made substantial contributions to identifying and prioritizing knowledge needs, less to informing specific policy choice. Further progress cannot be guided by a single canonical view of what assessment should be and do, but will proceed incrementally down multiple paths. Several paths currently appear promising: analytic approaches to better represent multiple actors, diverse preferences, and multiple valued outcomes; better representation and application of uncertainty, including diverse expert opinion; novel methods to link assessment with policy communities; and broader participation in assessment teams and explicit focus on negotiating and elaborating pragmatic, viable critical standards. Risbey et al.'s call to develop institutions for critical reflection, mutual learning, and self-improvement will be crucial in developing and evaluating the progress made down these paths.

Morgan and Dowlatabadi's checklist for desiderata of IA is a good starting point for a conversation about assessment standards, to which I would propose a few extensions and elaborations. First, there should be not just multiple assessments, but multiple assessment projects using diverse collections of methods and approaches. Second, assessment projects should explore novel methods for connecting their work with the policy community. Third, the approach should be iterative not just within each project, but across assessment projects and between them and the policy community. Fourth, assessors should not be embarrassed by, or seek to disguise, results that are merely illustrative, non-authoritative, and suggestive; these should be acknowledged as such, and the vigorous questioning and critique that will come, including partisan critique, accepted. Do not seek to avoid criticism by mumbling. An important limit to this checklist approach is suggested, though, by the way various writers have groped to define assessment standards by analogy to other domains, revealing how limited is our understanding of how to evaluate assessment. Risbey et al. refer to ‘connoisseurship’, as if assessment is like fine wine; Clark and Majone (1985) refer to artistic criticism, as if assessment is like opera singing. If these analogies are appropriate, then pursuing a single set of critical standards for assessment is at least premature, possibly erroneous. Rather, there should be a diversity of approaches, perhaps so broad that no single set of criteria for excellence could be defined. The pragmatic middle way between the too-limiting application of a single set of standards, and an anarchic refusal to evaluate, will have to be negotiated, defined, and improved incrementally.