Introduction

Science and innovation policy (SIP) invokes a wide array of values, from national security to quality of life. Although some of these values are inherently economic or can be reduced to economic surrogates (OECD 1995), economic values rarely serve as preferred end-state values for public policies (Bozeman 2007; Wilsdon et al. 2005). When economic values serve as surrogates for public and social values (e.g., the economic value of conservation of habitat or the marginal cost–benefit of improved health), these indirect indicators often prove problematic for stakeholders (Anderson 1995; Cummings and Taylor 1999; Norton and Noonan 2007).

Despite the fact that a range of values motivating investments into science and technology in general—and nanoscale science and engineering (NSE) in particular—cannot conveniently be evaluated in economic terms, there still exists a long-standing tendency to resort to market-oriented measures and even metaphors in assessing the activities that occur in and around US science and technology laboratories and centers (Bozeman et al. 2007). This practice is partly owing to the traditional economic growth rationale for public investment in science (Solow 1957; Nelson 2000). Additionally, economics presents a set of analytical tools useful in some cases for making and for conceptualizing public and social choices. For decision-makers wishing to consider social values that might inform the public interest, available concepts and theories tend to be unfocused, imprecise and underdeveloped (Bozeman 2007). Moreover, analytical tools focused on public values may seem blunt instruments, poorly attuned to the complexity of cotemporary planning and SIP decision-making.

We demonstrate a novel approach for analyzing public and social values underpinning SIP discourse. We use the case of nanotechnology, or more precisely NSE,Footnote 1 a policy focus in which economic rationales arguably under-serve public and social values and one characterized by multiple, competing criteria and stakeholders (Guston and Sarewitz 2002). This situation, accompanied by widespread “hype” related to NSE (Berube 2005), ambivalence about science policy priorities (Goorden et al. 2008) and rapidly emerging nanotechnologies (PEN 2010) provides an opportunity for prospective research to clarify the baseline of non-economic values that appear to animate major investments in NSE—while adjustments are still possible and before outcomes are known (Sarewitz and Woodhouse 2003). Considering that NSE policy is a moving target, with a churn of competing policy aspirations, innovation objectives and implementation strategies pursued by a diverse set of research and technology performers, some means of public value mapping (PVM)Footnote 2 would undoubtedly be useful, if only one could identify an effective approach to doing so (Bozeman 2007). While our approach is still in the test stage, we suggest it can ultimately be used in other SIP areas to identify underlying public value dimensions and to relate changes in these dimensions over time to changes in such factors as actors’ influence, performers’ activities, and even productivity measures. Perhaps most important, our approach shows some promise for holding the public value foot to the fire by relating the identification and mapping of public values to their enactment in research planning, research activities and, ultimately, social outcomes.Footnote 3

The primary rationales for the PVM of science policy are twofold: (1) SIP is justified in terms of end-state social goals and public values; and yet (2) current research evaluation and SIP analysis methods and techniques, while useful in many important respects, are insufficient for such analysis of research outcomes in relation to public values.

The importance of identifying the public values invoked in support of NSE is clear given the major and growing investment in this area. The President’s 2010 Budget provides $1.6 billion for the National Nanotechnology Initiative (NNI), making the cumulative public investment in NSE since 2001 nearly $12 billion. Cumulative investments in research on the ethical, legal and other societal dimensions of NSE since 2005 reportedly total over $220 million (NSTC 2009), demonstrating continuing policy interest in framing the public and social dimensions of the endeavor (Macnaghten et al. 2005). In the midst of these substantial public expenditures, a complex network of NSE-related policy actors has emerged (Bainbridge 2004). The multitude of NSE funding initiatives and policy actors calls for careful consideration of the adequacy of future and current investments and their connection to envisioned social benefits.

Method

We analyze value-laden statements using quantitative methods in a wide variety of public documents produced by major participants the formal NSE effort. To ensure a robust yet coherent data set, we consider a “stream” of functionally related policy actors that comprise a distributed decision process (Fisher et al. 2006). We thus use documents from the following subgroups: the US Congress as resource allocator; the National Science Foundation (NSF) as both resource recipient and resource allocator; and the NSF-funded NSE laboratories as both resource recipients and research policy agents.

We compile and analyze 1,020 documents with over 100,000 pages of documentation, selected after consultation with SIP scholars and practitioners, including a (now former) US House of Representatives science and technology committee staff member; NSF senior advisors, program officers and staff; researchers familiar with the NSF; and laboratory directors actively funded by the NSF to conduct NSE.

For the legislative level, we review congressional committee reports (N = 189) noting keyword “nano*” from 2000 to 2008. For the funding agent level, we used NSE-specific NSF program solicitation summaries (N = 96) for programs in effect during the study period. The NSF receives a major portion of the federal investment into NSE. The primary mechanisms whereby the NSF—pursuant to the design features of the NNI—supports and substantially funds NSE are Nanoscale Exploratory Research (NER), Nanoscale Interdisciplinary Research Teams (NIRT), and Nanoscale Engineering Centers (NSEC). For the laboratory level, we review abstracts from funded NER, NIRT and NSEC proposals written in response to these same program solicitations (N = 735).

The data we analyze are developed by searching for 84 science value terms (Lacey 1999) across this collection of public NSE documents. The vetting process to identify search terms and structure involved review of sample annotated records and reports, comparison and analysis of sample value statements, and grouping and selection of indicators. We then perform systematic content analysis using NVivo (NVIVO 2009), which is useful in managing large datasets (Smyth 2006), and data reduction using STATA (STATA 2009). We employ principal components analysis to identify the dimensional properties of the textual data obtained from content analysis. Factor analysis techniques—of which principal components analysis is a familiar and mathematically appropriate specification of the factor analytical equation—are applied here for two reasons: Factor analysis proves useful for data reduction (Fabrigar et al. 1999; Tipping and Bishop 1999); and it has as its original objectives (e.g. Thurstone 1948; Guttman 1954; Rummel 1967) the identification of underlying dimensions, usually considered as a test against an empirical or a statistical-properties theory (e.g. Eden and Leviatan 1975). Both of these constitute major challenges for PVM (Bozeman 2007).

Our hypothesis is that the content of these NSE documents will reveal diverse public values, ordered in ways across the three levels of stakeholder subgroups that will be interpretable and theoretically meaningful (i.e. values will cluster in ways that can be related to current PVM and NSE literature).

Results

The result of vetting produced 84 value statement search terms for analysis in all three levels with 100% of the terms found in at least one record at the Congress level, 55% in at least one record at the NSF level and 67% in at least one record at the laboratory level. Table 1 below presents the descriptive statistics representing relative frequency of the 84 search terms in the data set. The six most frequently occurring terms within the data set as a whole relate to development, security and defense, and society.

Table 1 Descriptive statistics
Full size table

The principle components analysis of the 84 search terms, summarized in Table 2 below (as in Van den Beselaar and Leydesdorff 1996), yields three factors with eigenvalues greater than 1.00 and a contribution of at least 5% to the variance explained. The first two factors, which we term Society and Economy and Security and Defense account for 33 and 10% respectively of variance explained. A third factor, termed Energy and Environment explains 8.6% of the variance. We retain a given search term in each factor if it meets the following conditions: loads at 0.50 or higher on the factor, does not load higher than 0.50 on more than one factor, constitutes highest factor loading for the term and is conceptually relevant. Factor 1 (Society and Economy) has 26 terms, Factor 2 (Security and Defense) has 6 terms and Factor 3 (Energy and Environment) has 7 terms. We analyze the internal consistency coefficients for the factor structure. The subscales, Society and Economy, Security and Defense, and Environment and Energy demonstrate good to excellent internal consistency with Cronbach alphas of 0.798, 0.792 and 0.927, respectively.

Table 2 Principle components analysis
Full size table

A related analysis investigates whether the factor structure is consistent within each subgroup, i.e., whether the individual Congress, NSF, and laboratory levels independently reflect similar underlying public value factor structures. We first run a separate factor analysis for each subgroup and find compatible component structures in the top three factors from each level, characterized as follows: Society/Economy, Military/Defense and Energy/Environment for Congress (eigenvalues are 21.8, 10.57 and 9.85, and variance explained is 27.2, 13.2 and 12.3%, respectively); Society/Knowledge Creation, Technological Performance, and Equity for NSF (Eigenvalues are 12.8, 4.59 and 3.56, and variance explained is 43.7, 15.6 and 12.1%, respectively); and Defense/Security, Renewable Energy, and Terrorism/Defense for laboratories (Eigenvalues are 2.35, 2.26 and 2.07, and variance explained is 13.0, 12.4 and 11.4%, respectively).

Following the subgroup factor analyses, we use the covariance coefficient matrices to analyze between subgroup comparisons. Table 3 presents comparison of retained factor scores for subgroup pairs (Congress-NSF, Congress-laboratories and NSF-laboratories) on the factor structure for the combined groups (see Rummel 1967).

Table 3 Comparing factor scores among stakeholder groups
Full size table

None of the Congress specific factors were significantly correlated with the NSF factors. Factor 2 was correlated for Congress and laboratories (p < 0.05) and factors 1 and 2 were highly correlated for NSF and laboratories (p < 0.01).

Discussion

The purpose of this quantitative approach to PVM is to determine possible underlying public value structures justifying publicly funded NSE research across a multi-level network of major policy actors. We present evidence that public value articulations cluster into three major structures, each of which contain dual subclusters: Society and Economy, Security and Defense, and Energy and Environment. These results are consistent with NSE scholarly literature and with prominent government literature outside the data set.

Taken together, the presence of two subclusters within the most pervasive structure, Society and Economy, represents a characteristic and fundamental tension that is found elsewhere in prominent NSE policy discourse. Society denotes a range of normative social, ethical, environmental and governance concerns over the unchecked development of NSE; meanwhile economy denotes NSE’s envisioned commercial success and by extension US leadership in this area. The same uneasy entanglement of social and economic values appears in a number of other sources not considered in our analysis, including federal legislation, federal agency spokespersons, private sector testimony, and the scholarly literature. Public Law 108-153 (US Congress 2003) prescribes the twofold goals of promoting economic competitiveness while being responsive to social concerns (Fisher and Mahajan 2006; Guston 2008). The logical connection proposed between these two seeming “contradictory” goals is that NSE promoters view addressing broad social and ethical concerns as a means to ensure public adoption of NSE, since NSE’s commercial success is partly dependent upon public perceptions and public confidence in the NSE enterprise (Fisher and Mahajan 2006; cf. Cameron 2006). As one federal policy maker stated, commenting on the legislation, “As a business proposition we must identify legitimate ethical and societal issues and address them as soon as possible” (Fisher 2005). Similar links between social concerns and economic aspirations are proposed elsewhere in scholarly literature (Currall et al. 2006) and in public statements from private sector analysts (Nordan 2005).

That Security and Defense would emerge as pervasive in this type of policy content analysis accords with several conditions, including the broadening of security concerns over roughly the same time period studied (Ratner and Ratner 2004), and the potential for NSE to contribute therein (Vandermolen 2006). The 2007 NNI strategic plan lists nine areas in which NSE has the potential to significantly impact numerous fields, including homeland security and national defense (NSTC 2007). Military warfare is arguably undergoing a dramatic revolution, largely enabled by exploitation of emerging technologies (Krepinevich 1994), a designation that is endemic to NSE. NSE has been characterized as spanning all areas of warfare (Altmann 2004) and as surpassing other emerging technologies in its capability to revolutionize warfare (Lovy 2004). The US FY 2008 research and development budget devoted approximately 58% to defense, approximately $475 million of which was projected for NSE in the Department of Defense.

The emergence of an Energy and Environment factor is consistent with the prominence that government literature promoting NSE gives to energy and environmental considerations as well as with scholarly literature that identifies a discursive theme of ‘green nano’. Two of the nine areas that NSE has the “potential to significantly impact” listed in the 2007 NNI strategic plan include “energy” and “the environment” (NSTC 2007); and the second sentence of the preface to the 2004 NNI strategic plan states that NSE has the potential “to increase the efficiency of lighting, enhance the performance of electronic devices, decrease waste and pollution during manufacturing … and provide more cost-effective solar energy conversion” (NSTC 2004). As commentators have observed, ‘green nano’ has been an emphasis in government literature promoting NSE since the late 1990s and is an area that has received tremendous attention that verges on ‘hype’ (Jorgensen and Jorrgensen 2009; Lubick 2009; Schwarz 2009).

In accordance with principal-agent theory for science policy (Braun and Guston 2003), we would expect to find correlation between factors of Congress and NSF (we did not) and between factors of NSF and laboratories (we did). The correlation in the second case may be explained by funding as a mediator or shared incentive. The lack of correlation in the first case may be due to distance between the two groups with respect to communication.

Conclusion

Our method yields interpretable and theoretically meaningful value structures that emerge from a diverse set of documents produced across a multi-level network of research policy subgroups. It thus offers a basis for credible and potentially robust public value mapping of science and innovation policy. Three underlying factors reflect distinctive value themes that are evident elsewhere in related science policy discourse and that are found through alternative scholarly methods. This provides confirmatory evidence of the centrality of the emergent value structures. In particular, the primary theme that emerges is that of a pervasive if not fundamental tension between economic values and non-economic social and public values. This theme reinforces the rationale behind developing a sound approach to public value mapping. Secondary and tertiary value themes animating the US NSE enterprise emerge as national security and defense and as energy and the environment, respectively.

With so many seemingly disparate agencies and programs addressing the volatile and sometimes contentious issue of NSE and its societal and ethical dimensions, methods to identify, categorize and relate numerous, diverse and inter-related values that are invoked to justify massive public expenditures on research and development could substantially inform policymaking, deliberation, and analysis. Methodologically, this novel application of PVM continues previous efforts but expands the quantification and systematic analysis of latent public values. Our use of content analysis and principal components factor analysis on a wide variety of public documents produced within related programs and across a stream of policy enactors—legislature, funding agency, laboratories—is a modest step in the direction of a science of science policy based on something quite different from microeconomic and bibliometric approaches, useful as these are.

The results suggest follow up research. First, it should prove useful to explore whether public values can help anticipate changes in policy agendas, developments, innovation activities and outputs and, ultimately, outcomes. Can PVM signal policy changes and innovation and research activities? Furthermore, our findings suggest a more aggressive use of PVM. Do NSE values identified here correlate to budgetary decisions, research programs and funding awards made to certain types of institutions? To the extent that core values produce policy shifts, there is a role for policy scientists to engage in empirical tests of the relationship between value statements and the host of potentially important outcomes both for the science of NSE and for affected members of society (Jorgensen and Bozeman 2007). Ultimately, it might prove possible to develop policy response functions (Buiter 1981). At the very least, having better methods for analyzing underlying values for prominent research and development areas provides a more unifying framework for further discourse on science and engineering imperatives and priorities.