Introduction

This paper presents work currently being carried out in California on evaluating the market effects of investor-owned utility (IOU)-sponsored energy efficiency programs. In this context and as stated in California’s energy efficiency evaluation protocols ([CPUC] California Public Utilities Commission 2006), a market effect is a change in the structure of a market or the behavior of participants in a market that is reflective of a change in the adoption of energy-efficient products, services, or practices, and is causally related to the programs being examined. The market effects reflect the actions taken by participants (direct impacts and participant spillover) and by non-participants (non-participant spillover). Currently, many evaluations of energy efficiency programs focus on the direct impacts of programs; as resource acquisition programs, the “spillover” of these programs is often not evaluated. In contrast, the evaluation of market transformation programs looks at all market effects and does not distinguish direct effects from indirect effects, such as spillover. In California, the evaluation of market effects received some attention in the late 1990s; however, the energy crisis of 2000/2001 forced utilities and state government to re-focus on resource acquisition programs, which focus more on direct and near-term impacts. In the last year, however, there has been a resurgence of interest in market effects and market transformation in California (e.g., [CPUC] California Public Utilities Commission 2008) and the California Public Utilities Commission (CPUC) has invested significant resources in measuring market effects, and this may lead to significant advances in the field.

The CPUC recently prepared a long-term energy efficiency strategic plan that reflects the efforts of many stakeholders over the past year to collaboratively develop short-, mid-, and long-term strategic initiatives with a focus on market transformation ([CPUC] California Public Utilities Commission 2008). In parallel with this process, the CPUC (in CPUC Decision 07-10-032, October 18, 2007 ([CPUC] California Public Utilities Commission 2007)) directed its staff and consultants to examine the market effects from California’s energy efficiency programs, as a result of (1) direct effects from participants installing measures, (2) participant spillover, and (3) non-participant spillover. To examine non-participant spillover, the CPUC commissioned three market effects studies on programs that promote: (1) compact fluorescent lamps (CFLs), because of the large amount of resources devoted to promoting CFLs; (2) residential new construction (RNC), because of the long-standing programs that have tried to transform the market; and (3) high-bay lighting (HBL), because it is a relatively new target area in California.Footnote 1 It is important to note that these programs were not designed as market transformation programs. Also, the objective of these studies was not to have the CPUC attribute any savings from market effects to the utilities at this point in time, but rather to examine the feasibility of measuring any market effects that may exist. Working with the CPUC who is administering these studies, the California Institute for Energy and the Environment (CIEE) developed study plans for each of these studies to examine the market effects of the IOU programs that were implemented during 2006–2008.

In this paper, we first outline an approach for conducting market effect studies that includes the six key steps that were developed in the study plans: (1) a scoping study that characterizes a particular market, reviews relevant market effects studies, develops integrated market and program theories, and identifies market indicators; (2) analysis of market evolution, using existing data sources; (3) analysis of market effects, based on sales data and interviews with key market actors; (4) analysis of attribution; (5) estimation of energy savings; and (6) assessment of sustainability (i.e., the extent to which any observed market effects are likely to persist in the absence or reduction of public intervention, and thus has helped to transform the market). While the market effects studies have not been completed, we believe that this framework for evaluating market effects will be of value for those interested in using this approach for evaluating the impact of energy efficiency programs on the marketplace. Based on results to date on the three market effects studies described above, we describe the challenges in conducting this type of analysis and provide some suggestions for future research on the evaluation of market effects.

Approach for evaluating market effects

CIEE developed study plans for each of the market effects studies (e.g., Prahl 2008). Each of the studies then developed a Work Plan that was reviewed by CIEE, CPUC staff, and CPUC consultants. After the Work Plan was approved, the contractors developed a Scoping Study and Work Plan that was reviewed by the public and then implemented. All three studies are pursuing the following steps in evaluating market effects, though the emphasis varies among the studies (see below). As noted below, as of June 2009, the three market effects studies are underway: the CFL market effects study is farthest along, followed by the RNC market effects study, and the HBL market effects study. Accordingly, most of the examples in this paper are taken from the CFL study.

  1. Step 1.

    Prepare a scoping study

California’s EM&V protocols ([CPUC] California Public Utilities Commission 2008) for market effects evaluations emphasize the importance of performing a scoping study before actually embarking on a market effects study. In the words of the protocols:

“The appropriate approach for a market effects study cannot be readily determined without a scoping study to define the market to be studied, develop a market theory to test in the analysis, assess data availability for the market effects study, specify a model of market change, develop a methodology for data collection and recommend an analysis approach.” (p. 149.)

A later passage in the market effects protocol succinctly summarizes the required components of a scoping study when performed at an enhanced level of rigor, as follows:

“Define the market by its location, the utilities involved, the equipment, behaviors, sector and the program years of interest. Develop market theory and logic model. Detail indicators. Identify available secondary data and primary data that can be used to track changes in indicators. Outline data collection approach. Recommend hypotheses to test in the market effects study. Recommend the analysis approach most likely to be effective.” (p. 150.)

CIEE’s study plan for each of the market effects studies was the first step in developing the scoping plan, which was modified in the contractors’ work plan and which included all of the components summarized above. For example, the CFL scoping study (The Cadmus Group, Inc et al. 2008) included the following elements: (1) characterization of CFL market using existing data sources, (2) review of CFL market effects studies from other states, (3) development of market and program theoriesFootnote 2 that are integrated with one another, (4) the study approach, and (5) a list of detailed market indicators that were to be studied.

The scoping studies led to an increased understanding of the markets for each of these technologies and led to some changes in the conduct of the studies. For example, the CFL scoping study led to the following changes in the larger study: (1) a sharpened focus on the effects of the Upstream Lighting Program (ULP) rather than on CFL programs more broadly, as the ULP was found to account for the vast majority of all in-program sales in California; (2) a reduction in the weight attached to the regression analysis vis-a-vis the comparison state approach, since the data available for conducting regression analysis were only available for 1 year and covered only 75% of all CFL sales nationally (and likely fewer in California); and (3) the rejection of a comparison store approach, since California’s programs were focused on promoting CFLs in non-traditional channels (e.g., small grocery stores), in contrast to states without mature programs where CFLs were more commonly sold in traditional channels (e.g., mass merchandising stores).

The RNC scoping study led to a clearer articulation of the possible ways for the RNC programs to achieve the long-term goals of reduced energy use, demand, and emissions: (1) by improving compliance with existing code, (2) by facilitating construction that is more efficient than required by the current code, and (3) by contributing to code upgrades. By identifying the linkages between program activities and these expected long-term outcomes, the logic modeling process laid out indicators of short- and medium-term outcomes that could be measured, and pointed the way towards how to attribute savings to the programs. Furthermore, the scoping study helped to identify the relative importance of key actors involved in the RNC market (e.g., Title-24 consultants) and a target for additional research.

The HBL scoping study also led to a clearer articulation of the possible ways for the HBL programs to achieve the long-term goals of reduced energy use, demand, and emissions. For example, incentives for HBL technologies are often leveraged as an “ice-breaker” to both end-users and installation contractors—opening the dialog with the program and for opportunities to implement other higher impact measures. The scoping study helped to identify the relative importance of key actors involved in the HBL market (e.g., contractors were confirmed as the critical decision-maker, but big box do-it-yourself stores, such as Home Depot, had a larger role than previously theorized) and topics for additional research.

  1. Step 2.

    Analyze market evolution

Because market effects generally occur slowly over time, understanding the long-term evolution of the market is critical to any market effects evaluation. Ideally, this is achieved through ongoing evaluation efforts over the course of many years. However, there are times when we do not have this luxury. Instead, we can only conduct a one-shot effort to develop the best understanding of the long-term market effects of the considered energy efficiency programs. As a result, it is necessary to resort to a wide range of existing data sources to reconstruct the evolution of a particular energy efficiency market, both within and beyond California. A central focus of this step is to reconstruct historic trends in actual sales of a particular energy efficiency technology (e.g., CFL) or practice(s) in California. Trends in other key variables such as consumer awareness and attitudes, technology costs, and retailer stocking behavior are also included, where appropriate. For example, the CFL Interim Report (The Cadmus Group, Inc et al. 2009) included the following sources of data in examining the market evolution of CFLs: (1) data on consumer purchases and awareness and CFL retail prices, (2) a qualitative assessment of cumulative historic market effects based on interviews with program managers and stakeholders, and (3) a review of prior California IOU CFL program evaluations.

As another example, the RNC Phase I report (Nexus Market Research, Inc et al. 2009) examined historic trends in: (1) RNC efficiency practices; (2) builder awareness, attitudes and practices; (3) other market actors’ awareness, attitudes, and practices; (4) home buyers’ awareness and attitudes; and (5) incremental costs of efficiency measures. The HBL scoping study revealed step-wise changes in the market for HBL technologies, in which new technologies surpassed and mostly supplanted the previous technologies over a 15-year period, such as mercury vapor giving way to pressurized sodium products, then to metal halides, then to fluorescents. In each case, the progression in the market represented not only broader penetration in terms of energy efficiency but also reflected improvements in other features such as reduced installation costs, better understanding of the lighting applications, enhanced color rendering, and ease of maintenance.

  1. Step 3.

    Analyze market effects

While the analysis of market evolution described above is expected to contribute to the assessment of market effects from California’s energy efficiency programs, the centerpiece of these studies is a quasi-experimental comparison of current actual and baseline sales patterns of particular technologies (e.g., residential new construction), buttressed by interviews with key stakeholders (e.g., manufacturers, retailers, builders) and consumers regarding the market effects of the programs. By “baseline” we mean a hypothetical projection of what sales patterns would have looked like in the complete absence of the specific program(s) promoting that specific technology in California, either now or at any time in the past.Footnote 3

As with the analysis of market evolution, a wide range of market indicators other than sales are of potential interest, such as measures of consumer awareness, attitudes, and behavior; retailer stocking, promotional, and pricing practices; and manufacturers’ business strategies. These market indicators are identified during the development of the program and market theories. The primary objective in developing and measuring these non-sales, market indicators is to build a convincing case regarding market effects by assessing whether or not the indicators have changed in a manner consistent with what would be predicted by the program theory.

The core of the effort to analyze market effects consists of a quasi-experimental comparison of current actual and baseline sales patterns in California (where possible), with the baseline deriving from current sales patterns in a number of alternative comparison areas. Underlying this approach is the assumption that one or more comparison areas can be found that are reasonably representative of what would be happening in California in the absence of public purpose energy efficiency programs. In an ideal world, one would use a more powerful quasi-experimental design, such as a pre-test–post-test comparison design, under which we would compare the change in sales between two periods for the test versus the comparison area. However, because these studies are primarily retrospective studies, for the most part we do not have the luxury of collecting detailed pre-program data. Key to the effort to strengthen validity will be the use of multiple methods both to analyze actual sales patterns or technology practices in California and to develop comparison areas.

  1. Step 4.

    Assess attribution

This step involves sifting through all the evidence developed by the evaluation to make a case regarding the nature of the market effects produced by California’s energy efficiency programs, if any, and, where possible, the total number of sales of a particular technology induced by these market effects that occurred in the years 2006–2008. Based on study plans and work plans, conclusions regarding these issues are to be based on such assessments as the following (this is not an exhaustive list, just illustrative):

  • Whether comparisons between estimates of actual and baseline sales of a particular technology in California for a particular year (or group of years) consistently show significant differences.

  • Whether supply-side informants attribute market effects to the programs, and if so, what kind and how much.

  • Whether the results of the attempt to reconstruct the evolution of the particular energy efficiency technology market within and beyond California are suggestive of long-term market effects.

  • Whether differences in the specific pattern of sales of the particular energy efficiency technology over time under the actual and baseline scenarios show differences that are suggestive of market effects.

  • In the case of CFLs, whether the regression analysisFootnote 4 shows that the programs have long-term pricing effects, and that consumers have responded to these effects.

  • In the case of CFLs, whether the analysis of CFL marketing efforts conducted as part of the marketing and outreach (M&O)Footnote 5 evaluation shows significant marketing impacts on sales, above and beyond the effects of specific CFL programs.

Determining which market changes can be attributed to California’s energy efficiency programs—and, hence, are market effects—is based on the extent to which all of the above findings are consistent with one another and with the program theory developed as part of previous steps. At the end of the day, attribution in the studies will be based on a preponderance of evidence approach, under which the researcher attempts to construct an argument as to just what has transpired based on the convergence of evidence from a wide range of sources, and the consistency of this evidence with the program theory.

  1. Step 5.

    Estimate energy savings

The quantitative results of the analysis of market effects discussed in the previous step are converted into a stream of estimated energy savings. Initial estimates of savings from market effects may be based upon the difference between total actual and baseline sales of a particular technology (e.g., CFLs), with triangulation among the alternative estimates of these two quantities, and adjustments as appropriate based on other evaluation findings. In the case of CFLs, because we are comparing actual CFL sales with a hypothetical estimate of the level of sales that would exist in the historical absence of any CFL programs, the difference between actual and baseline CFL sales represents the current, total, cumulative effects on sales of all programs that have ever been run in California. As such, it does not differentiate between impacts induced recently versus in the past, between different categories of impacts such as direct program or spillover or between impacts induced by one program versus another. As noted earlier, the ultimate objective for the CPUC in this step is to estimate savings from program effects in years 2006–2008 which are not covered in the direct and participant spillover effects being measured by the impact evaluation studies (i.e., non-participant spillover)Footnote 6. In order to reach this objective one must subtract from the initial savings estimate all savings estimates produced in the 2006–2008 impact evaluationsFootnote 7 that: (1) are clearly associated with retail sales of a particular technology; (2) are program induced; and (3) are not counted in other impact evaluation results. In mathematical terms:

  • Non-participant spillover = total program-induced savings − direct savings − participant spillover

  • If one does not care about distinguishing non-participant spillover from other types of spillover, then the equation is simplified:

  • Spillover = total program-induced savings − direct savingsFootnote 8

  1. Step 6.

    Assess sustainability

As defined by California’s EM&V Protocols, sustainability refers to the extent to which the observed market effects can be expected to last into the future. Gaining an understanding of the sustainability of any observed market effects is very helpful in shaping the direction of future programming efforts in any energy efficiency market. For example, in the case of CFLs, particularly critical is the question of whether the huge surge in CFL sales nationally beginning in 2006 would be likely to sustain itself if support programs (in California or elsewhere) were withdrawn or scaled back.

Sustainability analysis is discussed in California’s EM&V protocols:

Identifying changes in market structure and operations, and how the changed market contains mechanisms to sustain them. This could include examining profitability analyses for important support businesses or business operations and how these are maintained without continued program intervention.

The following questions could be asked to help assess the extent to which a market has been transformed (e.g., see Hewitt 2000):

  • Is someone making money by offering it?

  • Has a private market developed to continue the facilitation?

  • Has the profession or trade adopted it as a standard practice?

  • Would it be difficult or costly to revert to earlier equipment or practices?

  • Are end-users requesting or demanding it?

  • Have the risks to private market actors been reduced or removed?

In summary, how will we know when a particular market is self-sustaining? What other indicators should one look for?

Issues faced in the California evaluations

The three market effects studies are underway. The CFL market effects team is farther along and has produced an Interim Report on its initial findings (The Cadmus Group, Inc et al. 2009); the RNC market effects team has finalized its Interim Report (Nexus Market Research, Inc et al. 2009); and the HBL market effects team has prepared a scoping study and is collecting and analyzing their data. As the studies have been implemented, they have been able to successfully follow most of the framework described above. However, they have also diverged a little because of the availability of data: e.g., the RNC market effects study has primarily relied on interviews with key market actors coupled with the on-site assessment of actual efficiency levels to assess market effects, while the CFL market effects study is examining market effects via interviews and through a comparison of sales data in comparison states (see below).

The work thus far has confronted and addressed serious challenges to the evaluation of market effects, as discussed in more detail below:

  • Selecting a comparison state(s) to California for a baseline

  • Availability and quality of data (limiting analyses)

  • Inconsistent patterns of results

  • Conducting market effects evaluations at one point in time, without the benefit of years of accumulated research findings

Some of these issues have also occurred in other states, as discussed in Rosenberg and Hoefgen’s (2009) review of the evaluation of market effects. Furthermore, it is important to note that the market effects studies are ongoing, and some of the methodologies are still being tested. As a result, our analysis below should be treated as “preliminary”; once the studies are completed, we will have better information on the challenges ahead and the needed methodological guidance for evaluating market effects.

Selecting a comparison state(s) to California for a baseline

Quasi-experimental comparisons with non-program states have historically been a mainstay of market effects evaluations. However, choosing a state or a group of states as a baseline, to compare with California, is subject to many caveats. The baseline comparison approach assumes a non-program area that is the theoretical equivalent to California in the absence of a (CFL, RNC, etc.) program. In the case of CFLs, the CFL Market Effects team selected respondents in three comparison states—Georgia, Kansas, Pennsylvania—through random-digit dialing and on-site visits to homes and retailers. They assumed that CFL sales and usage patterns in these state approximated baseline market conditions for California (i.e., sales that would have occurred in California in the absence of IOU program intervention). The comparison states were chosen because they did not have long-term or significant histories of utility- or regional government-sponsored programs to promote CFLs, and because they shared various socio-economic indicators with California.

The primary shortcoming of using this methodology is that no single state is really comparable to California—which is why a group of states was used as a comparison, rather than a single state. Another possible shortcoming of this approach is that manufacturer and retailer sales strategies in-program and non-program states may be interdependent: it is possible that some manufacturers and retailers may make decisions about how to sell CFLs in one state or region based on what they are doing in another state (e.g., California). Similarly, California programs may have impacted the national market and, therefore, may have influenced the baseline comparison states.

Finally, another issue related to this methodology has to do with time. If a market effects study had been conducted 5–10 years ago, the differences between California and the comparison states might have been significant with respect to the purchase of CFLs. However, in recent years, other states are becoming more similar to California with respect to CFL sales (e.g., as reported in the CFL market effects interim report, significantly more households in the comparison states purchased light bulbs in the past 3 months (57%) than in California (47%)). This may reflect more activity in those areas as well as less activity in California because of past CFL programs that have increased the saturation of CFLs, therefore leading to less demand for CFLs (and sales) in California. If increases in CFL saturation driven by California’s many years of programs are indeed part of the explanation for the relatively low self-reported current purchase rates in California, this would suggest that the comparison states may not provide a valid baseline for California. In the language of quasi-experimentation, the test and comparison groups would be fundamentally non-equivalent (at least in the absence of some approach to controlling statistically for the differences in saturation).

In the case of RNC programs, selecting a suitable comparison state is fraught with difficulty. For another state to function as a model for a hypothetical California baseline, it needs to have similar conditions for RNC except that it has not had utility efficiency programs (or had very little) in the recent past. But California’s size, climate, and policy environment (especially with regard to efficiency aspects of building codes) are at least somewhat unique. It seems unlikely that this method would work in evaluating market effects of the utility RNC programs in California.

It is as yet unclear if an alternative approach will prove viable for the RNC market effects study. So far, this study has relied mainly on interviews with market actors to develop a qualitative understanding of the market effects of a set of utility programs focused on the RNC market, and the evaluators will use those findings to gauge how the IOUs’ RNC programs have affected the efficiency levels of newly constructed homes as observed in field visits. In addition, the study is using a Delphi approach that relies on responses from a number of experts in the field to develop a baseline of how the California market would have evolved without the considered programs (the slump in the housing economy will also be discussed). Whether these two approaches will prove sufficient for quantitative estimates of market effects is as yet uncertain.

In the case of the HBL market effects study, the contractors are starting to determine which comparison states to select (including a comparison with all non-program areas outside of California, in order to avoid potential biases associated with any regional or statewide networks or sales/distribution channels). This is particularly challenging because this type of technology and its application is specialized, changing and differentiated regionally, making it difficult to come up with a good match. For example, the building stock in the comparison states which is amenable to the use of HBL may be different than that in California (large commercial, big box, and industrial facilities). Moreover, the scale of the sum of individual operations in the comparison states will not likely be as large as in California.

Availability and quality of data (limiting analyses)

There is always some bias in the collection of qualitative and quantitative data. For example, some of the qualitative interview results were from interviews with people who participated, evaluated, designed, or implemented a program. In order to ensure more objectivity, interviews with non-participating stakeholders (e.g., manufacturers, retailers, builders, and trade allies) were also conducted. And there is always the potential for self-selection bias in finding respondents willing to answer a survey; complicating this situation is the possibility of respondents providing inaccurate responses to advance their own self-interest through their survey responses. There are ways of “validating” some responses from respondents. For example, to validate reported purchases of CFLs, the CFL Market Effects Team conducted in-home lighting audits to see and verify the number of CFLs each respondent reported through the CFL user survey. The analysis (results are still being analyzed) will provide estimates of recent CFL purchases, CFLs currently installed, and CFLs in storage, and compare these findings with the survey results.

Finding the “perfect” data source is very difficult, if not impossible. For example, in the regression analysis used in the CFL market effects study for assessing the influence of program activity on CFL sales, the dependent variable representing CFL sales data excluded major retail channels (e.g., groceries and small hardware stores that are often targeted by mature CFL programs), through which sales varied widely across the major groups of interest (The Cadmus Group, Inc et al. 2009).Footnote 9 Similarly, the binary term in the regression equation that indicated program versus non-program area was simplistic in that there were, in fact, various levels of programs.Footnote 10 In addition, the lack of reliable, cross-sectional time-series data on CFL sales prevented the team from looking at trends over time and possible lag effects. Finally, in the case of HBL, determining the total installed market share of efficient HBL technologies is not possible.

Inconsistent patterns of results

The interim findings from the CFL market effects study did not provide evidence that additional market effects in the form of energy/demand savings (non-participant spillover) could be unequivocally claimed due to the California IOU programs for the 2006–2008 time period. Instead, different conclusions were derived from different components of the study. For example, while the CFL user survey results indicated that there was little or no difference between California and comparison areas in the 2006–2008 time period (implying no market effects), the interviews with upstream actors did provide some evidence that market effects occurred in prior years. The two analyses are intertwined. For example, it is possible that the baseline sales of CFLs in the other states may have been overestimated because they would have been lower if no-program activity had taken place in California and other states with long-standing programs. Alternatively, the increasing saturation of CFLs in California has led to fewer CFL sales per household compared to household sales in other states. The CFL Market Effects Team has developed a number of other hypotheses to explain these inconsistent findings, and they will attempt to assess the validity of these hypotheses during the remainder of the evaluation.

Conducting market effects evaluations at one point in time, without the benefit of years of accumulated research findings

Based on work to date, the studies discussed above have revealed that the evaluation of market effects should be conducted through a program’s life cycle, rather than at just one point in time. There are risks in making policy decisions from a single snapshot study, and the findings from such studies should be used with caution until a long-term market effects evaluation can be implemented (as noted below). However, we argue that having some information (imperfect as it is) is better than having no information at all when making policy decisions, so that these snapshot studies are of value. Ideally, we encourage time-series research to begin early on in the program’s history rather than waiting later.

Conducting market effects evaluations throughout the program’s history allows the researchers both to study different market indicators as needed at different phases in the program’s life cycle, and to collect time-series data on key variables such as efficient market share. For example, as noted above, a rigorous assessment of program versus estimated baseline CFL sales conducted earlier in the life cycle of the California IOU’s CFL programs might have quantitatively identified potential market effects, if they exist. Furthermore, the total market effects in a given period of time reflect the cumulative effects of many years of program efforts (e.g., not just the efforts in 2006–2008). Conducting periodic market effects evaluations may help to disentangle the market effects from previous years. Of course, these types of studies are laborious and costly, and the implementation of multiple market effects studies creates even more strain on the system. However, several states (e.g., Massachusetts, New York, and Vermont) have been measuring markets over time. On the other hand, planning a long-term market effects evaluation allows evaluation expenses to be spread out over many years of program activity, so that the “annual budget” might not be too burdensome. In addition, the interest in similar data among multiple jurisdictions might provide opportunities for collaborative studies that can reduce research costs.

Conclusions

The study of market effects has significant policy implications. State regulatory agencies and utilities are facing aggressive targets in meeting challenging energy savings goals and greenhouse gas emissions reduction targets (especially in California). In order to reach these goals, the energy efficiency industry must pursue a more comprehensive strategy that includes more market stakeholders (besides program administrators), more funding, and more information, education, and training, in order to transform the market. As a result, a different type of evaluation will have to be conducted—one with a focus on markets, rather than programs. With the promulgation of market transformation programs (as California is attempting to do), the evaluation of market effects will be critical. We envision that these market effects studies will help lay the foundation for the refinement of techniques for measuring the impacts of programs that seek to transform markets for energy efficiency products and practices. It is possible that future market effects studies will not have to distinguish between the different types of spillover (e.g., non-participant versus participant) and direct effects. Their focus will depend on what policy framework is in place. Finally, we have provided a framework for evaluating market effects that we hope others will use for evaluating the market effects of their energy efficiency programs. Once more studies have been completed, we hope that the methods and findings from these studies can be compiled in a centralized database that is accessible to all, so that we can learn from each other.