Evaluation of Space Programs: Select Findings from the OECD Space Forum

Natural resource sustainability, safety, and transport efficiency are quickly growing concerns.


What Do We Want to Evaluate?
Space systems provide an interesting paradox: they are often considered and funded as research and development programs, but act in many cases as key infrastructures delivering unique public and private services, particularly Earth observation systems. When measuring socioeconomic impacts, basic definitions of what to measure vary.

Space Programs
The first full-scale space programs date from the late 1940s to early 1950s. From the start, they consisted of R&D projects to develop technologies and know-how to send objects into space and utilize this new dimension for science and security purposes. Today, institutional space programs worldwide still cover a wide range of technologies (i.e., launchers, satellites, space stations, ground segment) and disciplines (e.g., telecommunications, Earth observation, navigation, and astronomy), sometimes with "accompanying" programs to involve new users (e.g., commercialization of technologies outside the space sector). Space programs are usually undertaken nationally via dedicated agencies, but also often within a bilateral or multilateral international cooperation framework, particularly in the European context. Since 1980s, a number of private actors have conducted their own space programs directly, for profit (e.g., telecommunications satellite operators, commercial launch providers), but always within a regulatory framework put in place by governments (OECD 2005).

Space Applications
"Applications" are the resulting outcomes of many space programs. Sometimes they are actively sought, to develop specific space products and services (e.g., satellite television); on occasion such results are accidental. The data derived and/or signal issued from a large number of programs initiated for purely scientific purposes can be deemed relevant by large communities of users. Today the value chains for space applications vary, depending on the commercial or scientific benefits of the data or signal provided. That is where the distinction between pure R&D programs (set up for a limited period) and applications (often to be set up on an enduring operational basis) becomes blurry at times. It has been historically difficult to shift programs from the science and technology environment to the financially sustainable operational environment.

Space Infrastructure
The term "space infrastructure" encompasses all systems, whether public or private, that can be used to deliver space-based services. These include both the space and ground segments. As identified in OECD (2005), there are two complementary and interlinked space-based infrastructures. The first one focuses on the "front office", i.e., the one that is "user-oriented" and designed to provide information-related services including communications, navigation signals, and Earth observation data to governments and society at large. The second concerns the essential enabling "back office", i.e., the space transport, satellite manufacturing and servicing infrastructure.

Tracing Benefits to Satellites
There are different ways to try and trace socioeconomic benefits from satellites' unique capabilities. One can focus on those capabilities (links, signals or data) or base the analysis at the programmatic or infrastructure level. There are already many efforts underway to try and assess the economic impacts of science and R&D programs. As identified by the OECD Space Forum, some of the methodologies used are still evolving. Applying existing techniques to space programs provides some interesting lessons learned. Two examples are provided below.
The macroeconomic approach is often used in the case of large R&D programs or infrastructure to provide cost-benefit information, via economic input-output analyses. The main objective is to measure the growth of productivity in a region or country generated by the investment. Input-output analysis specifically shows how industries are linked together through supplying inputs for the output of an economy. Factors that can be used to construct indicators of productivity include employment, expenditures, income, production of goods and services, and competitiveness. Such factors are of interest at both the national and regional levels. Results of these analyses are derived from macroeconomic data such as changes in GDP, which can then be compared to changes in capital. The challenge when interpreting the material is to find the causal linkages between the program/infrastructure investments and the rise in productivity.
However, the findings of these studies are sometimes contentious. They are also highly dependent on the choice and evaluation of appropriate variables over long periods, as well as the calculations used to assess their cause and effect mechanisms. As an example, the U.S. Federal Aviation Administration's (FAA) Office of Commercial Space Transportation published a report in 2006 on the impacts of commercial space transportation and related industries in other economic sectors, specifically in terms of revenues and jobs that are generated. The economic impact analysis used an input/output method and the Regional Input-Output Modeling System (RIMS II) developed by the U.S. Department of Commerce, Bureau of Economic Analysis. The space sector, as defined in the study by the FAA, was found to be responsible-via direct, indirect, and induced impacts-for USD 98 billion in economic activity in 2004 and 551,350 derived jobs throughout the U.S. All major U.S. industry sectors were affected positively to some extent (e.g., the information services sector, manufacturing, finance and insurance, healthcare and social assistance). As a comparison, using the same methodology the economic impact of the civil aviation industry was found to be over ten times that of commercial space transportation and enabled industries. Methodology-wise, inputoutput analyses are valuable methods to measure economic impacts. On the other hand, one inevitable drawback of this type of analysis stems from the lack of precise space sector statistics, since the statistical codes used for the study by definition cover more than just space activities .
At the other end of the analytical spectrum, microeconomic analysis studies the behavior of individual organizations, firms, and customers and their interactions, usually determined by market demand and supply. The use of supply and demand curves is, however, not always directly applicable to space systems and their derived applications because of immature products (new technologies) and nonquantifiable demand. A real technical limitation of microeconomic analysis is the daunting task of assessing accurately all the markets liable to be affected by a specific space technology, and not just when it is innovative. Numerous studies of "spin-offs" have been conducted in the U.S. since the 1960s (such as outputs from NASA's Apollo program), notably of the transfers from space-related hardware and know-how to other sectors (e.g., medical imagery). The value of spin-offs is, however, not easily quantifiable, although they provide interesting illustrations of the diffusion of space technologies in different economic sectors.
A combination of macro and micro approaches tends to provide better estimates, although it will still fail to address potentially larger noneconomic impacts. The choice of a specific analytical technique for impact assessment is not random but context-specific; this is particularly true when looking at space applications. The timing and objective of the assessment, as well as the nature and scope of the public R&D funded, are factors that must also be borne in mind when selecting an analytical technique from an existing toolbox.

The Way Forward
One important lesson learned from existing evaluations of space programs is that more work is needed to provide evidence-based information to decision-makers and citizens on the benefits (and limitations) of space applications. Existing methods provide useful hints at actual socioeconomic benefits derived from the space infrastructure, particularly for Earth observation, but with diverse inherent caveats, so there is a need to refine further quantifiable analytical tools. In that context, it remains key to maintain the effort in building the international knowledge base on impact assessments to provide know-how and valid experiences to practitioners (avoiding reinventing the wheel). The OECD Space Forum contributes to this activity and will continue engaging actors in the space community and beyond to explore the broad economic and social dimensions of space-based applications.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.