A case for trading risk in complex conceptual design trade studies

Abstract

Complex conceptual system design trade studies traditionally consider risk after a conceptual design has been created. Further, one person is often tasked with collecting risk information and managing it from each subsystem. This paper proposes a method to explicitly consider and trade risk on the same level as other important system-level variables during the creation of conceptual designs in trade studies. The proposed risk trading method advocates putting each subsystem engineer in control of risk for each subsystem. A risk vector is proposed that organizes many different risk metrics for communication between subsystems. A method of coupling risk models to dynamic subsystem models is presented. Several risk visualization techniques are discussed. A trade study example is presented based upon a simplified spacecraft model. Results from introducing the risk trading methodology into a simulated Collaborative Design Center are presented. The risk trading method offers an approach to more thoroughly consider risk during the creation of conceptual designs in trade studies.

Appendices

Appendix 1: Subsystem development

To represent the spacecraft, four representative subsystems including Communication, Data Handling, Attitude Control, and Power were chosen. The Communication Subsystem is a function-based model that accepts user input for the Antenna Size and Frequency Downlink variables. Function-based subsystem models are function-driven over a range of numeric inputs, while component-based subsystems have a predefined, limited selection of potential subsystem components. Antenna size can range from 1 to 4, and Frequency Downlink can range from 1 to 18, including decimal values. Both of the user input fields have corresponding instructions for the user to maintain input values between the allowable ranges. The Communication Subsystem Power requirements, Mass, and Cost output variables were computed using the formulas shown in Eqs. 4, 5, and 6, respectively.

$$ \hbox{Power}= -\hbox{Antenna Size} + 0.6 \times\hbox{Frequency Downlink} + 3 $$

(4)

$$ \hbox{Mass} = \hbox{Antenna Size} \times 2.5 + 2 $$

(5)

$$ \hbox{Cost} = \hbox{Antenna Size} \times 0.75 + \hbox{Frequency Downlink} \times 0.1 $$

(6)

The Data Handling Subsystem is a component-based model that contains two user inputs in the form of drop-down selection boxes. The first user input, System Complexity, has the options of “simple,” “typical,” and “complex.” The other user input is Spacecraft Bus Configuration which allows the user to select either “one unit,” “two unit,” or “integrated” which refer to the spacecraft having one or two primary computing units and distributed subsystem computers, or an integrated unit that handles all command and data handling functionality. The resulting Data Handling subsystem outputs are shown in Table 2.

Table 2 Data handling subsystem input and output variables

The Attitude Control Subsystem is a component-based model that gives the user control over two inputs via drop-down selection boxes. The inputs are “Stability Method” and “Pointing Method.” Table 3 displays the full range of user-selectable components and the corresponding output variable values.

Table 3 Attitude control subsystem input and output variables

The Power Subsystem is driven by a component-based model that has two inputs, namely “Power Source” and “Energy Source,” which are controllable via drop-down selection boxes. Table 4 presents the range of possible user-selectable input variable combinations and their corresponding output variables. Unlike the other three subsystems, the Power output variable for the Power Subsystem indicates how much power is available to the entire spacecraft system from the power produced within the Power Subsystem.

Table 4 Power subsystem input and output variables

In addition to the four participant-controlled subsystems, a Payload Subsystem was also developed from Wertz and Larson (Wertz and Larson 1999). It is used only to set the mission objectives and requirements. The two possible payloads consist of a weather and navigation package. Only one payload package is selectable at any given time. The Payload Subsystem outputs power, mass, and cost variables. It also produces data on system constraints due to the payload. Table 5 presents the two payload choices and corresponding output data.

Table 5 Payload subsystem input and output variables

Appendix 2: Problem statements

The riskless trade study session used a simple navigation satellite problem. The problem statement is as follows:

This satellite is designed as a navigation satellite to add to the GPS network allowing GPS units to acquire more accurate data on Earth. It carries equipment on board to support its mission. Because of this, the following constraints are given for the mission:

• POWER SUBSYSTEM Power Source: photovoltaic

• COMMUNICATIONS SUBSYSTEM: Frequency downlink: 18

• DATA HANDLING SUBSYSTEM: Required processing: 110

• TOTAL SPACECRAFT: Maximum mass: 30 Maximum cost: 18

The trade study session conducted using the risk trading methodology used a simple weather satellite problem. The problem statement is as follows:

This satellite is designed as a weather satellite to monitor the climate on Earth and carries equipment on board to support its mission. Because of this, the following constraints are given for this mission:

• POWER SUBSYSTEM Energy Storage: primary and secondary battery

• DATA HANDLING SUBSYSTEM: Spacecraft bus: 2 units Required processing: 105

• TOTAL SPACECRAFT: Maximum mass: 27 Maximum cost: 17

Appendix 3: Questionnaire questions

Following each trade study session, participants were asked to fill out a questionnaire individually. The following questions were common to both trade studies.

• Rank the ease of use of each subsystem model on an Easy (1) to Hard (5) scale:

  • Attitude control

  • Data handling

  • Power

  • Communications

• Indicate the ease of use of the two types of subsystem models on an Easy (1) to Hard (5) scale:

  • Component-based

  • Function-based

Additional questionnaire questions were tailored to the risk trading session including:

• Describe any difficulties you encountered while understanding and using the subsystem risk models

• How did you find the transition from conducting trade studies without risk models to trade studies with risk models on an Easy (1) to Hard (5) scale?

• Indicate which set of models produced results in which you feel more confident on a Confident in no-risk model results (1) to confident in models with risk results (5) scale

• Indicate the ease of understanding risk data for each risk visualization technique on an Easy (1) to Hard (5) scale:

  • Fever charts

  • Glyph plots

  • Parallel axis

  • Numeric data

  • Dynamic fault tree

• Is there anything that should have been done differently when transitioning from trade study models not containing risk information to trade study models with components?

• Do you have any additional comments about the study or anything else you wish to convey to the researchers?

Appendix 4: Group discussion questions

Group discussion followed completion of the System Design Report and the questionnaire in both trade study sessions. The following questions were repeated at the end of both sessions:

• Were any of the subsystem models hard to understand and use? Were any particularly easy?

• Did you prefer component-based or function-based subsystem models?

The following questions were used in the group discussion only for the second trade study:

• Did you encounter any difficulties using subsystem models with risk data?

• Were you able to understand the graphical representations of risk? Which did you prefer? (Glyph plot, fever chart, parallel axis plot, dynamic fault tree)

• Is there anything that should have been done differently when transitioning from trade study models not containing risk information to trade study models with risk components?

• Do you have any additional comments about the study or anything else you wish to convey to the researchers?

Appendix 5: Work product template

At the end of both trade study sessions, participants completed brief reports about the work that they had just completed. The following free entry form was provided to the participants:

• Subsystem

• Design Decisions

• Rationale

• Comments

Most participants wrote a paragraph or more for each of the last three questions.

Appendix 6: Questionnaire results

Relevant questionnaire responses are aggregated in this appendix. Identifying information has been removed, and data have been anonymized.

Describe any difficulties you encountered while understanding and using subsystem risk models

• The risk models were extremely helpful and intuitive.

• The risk models were easy to understand but mitigating design problems was difficult.

• The only challenge was to observe how design changes propagated through the subsystem and system models.

How did you find the transition from conducting trade studies without risk models to trade studies with risk models

• Risk is just one more thing to analyze. Engineers should already be doing this.

• Trading risk was straight forward.

• The risk trading method provided more perspective and helps me to feel confident in the final design.

• Risk adds another variable for consideration that can make it more difficult to find a satisfactory solution.

• The risk method is more all-encompassing.

• Risk adds another parameter and is not hard to deal with.

Indicate which set of models produced results in which you feel more confident

• Knowing that design decisions are backed by the science of risk methods such as (FMEA) makes me very confident in our design choices.

Is there anything that should have been done differently when transitioning from trade study models not containing risk information to trade study models with risk components?

• No.

• The brief training was straightforward.

• The transition was straightforward.

• A better understanding of the trade-offs between risk metrics and other system variables would be useful.

Do you have any additional comments about the study or anything else you wish to convey to the researchers?

• The risk trading method and dynamic (FMEA) model are big improvements over existing methods. The method provides for another layer of reliability in the design.

Appendix 7: Group discussion results

Relevant group discussion responses are aggregated in this appendix. Identifying information has been removed ,and data have been anonymized.

• Using the risk trading method was not harder than not using the method.

• I liked the risk trading method. It validates that there is more to the model.

• The resulting design is more complete when using the risk trading method. The resulting design is safer.

• The risk trading method was as easy to use as standard trade study methods. It was more complex but not more difficult.

• I would be more comfortable to show my boss the conceptual design created using the risk trading method. (three participants stated this)

• Using the risk trading method helped me to make design decisions more comfortably.

• It makes sense from an engineering perspective that there is a trade-off between traditional variables such as power, mass, and cost, and engineering risk metrics.

• I am more confident in conceptual designs created using the risk trading method.

• I prefer using the risk trading method over not using the method.