Keywords

1 Introduction

In the fast-paced product development economies where the customer demands change rapidly and the market is globally competitive, new products need to be designed and manufactured in a short turnaround time with equal or improved quality. Products that do not meet customer demands rarely make it a market success. Thus, a successful product not only needs to be designed and manufactured in a quicker and cost-effective way, but must be easy and comfortable to use, create positive experiences, and hold minimum risk of injury. The product development strategy needs human-centered approach to incorporate human-factors engineering principles and guidelines. Traditionally, human-factors guidelines are usually applied in the later stage of preliminary design by employing physical prototypes because the detailed information regarding human product interactions are not readily available [1, 2]. Also, it is recommended to use prototyping early in the design stage to identify the human factors requirements [3]. However, the traditional reactive human factors approach which relies on physical prototypes or retrofitting and modifying ergonomically bad/poor products based on human factors guidelines and checklists have major shortcomings such as time-to-market and aggregated cost. Instead, a proactive approach which assesses the comfort and the risk of injury on computational models, as products are designed, has the potential to reduce time and cost associated with physical full-scale prototyping. One way of performing a proactive ergonomic analysis is by employing a full computational prototype or a mixture of computational or physical prototyping techniques during concept development stage. A full computational prototype may consist of a CAD (Computer Aided Design) model representing a product or a workspace and a DHM (Digital Human Modeling) software to evaluate ergonomics. In mixed prototyping, the product or the workspace can either be created using CAD and a human subject is immersed into the CAD environment using a VR (Virtual Reality) headset. Another alternative method is to superimpose CAD environment on a very rudimentary physical prototype using an Augmented Reality (AR).

Computational prototyping is less costly to build, more flexible, and less time consuming compared to full-scale physical prototyping [4, 5]. Though the computational prototyping with DHM is faster to build and is more flexible in comparison to full physical prototyping, it has limitations in modeling and simulating the complex interactions between the human and the product with high accuracy and precision [6]. Thus, one of the challenges that human-centered product designers face while generating and validating a design concept is the dilemma of whether to build a full-physical prototype, a full computational simulation or a combination of both.

This paper proposes a method to partially mitigate the dilemma of types of prototyping to be used in a human centered design approach via VR and a marker-less motion capture system. The study utilizes a Microsoft Kinect based motion capture system with a DHM software to predict the performance of a civilian airplane pilot during an emergency case. Specifically, the motion capture system is used with a VR headset to capture the variation in reaching strategies in an emergency situation. The different strategies of a reaching task adopted by human subject due to an emergency event or routine work can be identified using mixed prototyping where the subject uses a VR headset to interact with workspace and performs the emergency protocols. The comparison between full computational prototype and mixed prototype in the context of type of event, i.e. emergency or routine tasks, can help designers to choose what type of prototyping to be used early in design.

The organization of the paper is as follows: Sect. 2 gives a brief literature review of the types of prototypes available and the challenges that designers often confronts. Also, the research gap is discussed in this section. In Sect. 3, the design methodology and the case study are presented. Results are depicted in Sect. 4 and finally the insight and discussions are presented in Sect. 5.

2 Literature Review

Prototyping is referred as an essential part of the product development and manufacturing process required for assessing the form, fit and functionality of a design before a significant investment in tooling is made [7]. The purpose of prototyping should not be only limited to the evaluation of products but it should be also viewed as part of a design process [8]. Prototyping can help designers to reflect on the design ideas and to explore new ideas. However, one of the challenges human-centered product designers face while generating and validating a design concept is the dilemma of whether to build a full-scale physical prototype, a full computational simulation or a combination of both. Incorrect decisions taken regarding what type of prototyping to use could cause a company to waste time, money and even decrease the quality of the product. Other factors that also need to be considered before building a prototype is the level of complexity and the fidelity of the prototype to address human-product interactions. Also, the number of prototypes to be built and at what stage of the design process they should be built need careful considerations so that effective and efficient assessment of form, fit and functionality of the design can be executed.

The dilemma of using a full-scale physical prototyping, a full computational simulation or a mixed prototyping arises due to a number of reasons. Physical prototypes are often more accurate and effective for representing the shape, composition and functionality of the final product in comparison to virtual prototypes [9, 10]. However, virtual prototyping is less costly to build, more flexible, and less time consuming when compared to physical prototyping [4, 5]. Virtual prototypes used during the conceptual stage can help to reduce the use of physical prototyping, thus reducing time and money [11]. Though physical prototypes made by rapid prototyping are highly accurate, they sometime shrink or have rough surfaces which require further machining. Thus, they may be dimensionally inaccurate, whereas computational prototypes do not have this problem [12]. Also, physical prototypes are difficult or sometimes infeasible to modify or to add further modifications once they are constructed. This inflexibility also poses further problems when new design ideas need to be prototyped after receiving initial design revisions and feedbacks [13]. If the product to be designed requires high level of complex interactions with humans, where precision, accuracy, fidelity becomes a critical concern, then physical prototyping is often the preferred method. On the other hand, if the physical prototype is expensive to build or improbable to construct due its cost and environmental constraints (e.g., as zero gravity in space module) then computational prototyping is preferred. However, the literature does not provide guidelines to decide what type of prototype, i.e. full physical or full computational or a mix of both should be used when assessing the interactions between users and the products. There are also no comprehensive guidelines which categorize the scale or level of interactions between users and products.

There are various methods to create a full computational prototype. This paper focuses on the of CAD models to construct the product or environment and the use of DHM to assess human-product interactions. Digital Human Modeling (DHM) refers to the methodology of digitally representing humans within a computer or virtual environment to facilitate the prediction of performance and/or safety of a worker [1, 2]. DHM manikins consist of visualization of the human body as well as the mathematics and science in the background that enable designers to evaluate the effectiveness of a design, often, focusing on biomechanics (e.g., L4/L5 compression forces) and ergonomics (e.g., NIOSH lifting index) assessments of postures [15, 16]. DHM approach offers designers the capability to visualize and to evaluate product performance early in the design before a physical product is constructed [17, 18]. This proactive approach has the advantage of potentially reducing the need of full-scale physical prototyping and extensive human-subject data collection.

DHM also has shortcomings in taking into account of variation in posture changes (e.g., reach envelope) and fails to identify different strategies that can be used by human to perform the same routine task. This inadequacy is recognized in the literature of DHM and is considered as one of the grand challenges, including the fidelity of anthropometry, realistic visualization, accuracy of posture and motion, number of degrees of freedom (DOF), predictive capabilities, etc. In other words, fidelity concerns in DHM considers “to which extend does DHM represent/replicate the reality?” [19,20,21,22].

On the other hand, mixed prototype combines the advantages of full computational prototypes, i.e. quick evaluation of various design alternatives, flexibility, etc. and advantages of physical prototype such as sense of touch, individual differences, fidelity etc. [23]. One way of creating a mixed prototype is by immersing the human subject into the CAD environment of product or workspace through VR technology, which is defined in Academic Press dictionary of science and technology as ‘a computer simulation of a system, either real or metaphorical, that allows a user to perform operations on a simulated system and shows the effects in real time; e.g., a system for architects might allow the user to “walk” through a proposed building design, displaying how the building would look to someone actually inside it’ [24]. Although VR enables the user to interact with the product or workplace and gives the feeling of being part of the virtual scene, a standalone VR system does not provide haptic feedback [25].

Though mixed prototyping combines both computational and physical prototypes, it is neither as fast and flexible as a full computational prototype and nor as accurate as full physical prototypes. To the best knowledge of the author, there is no literature which addresses the type of prototyping that needs to be created to take into the account of reach strategies in different task conditions (e.g., during routine versus emergency protocol). Hence, in this paper, a comparison of two types of prototypes is made focusing on various reach strategies that can arise due to emergency event. Specifically, the case study of the emergency protocol involves various tasks that a pilot needs to perform in the event of a fire in cockpit, i.e. (a) reaching to the oxygen mask compartment, (b) reaching to the circuit breaker, (c) reaching to the front control panel and (d) reaching to the throttle lever are simulated using both full computational prototype and mixed prototype. More details about the case study is given in Sect. 4.

3 Methodology

Various attempts have been taken to develop a taxonomy of classification of prototypes such as cost, stage of design, level of abstraction or realism, and intended evaluation purpose [26, 27]. The shortcoming of these classifications is that they are unable to cover and distinguish the entire prototype design space [28]. It is out of the scope of this paper to refer to all the classification of prototypes and point out the effectiveness and completeness of each of them. Hence, one of the classifications of a prototype called Hierarchical Morphological Prototyping (HMP) Options Taxonomy is discussed in this paper [28].

Hierarchical Morphological Prototyping’s (HMP) classification of prototyping is based on variety, complexity and fidelity because based on these three types it should be possible to distinguish any complete prototype [28,29,30]. The first level of classification is in the terms of variety, i.e., whether the prototype is physical or non-physical. Non- physical prototypes are made using computational tools such as Computer aided design (CAD), Finite Element Analysis (FEA), Digital Human Modeling (DHM), sketches, pictures, etc. [28]. In the second level of classification, the prototype is divided based on complexity or in other words whether the whole system, e.g. a complete car, or a sub-system, e.g. an engine, or a component, e.g. piston rod is prototyped. The last level of classification is based on fidelity where the prototypes are categorized based on the depth of true representation of the final product. Figure 1 can further help to understand this category. One of the shortcoming of this classification is that HMP has not considered mixed prototyping as one of the variety of prototype.

Fig. 1.
figure 1

Human aspects of design in a digital enterprise are shown on a continuum [31, 32]

Full size image

Complexity of human-product interaction is a concern that designers need to be wary about the amount of interactivity required between user and product during prototyping. From Fig. 1, it can be seen that if there is high interaction between user and product then the practice is to lean towards full prototype, i.e. physical prototype. On the other hand, if the interactivity level is low then full simulation, i.e. computational prototyping is preferred. It is because of the lack of fidelity in the human modeling software to accurately predict and represent human postures and tasks. There are limitations to simulate the lifelike human motion and accurate posture predictions for many complex tasks by the existing digital human modeling tools [31].

As stated above that incomplete classification and reasoning for the type of prototyping exist in the literature, however, there is no taxonomy and reasoning of prototyping to be employed is found for prototyping human-product or human-workspace interactions. In this research, a partial classification of prototyping human-product interactions is given in Fig. 2 based on the concept of Hierarchical Morphological Prototyping (HMP) Options Taxonomy.

Fig. 2.
figure 2

Human-product/workspace prototyping classification

Full size image

Figure 2 provides a classification of prototyping for products or workspaces which require human interaction. The vertical axis represents the level of human-product interaction and horizontal axis represents types of prototypes which is further categorized into level of fidelity. Figure 2 shows that the products which have low level of interactivity can be prototyped by either using digital sketchpad, CAD or simply by paper drawing. However, sophisticated and expensive prototypes need to be created to evaluate products which have high-level of human-product interaction. Full scale computational prototypes created using CAD, FEA and DHM can evaluate products which requires low to mid-level of human interaction. Products with high level of human-product interactions can be evaluated by creating physical prototypes representing the partial or finished product with Motion Capture (MoCAP) and human subject or by using mixed prototypes.

However, no comprehensive guidelines exist which states what type of prototype, i.e. full computational, physical or mixed, should be used based on the level of human-product interaction. Also, guidelines regarding what fidelity and the complexity of the prototype should be created which can take into account of the level of human-product interaction is not comprehensive. Hence, in this study two types of prototypes, i.e. full computation (CAD and DHM) and mixed reality (CAD, VR and human subjects) are studied on the pilot-cockpit case study. This study helps to understand what type of prototype to be used based on the level of human interaction. In the next section, a case study is performed to assess reach posture strategies inside a cockpit using fully computational prototype and mixed prototypes. The goal is to get a more detailed insight on the type of prototypes to be used based on human-product interaction and the level of fidelity required.

4 Case Study

Several civilian air-planes have been reported to have the issue of cockpit fire originating from a heater due to a loose screw which causes jolts of electricity around the cockpit panel, flames, smokes and shattering of inner ply of windscreen [33,34,35,36]. In this event of emergency due to fire in cockpit, pilots wear an oxygen mask and reaches the kill switch/circuit breaker located at the overhead panel and then grabs fire extinguisher to put out the fire [33, 36]. In this paper this emergency event is studied using both fully computational prototype and mixed prototype to see how pilots ‘perform a sequence of task, i.e., (a) tries to reach the oxygen mask, (b) reaching circuit breaker, (c) reaching front panel and (c) reaching the throttle. Cockpit of Boeing 767 is shown in Fig. 3.

Fig. 3.
figure 3

Partial Cockpit of Boeing 767

Full size image

From Fig. 3, it is seen that the oxygen mask is located inside a box which is positioned at the left side of the pilot and the circuit breaker is located on the overhead panel which is 153 cm above from the cockpit floor. The cockpit CAD model represents the pilot seat, oxygen mask box, position of circuit breaker and the distance between the overhead panel and base and other instruments dimensionally accurate to the best ability of the author. Figure 4 shows the detail of the circuit board and top view of Boeing 767 cockpit. The CAD model in Figs. 3 and 4 is used in both of the prototypes to represent a generic cockpit. The full computational prototyping and mixed reality prototyping of Boeing 767 to perform a sequence of task by a pilot is discussed in the following sections.

Fig. 4.
figure 4

Details of circuit breaker and top view of Boeing 767 Cockpit

Full size image

4.1 Computational Prototype

The full computational prototype is created using SolidWorks by CAD modeling and using JACK for ergonomic analysis as can be seen in Fig. 5.

Fig. 5.
figure 5

Full computational prototyping using JACK.

Full size image

As stated earlier, the computational prototype of the cockpit is created using SolidWorks to build a CAD model of the cockpit. Then, it is imported in JACK to perform and analyze the sequence of task a pilot goes through in the event of a cockpit fire emergency. The pilot is represented in JACK using a custom created manikin of 168 cm height and 70 kg weight. JACK is used to analyze: (a) the reach strategy of the pilot while reaching the oxygen mask on his left side of the cockpit, (b) reaching the circuit breaker in the overhead panel, (c) reaching the front panel, and (d) reaching the throttle lever in the event of emergency fire. The results, i.e. reach strategies are discussed in Sect. 5. In the next subsection, the creation and usage of mixed reality is presented.

4.2 Mixed Prototype

The mixed prototype is created using the same CAD file as seen in Fig. 4 and imported in SimLab Composer 8 and projected using HTC Vibe Virtual Reality (VR). A human subject of exact anthropometry as inserted in computational prototyping (JACK) is used in the mixed reality prototyping to mimic a pilot. The same sequence of tasks as performed in computational prototyping are performed by the human subject while seating over a physical chair which has the same height as the pilot seat shown in Fig. 5. The mixed reality setup is shown in Figs. 6 and 7.

Fig. 6.
figure 6

Cockpit CAD viewed in SimLab

Full size image
Fig. 7.
figure 7

Mixed reality setup.

Full size image

As can be seen in Figs. 6 and 7, the human subject is immersed in the virtual reality of the cockpit and placed on the exact position as the manikin is positioned on the pilot’s seat in Fig. 5. The human subject is sitting on physical chair which has the similar dimensions as the pilot seat. The subject is holding wands (virtual controllers) in both hands which mimics grabbing the yoke in a cockpit as shown in the laptop screen of Fig. 7.

The human subject is instructed to do the exact sequence of task as done by the manikin in computational prototyping twice. During the first time, the human subject performs the sequence of task in a routine phase which mimics what a pilot is doing during a regular routine operation. However, during the second time, the subject is prompted to do the same of sequence of task but in an emergency case (e.g., the cockpit is in fire) so that subject’s actions now mimic the actual sequence performed by a pilot in the event of cockpit fire emergency. In both cases, Kinect is used as marker less motion capturing device to capture the various hand and head angles of the human subject in real-time while performing the sequence.

The computational prototype and mixed reality prototype are evaluated by comparing the corresponding angles created by the Manikin in JACK and human subject in mixed reality which is discussed in next section.

5 Results

As mentioned in Sect. 4 that a sequence of task i.e. (a) the reach strategy of the pilot while reaching the oxygen mask on his left side, (b) reaching the circuit breaker in the overhead panel, (c) reaching the front panel and (d) reaching the throttle lever in the event of emergency fire is simulated using both full computational and mixed reality prototyping. Figure 8 shows the task completed using these two prototypes side by side.

Fig. 8.
figure 8

Simulating a sequence of task, reaching (a) oxygen mask box (b) circuit breaker, (c) front panel and (d) throttle lever in Boeing 767cockpit using JACK (left) and mixed reality (right)

Full size image

The intra-class correlation (ICC) analysis is done for mixed reality prototype to see whether the trials are consistent or not. Also, intra-class correlation between the routine and emergency event is measured to see the effect of emergency event on the reach postures. The intra-class correlation is presented in Table 1.

Table 1. Intra-class correlation between types of prototypes and between routine and emergency event
Full size table

The results from these two types of prototyping is presented in Table 2. It shows the angles for Upper Arm Flexion Right/Left, for both type of prototypes and for all the steps in the sequence. Further, the mixed reality prototype has two sets of data which represents a pilot following a routine procedure and the other one is an emergence procedure. JACK does not simulate emergency procedure, so it has only one set of data.

Table 2. Descriptive statistic - comparison of reach posture strategy between full computational prototype and mixed reality prototype.
Full size table

Table 2 shows the descriptive statistical values of reach postures angles for computation and mixed reality prototyping. Full computational prototype provides the same angle value for any particular reach posture. Also, it does not have the capability of simulating the emergency event hence the full computational prototype has only one set of data of zero standard deviation. The mixed reality prototype produces different set of data in each trial and can also account for emergency event.

6 Discussion

Table 2 shows the results obtained by comparing the two types of prototypes used to simulate the routine procedure and emergency event procedure in Boeing 767 cockpit. Table 1 shows that the intra-class correlation is 0.779 between computational prototype and mixed reality prototype for routine reach postures. It is due to the low fidelity of the Kinect to capture angles accurately. The inaccuracy is further exacerbated due to the sitting posture of the human subject. The lack of fidelity in capturing the human subject motion has caused a difference of result between the computational prototype and mixed reality prototype.

The minimum, maximum, mean and standard deviation of these trials are also presented in Table 2. The intra-class correlation within the three trails and four trails of the routine and emergency postures are 0.927 and 0.975 respectively as can be seen in Table 1. ICC results show that there is an excellent consistency within the trials of routine and emergency reach postures respectively. Also form Table 1, the intra-class correlation between the mean values of routine and emergency reach postures are 0.743, which shows a fair correlation.

Figure 9 shows the difference of upper arm flexion angles due to routine and emergency reach postures. It shows that emergency event does affect the normal postures. It is because in an emergency event, pilot tries to perform the task quicker compared to routine work. Another possible reason of difference is that pilots may even assume an uncomfortable posture so that the task can be done immediately.

Fig. 9.
figure 9

Posture analysis in routine and emergency events in mixed reality prototype

Full size image

This result is also in accordance to the study of whether discomfort affects the arm reaching movement while seated [37]. In this paper performing the sequence of task in a normal routine manner is regarded as comfortable reaching strategy and performing the sequence in emergency event is considered as discomforting reaching strategy. It is because in an emergency situation pilots tend to perform the task as fast as possible without considering the comfort of postures whereas in routine procedure pilots performs the tasks by taking the time and reaching through comfortable postures.

This study compares two types of prototypes, i.e. full computational prototype and mixed reality prototype by comparing the reaching motion strategy during routine procedure and emergency procedure in case study of Boeing 767 cockpit. It is found that most of the angle values between computational prototype and mixed reality prototype has fair correlation according to intra-class correlation analysis. This difference is attributed to low fidelity of the marker less motion capture device used and also due to the seated posture of the human subject. However, it is found from intra-class correlation analysis that there is an excellent consistency on correlations results of 0.927 and 0.975 within the trials of the mixed reality prototype for capturing routine work and emergency event reaching postures. It is also found that the intra-class correlation between the routine and emergency event is fairly correlated with correlation value of 0.743. Results suggest that emergency event does have an effect on reach strategies of pilots. The small difference of upper arm flexion angle shows that the human subject performs a different reach strategy during an emergency event as compared to normal routine posture. Hence, these studies show that while simulating emergency situation, a mixed reality prototype can better capture the different human postures compared to digital human modeling approach. It is because DHM always uses the same inverse kinematic algorithm to predict a posture for a given starting and ending point and cannot take into account of different postures that arise due to emergency situation.

This study has compared two prototypes in a case study of Boeing 767 cockpit emergency fire event. However, there are some limitations in this study. The first limitation is that a low fidelity marker-less motion capture device is used to capture the motion of a human subject. The second limitation is that only one human subject is used in the study to show a proof of concept, so the statistical significance of the difference of result is not established. The third limitation is that only the end static posture of the human subject is studied and the whole dynamic postures from start finish are not evaluated. A study of the dynamic posture from start to end between the routine event and emergency event would provide a more insight on how the posture changes due to emergency event.

Future studies will be done to address the limitations mentioned above. A high fidelity motion capture system can provide a more accurate data collection. Also, a comprehensive study with a larger subject population is required to capture variation among people from different anthropometries. Finally, studying the dynamic posture can shed more insight on how the routine posture and emergency posture differs so that it can be taken into account while designing a cockpit.