Advertisement

You Can(’t) Teach an Old Dog New Tricks: Analyzing the Learnability of Manufacturing Software Systems in Older Users

  • Katrin ArningEmail author
  • Simon Himmel
  • Martina Ziefle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9755)

Abstract

Modern manufacturing processes are based on complex computer-aided planning processes, which are provided by CAM (computer-aided manufacturing)-software systems. Due to increased functional capabilities of CAM software, the complexity of these systems and the demands on CAM users are rising. Facing the demographic change (cognitively aging users, retiring of experienced CAM experts who are succeeded by inexperienced users), not only general learnability issues but also user-specific requirements are becoming increasingly important. An online-survey focusing on the learnability of CAM-software, and existing learning environments and strategies in manufacturing practice was conducted (n = 76) and effects of age and CAM expertise were analyzed. Implications for CAM skill acquisition among users of different age and expertise groups were derived.

Keywords

Computer-aided manufacturing software User diversity Age Expertise Learnability Training Survey 

1 Introduction

Todays’ automated product manufacturing processes require complex computer-aided planning processes, which make use of CAx-software systems (computer-aided technologies). CAx-software systems integrate a broad range of components such as CAD (computer-aided design), CAM (computer-aided manufacturing), computer-aided process planning (CAPP), and several other simulation features like FEM (finite elements method), and CFD (computational fluid dynamics) [1]. The increasing number of machine tools and functions, simulation- and visualization features come at a price: CAx software systems have become highly complex. Even with standardized tools and functions, the complexity of CAD/CAM-software has already risen to a level where only highly trained experts are able to effectively use current CAx-software. The development of computer-integrated manufacturing (CIM) - the simulation of the entire production process and organization with fully integrated CAx-features - will boost complexity to an even higher but also inevitable level if manufacturers want to stay competitive [2]. Due to the enormous functional spectrum of CAx-software, we focused on only one part of the CAx process chain (Fig. 1): computer-aided manufacturing (CAM) and its software (CAM system).
Fig. 1.

CAx process chain

Since CAx/CAM-systems are too complex and demanding to be intuitively used, novel users usually receive extensive trainings. However, workplace practice shows that despite of this learning support most CAx/CAM novices are not able to successfully handle their software, which leads to longer training periods and frustration on the users’ side and, as a consequence, to an inefficient allocation of resources, reduced product quality, and delivery delays on the business side. Hence, the learnability of CAx/CAM software systems is an important, but rather neglected issue so far.

According to the HCI community, the learnability of a system is a central aspect of its usability [3]. Learnability refers to the initial learning experience of a user until he/she is able to successfully interact with a technical system [e.g., 4]. However, most of the HCI learnability research since the 1980ies focused on desktop computing [e.g., 5] or the usage of (mobile) “everyday technology” [e.g., 6], but little is known about the learnability of CAx/CAM software systems. Facing the demographic change and its implications, i.e., cognitively aging users or the retiring of experienced CAx/CAM experts who are succeeded by inexperienced users, not only general CAx/CAM learnability issues but also user-specific requirements in software skill acquisition become increasingly important.

1.1 Software Skill Acquisition in Older Users

Research concordantly shows, that older users face greater difficulties in interacting with software systems and in acquiring computer skills. Training takes significantly more time for older adults, they commit more errors in post-training evaluations compared to younger learners and considerable age differences remain in computer performance after receiving instructional support [e.g., 7]. These age-related differences in technical skill acquisition are explained by declines in sensory, motor and cognitive abilities, lower levels of technical experience and inadequate mental models [8]. Domain expertise was found to reduce or fully compensate age differences in performance and learning [e.g., 9]. Hence, apart from users’ age, the impact of user diversity factors such as CAM expertise should also be considered when investigating the learning conditions for a successful CAM-software skill acquisition. Although a lot of research was conducted on user interface interaction and technical skill acquisition in recent decades, most studies evaluated specific trainings formats (e.g., “training wheels” [10]) based on specific learning theory assumptions (e.g., “constructivism” [11], “active learning” [12]) or procedural instructional design schemes [13] without being connected to existing learning conditions or constraints in manufacturing practice. Therefore, the purpose of this study was to increase our knowledge about the learnability and learning conditions of CAM software systems in workplace reality. Our study focused on an evaluation of perceived learnability of CAM software systems, an assessment of formal learning environments and individual learning strategies, the availability of CAM support and CAM learning outcomes. In order to consider user diversity and user-specific demands, the factors age and expertise were integrated into the analysis to get insights into “tailored” optimal learning conditions for different CAM users.

2 Method

2.1 Questionnaire

The questionnaire started with a demographic section (age, gender, education, profession) and an assessment of CAM expertise (type of CAM system, usage experience in years, usage frequency, self-ratings of CAM system knowledge and problem-solving competency). The following items dealt with a learnability evaluation of the CAM system, an evaluation of formal and individual CAM learning environments and strategies, and an evaluation of the CAM support. Questionnaire items were answered on a six-level Likert-scale (totally disagree – totally agree).

2.2 The Sample

The study was run as online-questionnaire, which was distributed in several German manufacturing companies and in CAM-related online forums. A total of n = 119 participants volunteered to answer the questionnaire, but only n = 76 data sets were used for statistical analysis due to incomplete data.

Respondents’ age range was between 23 and 62 (M = 41.1, SD = 10.5), the majority (97.4 %) of participants was male. Asked for the level of education, 36 % held a university degree, 30 % completed an apprenticeship, 17 % had a secondary school degree, and 14 % a technical diploma. Half of the participants (49 %) were software developers, 24 % were technical draftsmen, and 21 % were toolmakers. CAM usage experience was between 2 months and 39 years (M = 8.1 years, SD = 6.9). Respondents frequently used their CAM-software (73 % several times a day, 21 % several times a week). The majority (43 %) worked with Siemens NX, followed by Tebis (8 %), CATIA, and SolidCam (both 7 %). Since the study focused on a learnability evaluation, no comparisons between different CAM-software solutions were made.

2.3 Data Analysis

Data was analyzed by MANOVAs (level of significance = 5 %). Due to the higher heterogeneity of older samples, marginally significant findings (level of significance = 10 %) are also reported. The Likert-scale range was transformed to −2.5 (totally disagree) to 2.5 (totally agree) with ratings <0 indicating negative evaluations and ratings >0 indicating positive evaluations. To analyze effects of age and expertise, the sample was divided into subgroups according to age (young = 23–34 years, middle = 35–45 years, old = 46–62 years) and expertise (novices and experts). To quantify expertise, an expertise score was calculated based on the multiplication of “CAM knowledge”- and “problem-solving competency”-ratings (M = 1.94, SD = 2.18, min = −6.25, max = 6.25). Two expertise groups were derived: novices with an expertise score <0 and experts with an score >3.75. Novices had M = 6.2 years (SD = 5.4) and experts had M = 9.6 years (SD = 7.8) of CAM usage experience. A longer duration of CAM usage was related to higher expertise levels (r = .24; p < 0.05). Age and expertise were not correlated (r = .19; p > .1), i.e., age and CAM expertise were independent from each other.

3 Results

3.1 CAM Learnability Evaluation

CAM users evaluated the general learnability of CAM software as not very high (M = 0.2, SD = 1.6). The evaluation of the duration of the individual CAM learning process and the achieved CAM skill level was more positive: CAM users perceived the trainings process duration as “rather short” (M = 0.6, SD = 1.5) and rated their individual CAM skill level as “rather good” (M = 1.3, SD = 1.1).

A 2 × 3 (expertise × age) MANOVA revealed, that learnability ratings significantly differed between CAM experts and novices (F(3,23) = 63.0, p < 0.000) and between CAM users of different age- and expertise groups (F(6,66) = 1.9; p < 0.1). Novices evaluated the CAM learnability negatively, perceived the learning period as longer and estimated their CAM skills to be lower than experts (Fig. 2).
Fig. 2.

CAM learnability ratings for experts and novices

A more detailed picture provided the differentiation of CAM learnability ratings in the age- and expertise-groups (Fig. 3). For younger CAM users, the learnability evaluations of novices and experts were similar and not very high. In the middle-aged group, differences in evaluations emerged with a more positive learnability evaluation of experts. In the group of older CAM users, the deviations even further increased: older CAM novices evaluated the learnability of CAM software negatively (M = −1.2), whereas older experts perceived a good learnability (M = 1.6) of their CAM software system.
Fig. 3.

CAM learnability ratings for the age- and expertise-groups

3.2 CAM Learning Environments and Strategies

3.2.1 Formal CAM Learning Environments

The preferred and best ranked environment for acquiring CAM knowledge was during university studies or in lectures (M = 1.29, SD = 2.4, Fig. 4).
Fig. 4.

Ranking of formal CAM learning environments

Further frequently used learning strategies were internal trainings (M = 1.9, SD = 1.9) or by “learning by doing” (M = 2, SD = 1.3), external trainings (M = 2.1, SD = 1.9), video-tutorials (e.g., “youtube”, M = 2.6, SD = 2.5), software-integrated tutorials (M = 3.0, SD = 2.6), advice by colleagues (M = 3.0, SD = 1.9), and, as learning strategy with the lowest rank, visiting online-forums (M = 3.3, SD = 2.9).

A 2 × 3 (expertise × age) MANOVA revealed marginally significant age differences in CAM learning environments rankings (F(16,58) = 1.8; p < 0.1, Fig. 5). Middle-aged CAM-users ranked “external trainings”, “forums”, and “advice by colleagues” significantly lower than younger and older CAM-users. In contrast, older users reported to use external trainings and online forums more frequently than the other age groups. Significant effects were neither found for other learning environment rankings nor for expertise or an interaction between both factors.
Fig. 5.

Age effects in CAM learning environment rankings

3.2.2 Preferred Formal CAM Learning Environment

Asked for the CAM leaning environment they would prefer in future (note the different answering scale – max/approval = 2.5 – min/rejection = −2.5), respondents favored integrated software tutorials (M = 0.7, SD = 1.4) and external trainings (M = 0.7, SD = 1.3), advice by colleagues (M = 0.7, SD = 1.5), followed by video-tutorials (M = 0.6, SD = 1.5) and internal trainings (M = 0.6, SD = 1.4, Fig. 6). Less preferred was “learning by doing” (M = 0.3, SD = 1.3). CAM learning by using online forums was rejected (M = −0.1, SD = 1.9) as well as university studies or lectures (M = −0.8, SD = 2.9), which was least preferred and showed a high variance in ratings.
Fig. 6.

Preferred formal CAM learning environment

The 3 × 2 (age × expertise) MANOVA yielded a significant age effect (F(16,16) = 2.3; p < 0.05). The group of older CAM-users evaluated internal trainings for future CAM knowledge acquisition rather neutrally (Mold = 0.1, SD = 1.4), whereas the two younger age groups rated internal trainings positively (Mmiddle = 0.9, SD = 1.3; Myoung = 0.8, SD = 1.3).

Although the main effect of expertise and the interaction of age and expertise missed statistical significance, the ratings of internal trainings in the different age × expertise groups can help to achieve a deeper understanding of the above reported age effect (Fig. 7). Especially the group of older CAM experts rejected internal trainings, whereas older novices and younger CAM users in general – independent from their CAM expertise – evaluated them positively as preferred future learning environment.
Fig. 7.

Preferences for internal CAM trainings for the different age- and expertise-groups

3.2.3 Individual CAM Learning Strategies

Apart from an evaluation of formal CAM learning environments, respondents assessed different individual CAM learning strategies. Long-term usage was perceived as most effective learning strategy (M = 1.9, SD = 0.9), followed explorative learning (M = 1.0, SD = 1.3), by advice from colleagues (M = 0.4, SD = 1.4) and interacting with the CAM software in a “trial-and-error”-modus (M = 0.1, SD = 1.3). Transferring knowledge from other programs received the lowest ratings and was not perceived as effective individual learning strategy (M = −0.1, SD = 1.6).

Novices and experts significantly differed in their preferred learning strategies (F(5,28) = 6.8; p < 0.000, Fig. 6), as well as the different age × expertise groups for the strategy “explorative learning” (F(10,58) = 1.8; p < 0.1, see Fig. 8). Even though both expertise groups gave positive ratings, experts estimated the importance of long-term usage, trial and error and explorative learning higher than novices. Advice from colleagues was seen as equally important by novices and experts. However, the perception of knowledge transfer differed: meanwhile experts approved “knowledge transfer from other programs”, novices rejected this as effective learning strategy.
Fig. 8.

Evaluation of individual CAM learning strategies for experts and novices

Further insights were derived from the evaluation of explorative learning as effective individual CAM learning strategy in the different age- and expertise-groups (Fig. 9).
Fig. 9.

Explorative learning ratings for the different age- and expertise-groups

Experts of different age groups positively rated “explorative learning” as effective learning strategy. This positive perception was also present in younger novices, but explorative learning was seen as less effective learning strategy – and even rejected – with increasing age by novices.

3.3 Evaluation of CAM Support

The evaluation of the CAM support in the company was comparably positive (Fig. 10). Respondents showed a high willingness to answer colleagues’ CAM questions and also felt confident to ask colleagues for CAM advice vice versa. The risk of loosing the main contact for CAM-related questions in future due to fluctuation or retirement was perceived to be low. The availability of several contacts for CAM support was approved to a lesser degree, as well as a fast response time until the CAM support answers.
Fig. 10.

Evaluation of CAM support

Respondents’ estimations of savings in working time per week if an improved CAM support was available revealed an enormous potential for efficiency improvement (Fig. 11). CAM users estimated to save on average M = 2.5 h per week (SD = 4.8) if a competent CAM support for problems and questions was provided. The time saving estimations differed significantly between experts and novices (F(1,19) = 5.8; p < 0.5). Novices estimated to save Mnovices = 5.3 h (SD = 8.2) with a competent CAM support in the background, while experts expected considerably lower time savings (Mexperts = 0.7, SD = 1.0). Again, it was highly informative to differentiate between age- and expertise-groups. In the group of young CAM users, the time saving estimations of novices and experts did not deviate much (Myoung novices = 2.0 vs. Myoung experts = 1.3). In the middle-aged group, novices reported to have a higher benefit of a competent CAM support (Mmiddle-aged novices = 3.2 vs. Mmiddle-aged experts = 0.7). This gap enormously increased in the older CAM user group. Meanwhile older experts reported not to benefit at all from a competent CAM support (Molder novices = 0.0), the group of older novices expected the highest time savings (Molder experts = 12.7).
Fig. 11.

Time savings estimations in hours/week (when having a competent CAM support) for the different age- and expertise-groups.

4 Discussion

Computer-aided software systems are a central component of modern manufacturing processes. Due to their high functional complexity Cax/CAM-software skill acquisition is often a long, demanding and even frustrating process for users. To gain knowledge about the learnability and learning conditions of CAx/CAM software systems in workplace reality, the present study aimed for an analysis of the current state of perceived learnability of CAM software systems, an assessment of learning environments and individual learning strategies on the workplace, the availability of CAM support and CAM learning outcomes by running an online-survey in German manufacturing companies. To account for effects of the demographic change on the workforce, user diversity factors such as age and CAM expertise where included into the analysis.

Learnability of CAM Software. At first sight, the learnability and learning conditions of CAM software systems in manufacturing companies receive positive evaluations. However, this finding needs to be revised, when a more detailed, user-group-specific perspective is taken. A high learnability evaluation of CAM software only applies for older and experienced CAM practitioners. We assume that the positive expert learnability evaluation of CAM interfaces is affected by a “hindsight bias”, where the invested efforts in the personal CAM learning history are retrospectively blinded out. In contrast, the CAM learnability evaluation by unexperienced and younger CAM users is not satisfying at all, since these groups formulate an enormous need for an improved CAM learning support. Inefficient working procedures are the economic consequence of the current state of CAM system learnability, as the time saving estimations (up to 12 working hours a week) if a competent support was provided, demonstrate.

Effects of User Factors on CAM Learnability. The CAM learnability analysis revealed, that one user group specifically needs improved CAM learning support: older novices. The older and less experienced CAM users are, the less they benefit from exploratory interaction experiences and the less he/she is able to transfer knowledge from other programs. This outcome fits well into the research body of age- and expertise research. Novices do not possess highly organized domain-specific knowledge structures. Hence, while learning to use CAM software or solving CAM interaction problems, novices cannot draw upon extensive domain-specific knowledge structures, which often leads to a superficial perception of problems and less flexible problem solutions [14]. Moreover, age effects in information processing abilities also contribute to problems in CAM skill acquisition. This especially refers to age-related declines in spatial abilities, processing speed, reasoning and memory abilities, which were identified as relevant cognitive abilities for a successful interaction with technical devices and the acquisition of computer skills [15, 16]. But not only cognitive or expertise-related factors should be considered in developing CAM learning support strategies, emotional and motivational factors also play a role. Due to a lower self-efficacy, the learning confidence in older learners is lower [17], which stresses the role of facilitating learning conditions during initial use [18].

Recommendations for CAM Learning Environments and Strategies. Future CAM learning environments should take the diversity of their users into account. For older and novice CAM learners a more structured and knowledge-structure-supporting learning support is recommended. Not only formal learning (in trainings), but also informal workplace learning should be strengthened [19], which stresses the role of older experts as “CAM mentors” in the CAM learning process. Considering the fact, that (older) CAM experts might retire or leave the company, the integration of social media into CAM learning environments is a promising way for 1:n-knowledge exchange and CAM learning support [20]. On the other side, when looking at expert users, HR practitioners should critically question their reliance on formal internal CAM-trainings, since especially older CAM experts reject this formal learning environment.

Limitations and Future Research. Future studies on this topic should aim for a larger sample size, even though it has to be noted, that the response rate in such specific user groups is usually low. The analysis of CAM system learnability should be widened, e.g. by including DIN evaluation criteria [21]. Since learnability and usability are closely connected, future studies should investigate usability improvements of CAM software interfaces and their effect on learnability outcomes. Finally, trainings formats, specifically designed for the needs of older and CAM-unexperienced users should be designed and evaluated.

Notes

Acknowledgments

We owe gratitude to Conrad Schnöckel and Sarah Völkel for research support.

References

  1. 1.
    Dankwort, C.W., Weidlich, R., Guenther, B., Blaurock, J.E.: Engineers’ CAx education—it’s not only CAD. Comput. Aided Des. 36, 1439–1450 (2004)CrossRefGoogle Scholar
  2. 2.
    Hunt, V.D.: Computer-Integrated Manufacturing Handbook. Springer Science & Business Media, Berlin (2012)Google Scholar
  3. 3.
    Nielsen, J.: Usability Engineering. Elsevier, Philadelphia (1994)zbMATHGoogle Scholar
  4. 4.
    Shneiderman, B., Plaisant, C.: Designing the User Interface: Strategies for Effective Human-Computer Interaction. Addison-Wesley, Boston (2010)Google Scholar
  5. 5.
    Carroll, J.M., Mack, R.L., Lewis, C.H., Grischkowsky, N.L., Robertson, S.R.: Exploring exploring a word processor. Hum. Comput. Interact. 1, 283–307 (1985)CrossRefGoogle Scholar
  6. 6.
    Ziefle, M.: The influence of user expertise and phone complexity on performance, ease of use and learnability of different mobile phones. Behav. Inf. Technol. 21, 303–311 (2002)CrossRefGoogle Scholar
  7. 7.
    Kelley, C., Charness, N.: Issues in training older adults to use computers. Behav. Inf. Technol. 14, 107–120 (1995)CrossRefGoogle Scholar
  8. 8.
    Hawthorn, D.: Possible implications of aging for interface designers. Interact. Comput. 12, 507–528 (2000)CrossRefGoogle Scholar
  9. 9.
    Nunes, A., Kramer, A.F.: Experience-based mitigation of age-related performance declines: evidence from air traffic control. J. Exp. Psychol. Appl. 15, 12–24 (2009)CrossRefGoogle Scholar
  10. 10.
    Carroll, J.M., Carrithers, C.: Training wheels in a user interface. Commun. ACM 27, 800–806 (1984)CrossRefGoogle Scholar
  11. 11.
    Gold, S.: A constructivist approach to online training for online teachers. J. Asynchronous Learn. Netw. 5, 35–57 (2001)Google Scholar
  12. 12.
    Arning, K., Ziefle, M.: Ask and you will receive: training novice adults to use a PDA in an active learning environment. Int. J. Mob. Hum. Comput. Interact. 2, 21–47 (2010)CrossRefGoogle Scholar
  13. 13.
    Mayhorn, C.B., Stronge, A.J., McLaughlin, A.C., Rogers, W.A.: Older adults, computer training, and the systems approach: a formula for success. Educ. Gerontol. 30, 185–203 (2004)CrossRefGoogle Scholar
  14. 14.
    Glaser, R., Chi, M.T., Farr, M.J.: The Nature of Expertise. Lawrence Erlbaum Associates, Abingdon (1988)zbMATHGoogle Scholar
  15. 15.
    Czaja, S.J., Sharit, J.: Age differences in the performance of computer-based work. Psychol. Aging 8, 59–67 (1993)CrossRefGoogle Scholar
  16. 16.
    Arning, K., Ziefle, M.: Effects of age, cognitive, and personal factors on PDA menu navigation performance. Behav. Inf. Technol. 28, 251–268 (2009)CrossRefGoogle Scholar
  17. 17.
    Maurer, T.J.: Career-relevant learning and development, worker age, and beliefs about self-efficacy for development. J. Manag. 27, 123–140 (2001)Google Scholar
  18. 18.
    Barnard, Y., Bradley, M.D., Hodgson, F., Lloyd, A.D.: Learning to use new technologies by older adults: perceived difficulties, experimentation behaviour and usability. Comput. Hum. Behav. 29, 1715–1724 (2013)CrossRefGoogle Scholar
  19. 19.
    Ellström, P.-E.: Informal learning at work: Conditions, processes and logics. In: Malloch, M., Cairns, L., Evans, K., O’Connor, B.N. (eds.) The SAGE Handbook Workplace Learning, pp. 105–119. SAGE Publication, London (2011)CrossRefGoogle Scholar
  20. 20.
    García-Peñalvo, F.J., Colomo-Palacios, R., Lytras, M.D.: Informal learning in work environments: training with the social web in the workplace. Behav. Inf. Technol. 31, 753–755 (2012)CrossRefGoogle Scholar
  21. 21.
    IEC, I.: 9126-1: Software Engineering–Qualität von Softwareprodukten-Teil 1: Qualitätsmodell. Int. Stand. Organ. Berl. Beuth Verl. (2001)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Human-Computer-Interaction CenterRWTH Aachen UniversityAachenGermany

Personalised recommendations