You Can(’t) Teach an Old Dog New Tricks: Analyzing the Learnability of Manufacturing Software Systems in Older Users
- 1.6k Downloads
Modern manufacturing processes are based on complex computer-aided planning processes, which are provided by CAM (computer-aided manufacturing)-software systems. Due to increased functional capabilities of CAM software, the complexity of these systems and the demands on CAM users are rising. Facing the demographic change (cognitively aging users, retiring of experienced CAM experts who are succeeded by inexperienced users), not only general learnability issues but also user-specific requirements are becoming increasingly important. An online-survey focusing on the learnability of CAM-software, and existing learning environments and strategies in manufacturing practice was conducted (n = 76) and effects of age and CAM expertise were analyzed. Implications for CAM skill acquisition among users of different age and expertise groups were derived.
KeywordsComputer-aided manufacturing software User diversity Age Expertise Learnability Training Survey
Since CAx/CAM-systems are too complex and demanding to be intuitively used, novel users usually receive extensive trainings. However, workplace practice shows that despite of this learning support most CAx/CAM novices are not able to successfully handle their software, which leads to longer training periods and frustration on the users’ side and, as a consequence, to an inefficient allocation of resources, reduced product quality, and delivery delays on the business side. Hence, the learnability of CAx/CAM software systems is an important, but rather neglected issue so far.
According to the HCI community, the learnability of a system is a central aspect of its usability . Learnability refers to the initial learning experience of a user until he/she is able to successfully interact with a technical system [e.g., 4]. However, most of the HCI learnability research since the 1980ies focused on desktop computing [e.g., 5] or the usage of (mobile) “everyday technology” [e.g., 6], but little is known about the learnability of CAx/CAM software systems. Facing the demographic change and its implications, i.e., cognitively aging users or the retiring of experienced CAx/CAM experts who are succeeded by inexperienced users, not only general CAx/CAM learnability issues but also user-specific requirements in software skill acquisition become increasingly important.
1.1 Software Skill Acquisition in Older Users
Research concordantly shows, that older users face greater difficulties in interacting with software systems and in acquiring computer skills. Training takes significantly more time for older adults, they commit more errors in post-training evaluations compared to younger learners and considerable age differences remain in computer performance after receiving instructional support [e.g., 7]. These age-related differences in technical skill acquisition are explained by declines in sensory, motor and cognitive abilities, lower levels of technical experience and inadequate mental models . Domain expertise was found to reduce or fully compensate age differences in performance and learning [e.g., 9]. Hence, apart from users’ age, the impact of user diversity factors such as CAM expertise should also be considered when investigating the learning conditions for a successful CAM-software skill acquisition. Although a lot of research was conducted on user interface interaction and technical skill acquisition in recent decades, most studies evaluated specific trainings formats (e.g., “training wheels” ) based on specific learning theory assumptions (e.g., “constructivism” , “active learning” ) or procedural instructional design schemes  without being connected to existing learning conditions or constraints in manufacturing practice. Therefore, the purpose of this study was to increase our knowledge about the learnability and learning conditions of CAM software systems in workplace reality. Our study focused on an evaluation of perceived learnability of CAM software systems, an assessment of formal learning environments and individual learning strategies, the availability of CAM support and CAM learning outcomes. In order to consider user diversity and user-specific demands, the factors age and expertise were integrated into the analysis to get insights into “tailored” optimal learning conditions for different CAM users.
The questionnaire started with a demographic section (age, gender, education, profession) and an assessment of CAM expertise (type of CAM system, usage experience in years, usage frequency, self-ratings of CAM system knowledge and problem-solving competency). The following items dealt with a learnability evaluation of the CAM system, an evaluation of formal and individual CAM learning environments and strategies, and an evaluation of the CAM support. Questionnaire items were answered on a six-level Likert-scale (totally disagree – totally agree).
2.2 The Sample
The study was run as online-questionnaire, which was distributed in several German manufacturing companies and in CAM-related online forums. A total of n = 119 participants volunteered to answer the questionnaire, but only n = 76 data sets were used for statistical analysis due to incomplete data.
Respondents’ age range was between 23 and 62 (M = 41.1, SD = 10.5), the majority (97.4 %) of participants was male. Asked for the level of education, 36 % held a university degree, 30 % completed an apprenticeship, 17 % had a secondary school degree, and 14 % a technical diploma. Half of the participants (49 %) were software developers, 24 % were technical draftsmen, and 21 % were toolmakers. CAM usage experience was between 2 months and 39 years (M = 8.1 years, SD = 6.9). Respondents frequently used their CAM-software (73 % several times a day, 21 % several times a week). The majority (43 %) worked with Siemens NX, followed by Tebis (8 %), CATIA, and SolidCam (both 7 %). Since the study focused on a learnability evaluation, no comparisons between different CAM-software solutions were made.
2.3 Data Analysis
Data was analyzed by MANOVAs (level of significance = 5 %). Due to the higher heterogeneity of older samples, marginally significant findings (level of significance = 10 %) are also reported. The Likert-scale range was transformed to −2.5 (totally disagree) to 2.5 (totally agree) with ratings <0 indicating negative evaluations and ratings >0 indicating positive evaluations. To analyze effects of age and expertise, the sample was divided into subgroups according to age (young = 23–34 years, middle = 35–45 years, old = 46–62 years) and expertise (novices and experts). To quantify expertise, an expertise score was calculated based on the multiplication of “CAM knowledge”- and “problem-solving competency”-ratings (M = 1.94, SD = 2.18, min = −6.25, max = 6.25). Two expertise groups were derived: novices with an expertise score <0 and experts with an score >3.75. Novices had M = 6.2 years (SD = 5.4) and experts had M = 9.6 years (SD = 7.8) of CAM usage experience. A longer duration of CAM usage was related to higher expertise levels (r = .24; p < 0.05). Age and expertise were not correlated (r = .19; p > .1), i.e., age and CAM expertise were independent from each other.
3.1 CAM Learnability Evaluation
CAM users evaluated the general learnability of CAM software as not very high (M = 0.2, SD = 1.6). The evaluation of the duration of the individual CAM learning process and the achieved CAM skill level was more positive: CAM users perceived the trainings process duration as “rather short” (M = 0.6, SD = 1.5) and rated their individual CAM skill level as “rather good” (M = 1.3, SD = 1.1).
3.2 CAM Learning Environments and Strategies
3.2.1 Formal CAM Learning Environments
Further frequently used learning strategies were internal trainings (M = 1.9, SD = 1.9) or by “learning by doing” (M = 2, SD = 1.3), external trainings (M = 2.1, SD = 1.9), video-tutorials (e.g., “youtube”, M = 2.6, SD = 2.5), software-integrated tutorials (M = 3.0, SD = 2.6), advice by colleagues (M = 3.0, SD = 1.9), and, as learning strategy with the lowest rank, visiting online-forums (M = 3.3, SD = 2.9).
3.2.2 Preferred Formal CAM Learning Environment
The 3 × 2 (age × expertise) MANOVA yielded a significant age effect (F(16,16) = 2.3; p < 0.05). The group of older CAM-users evaluated internal trainings for future CAM knowledge acquisition rather neutrally (Mold = 0.1, SD = 1.4), whereas the two younger age groups rated internal trainings positively (Mmiddle = 0.9, SD = 1.3; Myoung = 0.8, SD = 1.3).
3.2.3 Individual CAM Learning Strategies
Apart from an evaluation of formal CAM learning environments, respondents assessed different individual CAM learning strategies. Long-term usage was perceived as most effective learning strategy (M = 1.9, SD = 0.9), followed explorative learning (M = 1.0, SD = 1.3), by advice from colleagues (M = 0.4, SD = 1.4) and interacting with the CAM software in a “trial-and-error”-modus (M = 0.1, SD = 1.3). Transferring knowledge from other programs received the lowest ratings and was not perceived as effective individual learning strategy (M = −0.1, SD = 1.6).
Experts of different age groups positively rated “explorative learning” as effective learning strategy. This positive perception was also present in younger novices, but explorative learning was seen as less effective learning strategy – and even rejected – with increasing age by novices.
3.3 Evaluation of CAM Support
Computer-aided software systems are a central component of modern manufacturing processes. Due to their high functional complexity Cax/CAM-software skill acquisition is often a long, demanding and even frustrating process for users. To gain knowledge about the learnability and learning conditions of CAx/CAM software systems in workplace reality, the present study aimed for an analysis of the current state of perceived learnability of CAM software systems, an assessment of learning environments and individual learning strategies on the workplace, the availability of CAM support and CAM learning outcomes by running an online-survey in German manufacturing companies. To account for effects of the demographic change on the workforce, user diversity factors such as age and CAM expertise where included into the analysis.
Learnability of CAM Software. At first sight, the learnability and learning conditions of CAM software systems in manufacturing companies receive positive evaluations. However, this finding needs to be revised, when a more detailed, user-group-specific perspective is taken. A high learnability evaluation of CAM software only applies for older and experienced CAM practitioners. We assume that the positive expert learnability evaluation of CAM interfaces is affected by a “hindsight bias”, where the invested efforts in the personal CAM learning history are retrospectively blinded out. In contrast, the CAM learnability evaluation by unexperienced and younger CAM users is not satisfying at all, since these groups formulate an enormous need for an improved CAM learning support. Inefficient working procedures are the economic consequence of the current state of CAM system learnability, as the time saving estimations (up to 12 working hours a week) if a competent support was provided, demonstrate.
Effects of User Factors on CAM Learnability. The CAM learnability analysis revealed, that one user group specifically needs improved CAM learning support: older novices. The older and less experienced CAM users are, the less they benefit from exploratory interaction experiences and the less he/she is able to transfer knowledge from other programs. This outcome fits well into the research body of age- and expertise research. Novices do not possess highly organized domain-specific knowledge structures. Hence, while learning to use CAM software or solving CAM interaction problems, novices cannot draw upon extensive domain-specific knowledge structures, which often leads to a superficial perception of problems and less flexible problem solutions . Moreover, age effects in information processing abilities also contribute to problems in CAM skill acquisition. This especially refers to age-related declines in spatial abilities, processing speed, reasoning and memory abilities, which were identified as relevant cognitive abilities for a successful interaction with technical devices and the acquisition of computer skills [15, 16]. But not only cognitive or expertise-related factors should be considered in developing CAM learning support strategies, emotional and motivational factors also play a role. Due to a lower self-efficacy, the learning confidence in older learners is lower , which stresses the role of facilitating learning conditions during initial use .
Recommendations for CAM Learning Environments and Strategies. Future CAM learning environments should take the diversity of their users into account. For older and novice CAM learners a more structured and knowledge-structure-supporting learning support is recommended. Not only formal learning (in trainings), but also informal workplace learning should be strengthened , which stresses the role of older experts as “CAM mentors” in the CAM learning process. Considering the fact, that (older) CAM experts might retire or leave the company, the integration of social media into CAM learning environments is a promising way for 1:n-knowledge exchange and CAM learning support . On the other side, when looking at expert users, HR practitioners should critically question their reliance on formal internal CAM-trainings, since especially older CAM experts reject this formal learning environment.
Limitations and Future Research. Future studies on this topic should aim for a larger sample size, even though it has to be noted, that the response rate in such specific user groups is usually low. The analysis of CAM system learnability should be widened, e.g. by including DIN evaluation criteria . Since learnability and usability are closely connected, future studies should investigate usability improvements of CAM software interfaces and their effect on learnability outcomes. Finally, trainings formats, specifically designed for the needs of older and CAM-unexperienced users should be designed and evaluated.
We owe gratitude to Conrad Schnöckel and Sarah Völkel for research support.
- 2.Hunt, V.D.: Computer-Integrated Manufacturing Handbook. Springer Science & Business Media, Berlin (2012)Google Scholar
- 4.Shneiderman, B., Plaisant, C.: Designing the User Interface: Strategies for Effective Human-Computer Interaction. Addison-Wesley, Boston (2010)Google Scholar
- 11.Gold, S.: A constructivist approach to online training for online teachers. J. Asynchronous Learn. Netw. 5, 35–57 (2001)Google Scholar
- 17.Maurer, T.J.: Career-relevant learning and development, worker age, and beliefs about self-efficacy for development. J. Manag. 27, 123–140 (2001)Google Scholar
- 21.IEC, I.: 9126-1: Software Engineering–Qualität von Softwareprodukten-Teil 1: Qualitätsmodell. Int. Stand. Organ. Berl. Beuth Verl. (2001)Google Scholar