A transformation of human operation approach to inform system design for automation

Design of automation system relies on experts’ knowledge and experience accumulated from past solutions. In designing novel solutions, however, it is difficult to apply past knowledge and achieve design right-first-time, therefore wasting valuable resources and time. SADT/IDEF0 models are commonly used by automation experts to model manufacturing systems based on the manual process. However, function generalisation without benchmarking is difficult for experts particularly for complex and highly skilled-based tasks. This paper proposes a functional task abstraction approach to support automation design specification based on human factor attributes. A semi-automated clustering approach is developed to identify key functions from an observed manual process. The proposed approach is tested on five different automation case studies. The results indicate the proposed method reduces inconsistency in task abstraction when compared to the current approach that relies on the experts, which are further validated against the solutions generated by automation experts.


Introduction
The current manufacturing sector is experiencing transformational changes.New paradigms like Industry 4.0, Robotic and Autonomous Systems (RAS) and a decentralisation of decision-making driven by implementation of agent-based intelligent systems have the potential to significantly enhance the capabilities of manufacturing (Foresight 2013;Zhong et al. 2017).Adapting to technical challenges and skill shortages is of crucial importance for modern manufacturing businesses to stay competitive in a globalised market (Sungur et al. 2016).A potential solution is through increasing the use of sensors to enhance flexibility and intelligence of automation solutions in applications beyond repetitive, dangerous and traditional automation tasks, and decrease in the total cost of ownership.However, more efficient ways to support the implementation of automation into manufacturing businesses are required in the context of Industry 4.0 (Yao et al. 2019;Kong et al. 2019).
B Yee Mey Goh y.goh@lboro.ac.uk 1 Intelligent Automation Centre, Wolfson School of Mechanical, Electrical and Manufacturing Engineering, Loughborough University, Loughborough, Leicestershire LE11 3TU, UK Many industries, particularly in the UK, have not embraced automation due to the uncertainty and difficulty to design solutions that are flexible and meet the current productivity of manual processes.Root causes for feasibility related problems dealing with the automation of human tasks have been recognised (Bainbridge 1983;Reason 1987;Xiao et al. 1997), in particular related to the requirement for deep knowledge of the tasks for automation.(Goodrich and Boer 2003) stated "a lesson learned from process automation is that, in the absence of human factors consideration, even state-ofthe-art technological systems can be more problematic than beneficial".Part of the challenges is describing human tasks in a useful way as the basis for implementing automation (Everitt et al. 2015).
The investigation of human factors is the subject of extensive research in complementary areas.Examples are the investigation of trust-through-transparency impact on performance (Oduor and Wiebe 2008) or the impact of system complexity on trust (Bailey and Scerbo 2007).Even though the analysis of human tasks for automation has been investigated, for example Caird-Daley et al. (2013), no reliable way to automatically abstract human tasks has been identified when decision-makers evaluate automation business cases.
In this paper, a clustering-based approach is proposed to identify automation functions from a human task analysis 123 Process Modelling for Automation Determination and Assessment Human Task Analysis

InformaƟon flow
Fig. 1 Stages in automating a manual manufacturing process enabling a more comprehensive and consistent specification of automation design for business case evaluation.The underlying assumption is that the automated solution is not necessarily like the manual task due to different skills and capabilities of humans and robots (de Winter and Dodou 2014).Nevertheless, the ability to identify the functions provided by manual processes will enhance and widen the solution search space in the early design phase prior to technology selection.
The paper starts with a literature review ("Literature review" section) to identify the research gap and the methodology ("Proposed method" section) describes how a functional abstraction of human tasks is developed and introduces the proposed method before case studies are used to evaluate and compare the results to industrial experts' solutions ("Results" section).Finally, the last section discusses and concludes the paper reflecting on the contributions to the research topic ("Discussion and conclusions" section).

Literature review
Research in manufacturing automation has highlighted the need to consider human factors (Goodrich and Boer 2003).The general process of automating a manufacturing process consists of the creation of a process representation model derived from a manual task to design and create a technical solution.Figure 1 summarises the current methods and tools to support the automation design from the human task analysis to the evaluation and assessment.In this paper, the authors assume an automation process would generically follow the demonstrated stages: 1. Understanding the human task 2. Formalising the process representation

Synthesising the process representation model for automation 4. Evaluating the proposed automation solution
Starting with the collection of information about the human task via a human task analysis (HTA), process representation models are used to formalise the production process.The review of literature suggests there is a disconnect due to a weak link between HTA performed by human factors experts and process representation models typically constructed by engineers.Furthermore, the transition of a task analysis into the process representation model is exposed to the expert's interpretation.A solution is then created by synthesising the process model to specify the appropriate automation solution, relying on the accuracy and completeness of the formalised representation.
Each of the steps in this process, as well as the overall process, may be conducted iteratively until a desired level of usability is attained.However, iterations are expensive and time consuming.The first three stages of this process are reviewed for the methods and tools used to date.The final stage is an evaluation of the designed solution against the business case, considering economic and other factors before the solution is commissioned.For brevity, the readers are referred to other publications (Ketipi et al. 2014;Koulouriotis and Ketipi 2014) for more detail.

Human task analysis
Human factors has been studied widely in the manufacturing domain but their focus is wide-ranging and not necessarily on automation.For automation, the literature ranges from applications of artificial intelligence to automatically transfer human skills via demonstration to automation systems  (Chuck et al. 2017) and automation component mapping, for example a humanoid task-component mapping (Hanai et al. 2016).Other studies contribute towards the mental assessment and strains on humans combined with related decision-making for automation (Caird-Daley et al. 2013).
In this paper, the focus will be on a comprehensive task analysis and decomposition informing later processes of automation.The current state-of-the-art covers task decomposition (Phipps et al. 2011) from a physical and mental perspective.Other literature discusses the analysis of human tasks illustrating the importance of learning via demonstration, the hierarchical task analysis (HTA), the Sub-Goal Template (SGT), the Conceptual Task Analysis (CTA), and the work-process and Program for the International Assessment of Adult Competencies (PIAAC) approach.Table 1 summarises the efficiency and effectiveness for a selection of commonly referred methods to date.
The automation research community recognised the issue related to the abstraction of tasks without losing substantial information.Some researchers have investigated the HTA and cognitive work analysis to produce a comprehensive picture of manufacturing task analysis and present a variety of different applications (Stanton 2006;Salmon et al. 2010).The first approach by Phipps et al. (2011) extended the HTA by adding cognitive elements of tasks and information design requirements adding significant detail to the current knowledge of manufacturing task analysis.Caird-Daley et al. (2013) executed a task decomposition based on an HTA to capture physical and cognitive tasks to include the physical analysis for automation.Fasth-Berglund and Stahre (2013) confirmed the need to consider cognitive as well as a physical task as part of the automation strategies for reconfigurable and sustainable systems.Based on an HTA analysis, Everitt and Fletcher (2016) tackled the goal of a "robust, formal skill capture for assessing the feasibility and implementation of intelligent automation".Their dual methodology approach combines the existing HTA methodology with a classification system aimed to further increase the understanding of what an automated solution might look like.They extended the analysis with human perception senses, and a specific task classification, as well as a description of the decisions.
The research literature demonstrates the transfer of human tasks into automated tasks is essential.Yet, a transfer has not been achieved without substantial effort for the user in combination with a limited area of applications requiring detailed domain knowledge (Wong and Seet 2017;Pedersen et al. 2016;Wantia et al. 2016).In short, Zhao et al. (2015) pointed out a knowledge gap, where the transition from an extended HTA process towards automation system design is still missing.

Process representation method
A model is used to systematically represent the inner relations and functions of a system in an abstract way, according to a certain perspective, and to reduce the complexity.A key aspect of production research is the development of process representation models.Numerous standard processrepresentation models have been developed.Existing process representation models in manufacturing are divided into four different categories: production layout, production information, production schedule, and production optimisation: • Production layout represents a category for process representation tools capturing the setup of the production system.Models are used to describe the dimension of the production system, as well as skills and capabilities and the components of the production system.• Production information refers to the related models used to provide information about the manufacturing system.These models often display requirements of the production process.The process models can inform various aspects such as the representation of knowledge, dependencies between tasks, or the workflow and value stream within the production system.• Production scheduling describes a category of representations for modelling the schedule of a production system.
The representations are used to provide information about the time structure of a production process, the overlap within the production, as well as the transition from one production moment to another.The results can later be fed into optimisation tools or inform the design of the production system.• Production optimisation methods are used to improve the current situation by using a model that describes a production reality.An improvement of the situation can be achieved by predicting a future outcome, identifying bottlenecks, or optimising the service at production stations.
The current landscape of existing production representation models is analysed, and the outcomes are summarised in Table 2.The third column is provided to indicate whether the models will require the input of a task analysis, which was described in "Human task analysis" section.
The results presented in Table 2 indicate a lack of connection between the manual task analysis and the process representation models.As evident from the table, many of these models do not require a task analysis to be completed, which suggests that the process models are usually made by the opinion/understanding of engineers and not necessarily based on observation of how the tasks are being performed by the operators.This is important in automation because any decisions/variations performed by the operators that is not captured in the process models will be missed when specifying automation solution.Therefore, for a successful translation of a human task into an automation solution, this gap needs to be overcome.

Automation design specification
Four common ways are generally used to specify automation systems, namely requirements engineering, database approaches, technology selection, and generic design models.Requirements engineering is the development of an automation system based on anticipated requirements from the previous manual production process (Kaindl et al. 2009).The basis of this production process is an observation of requirements.The requirements can be recorded or extracted differently like, for example, via demonstration or requirements analysis as well as a via a combination with process representation models (Park et al. 2009).Examples for requirements engineering can be found throughout the literature related to design and architecture for holistic solutions (Hegenberg et al. 2012), functional requirements for reconfigurability and flexibility (Boschi et al. 2016), vision systems (Sitte and Winzer 2007), or software testing (Chung and Hwang 2007).Database approaches have been used to improve the design of automation solutions driven by CAD and composite structural data (Mayer et al. 1992;Hunten et al. 2013;Sanders et al. 2016).The overall use of multi-  Other generic design methods that can be used for the system design can be found in the tools Quality Function Deployment (QFD), Failure Mode and Effects Analysis (FMEA) and Activity Based Costing (ABC) analysis to select different processes for manufacturing based on cost, quality and risk aspects (Hassan et al. 2010).These methods require robust input information from the process requirements.

Summary
Although significant progress has been made regarding the analysis of a production task, the issue of transferring and mapping of human tasks against automation has not yet been tackled sufficiently.Several authors have contributed to the decomposition of human tasks and adding detail to the HTA.However, the additional detail were reliant on the experts leading to a higher variance and reduced reproducibility (Sheperd 2005).The problem is a reliable and systematic transfer of the qualitative human factors into requirements engineering.One could envisage the automation of the task analysis, using technology such as tracking and activity recognition (Rude et al. 2018), could be performed in the future.Therefore, the identified research gap in manufacturing literature is to provide an approach to bridge the gap between task analysis and the automation system design based on the identified task functions.The next section presents the methodology adopted in this paper to address the current knowledge gap.

Proposed method
The methodology presented in Fig. 2 is used to develop the approach to bridge the gap between the manual task analysis and automation system design.The authors propose a semi-automated approach using a clustering algorithm to classify and map the manufacturing tasks against automation functions.More specifically, after the analysis of human tasks, the paper presents a classification scheme allowing task breakdown from a process to be mapped against automation functions.Those functions can later be linked to automation components.

Hierarchical task analysis (HTA)
For each case study, multiple methods currently used for the decomposition of human tasks are combined.The initial input to the proposed approach is a hierarchical task analysis (HTA) as presented in Table 3.The table shows a fraction of the task analysis for the welding case study (the full operational structure is shown in Table 5).The task is decomposed into operations performed during a manufacturing process.At the same time, the operations are sorted with respect to time in a chronological manner starting with the first task (only operations are used in the clustering process in "Clustering" section).
The data structure established of the operation analysis is shown in Fig. 3, based on a defined decomposition structure.The hierarchical task structures of five case studies presented in "Results" section are used initially and extended to include the different SGT elements based on Ormerod et al. (1998).Every operation is labelled with a name and a specific sequential ID.The operation contains not only physical actions but Fig. 3 Extended HTA data structure for operation analysis also cognitive actions performed during a manufacturing process.The cognitive activity is related to the object or the tool, whereas the physical activity is related to specific multiple body parts.
After the HTA is performed on the process, the clustering algorithm is applied to identify the task functions.

Clustering
The application of clustering is to create task functions based on a HTA.The necessary requirement to enable clustering is a database to represent specific operation attributes of a specific action.Those attributes should be used to differentiate among dissimilar task functions.Before the clustering algorithm can be applied, however, a decision had to be made on the granularity as well as the factors of interest for the application.In this case, the clustering algorithm is abstracting manufacturing functions to define automation solutions.Therefore, a classification scheme is developed based on existing standards and the existing literature to attribute the process operations.

Classification scheme
The first task in developing a classification scheme is to identify existing classifications to enable a structured separation of manufacturing operations.At this point, different attributes according to the current literature of human factors are feasible for the classification process.For the application of a classification in this paper, the author has selected manufacturing attributes to create a functional task abstraction.
A careful analysis of the environment in this case has led the authors to the DIN8580 standard, which is (numerically) followed by other more specific standards.The application categories represent sub-levels of the manufacturing main categories joining, forming, etc. presented in DIN 8580.It is noted that the presented manufacturing classification considers physical manufacturing operations only.Supporting operations related to the perception mechanisms (visual perception, haptic feedback) are not covered in the related classification.
Hence, the authors have extended the existing manufacturing classification standards with perception mechanisms.A combination of research by Groover (2007) with Lederman and Klatzky (1987)

Clustering for automated function identification
The manufacturing classification against the operation as many-to-many relationship is expressed by binary variables.One operation can have multiple attributes and one attribute can be connected to multiple operations.Consequently, a database is created.The user will identify the attributes for every operation as depicted in Table 5 for welding case study (see "Results" section).
The user will enter as many different attributes to the manufacturing operations as the HTA requires.Every operation must be fully determined with the physical and perceptional attributes defined in the classification scheme.One operation can require multiple attributes for an operation.An example is grinding tip of the electrode operation (Operation 1.2.2) performed by an operator.In reality, the grinding operation is not just determined by an attribute responsible for grinding (cutting with geometrically undefined cutting edges), but also requires human-haptic feedback to control the production process (auxiliary operation).
For every process operation i recorded via the HTA analysis, the process attribute a ij related to the manufacturing classification attribute j is represented as a binary value.
Process Attribute a i, j a i, j ∈ {0, 1} (1) The binary value expresses, whether the specific process step incorporates operations that fulfil the criteria/pattern of a specific distribution of attributes.The different process attributes result in an attribute matrix A, which can be created as a result of the previous Eq.( 1): One process step can have multiple cluster features.The collected process operations are used by the clustering algorithm as a collection of many cases to determine the optimal number of clusters related to the distance measurement between certain clusters with the initial aim to maximise the jump between n-cluster and (n − 1)-cluster solutions.
The attribute matrix A represents the matrix of the analysed operations and will further be used for the abstraction process.The algorithm aims to divide n operations into k different clusters appending every observation (operation) to a cluster centre (so-called centroid) with the closest mean (Macqueen 1967).The closest mean is related to the distance of the contained clustering attributes from the centroid attributes.K-means clustering is considered difficult from a computational perspective, however, many algorithms converge quickly to an acceptable local optimum (Mahajan et al. 2012).
The generic K-means algorithm is as follows.
A set of observations (x 1 , x 2 , …, x n ) has an m-dimensional real vector.K-means clustering divides the n observations into k subsets S {S 1 , S 2 , …, S k } to minimise the sum of squared distances (Macqueen 1967).

k-means Clustering Algorithm arg min
Starting the K-mean clustering with randomised values limited only by the max/min sample value throughout every operation is generally possible.The assumption at this point is, that the patterns will translate into categorical data or attributes carrying binary values.In case the attribute values  are all binary, the identity matrix I j,n can be used to represent the starting centroids for the clustering process to advance the centroid handling algorithm explained in detail in the following paragraph.
K-means minimises the distance between the centroids c j,n and the attribute matrix by manipulating the centroid matrix C to reduce the distance vector D opt values over all distances.
The following functions show a detailed description of the steps needed to achieve the aim of Eq. (3).
The distance matrix D can be expressed according to the following equation: Table 6 depicts an example of a distance matrix for the welding case study.
The created distance matrix can now be optimised in a way that the distances are being minimised for different sizes of k.The value k represents the number of different clustercentres used.The optimal solution creates a distance vector D opt , which can be minimised using the sum of distances.The optimised distances in the previous table are marked by a (*).The distance vector represents the smallest distance of every column distance (d i,1 , …, d i,n ) according to the following equation.

Minimum Distance min D opt min D opt
The results of the equation are the minimum distances of different centroids.Table 7 shows different accumulated differences for specific k.
Five different centroids are used to cluster the existing sample.An increase of the cluster number k leads to an overrepresentation of centroids as the distances converge to zero.A comparison of the distances is an indication of the k-effectiveness.Once the distances are all zero for k, the centroids are purely a representation of all the single cases available (in terms of attribute distribution) and did not achieve the goal of reducing the dimension of the operation.
A possible solution to address this issue is a selection of an optimal k via an investigation of the distances between min D opt .As seen in Table 7, the optimal distances can be summarised to understand how well a specific number of centroids k covers the attribute vectors of the created attribute matrix A. Two criteria should be respected for the evaluation of a suitable cluster number k: • Firstly, k cannot be chosen in a way that allows the production of a trivial solution.A trivial solution means the selection of different cluster centroids k reproducing the original operations.Such an approach would not effectively reduce or cluster the operations.• Secondly, k should be pointing out the biggest 'jump' in the sum of optimal distances related to the chosen attribute matrix and cluster number k.
As seen in Table 7, the accumulated optimal distances decrease with a growing k.However, a rapid decrease of the optimised distance vector at a specific time is noticeable.This step points to several clusters significantly reducing the distance to the dataset's attribute distribution.The centroids indicate the main characteristics of the dataset.The next step considers the biggest jump in the optimised solution.
In this case, the biggest jump occurs for the optimal number of cluster centroids k 4 (Table 7, the largest difference between min D opt from k 3 to k 4).Particularly the determination of k is a research field itself.The authors have modified the Bayes Information Criterion (BIC) idea to find a possible solution to the depicted problem.The presented solution shows sufficient results for the application.Different reasons are responsible for that: • Firstly, an HTA generally has a dataset length (number of operations) considerably smaller than what is considered a large dataset in the data analysis community.• Secondly, the attribute values are binary (a i,j {0,1}) and, therefore, the created distances will be in similar dimensions and do not require a manipulation of the dataset.• Thirdly, the number of different attributes accumulated within operations is limited.
A combination of those factors reduces the number of cases in all dimensions of the dataset and the minimum distances significantly.For the case studies, the presented criterion was proven to deliver sufficient results, as reported in "Results".The determination of a specific k, representing the number of centroids, enables the allocation of specific attribute distributions to a specific cluster centre.Based on this distribution, a table (see for example Table 10) can be created presenting the percentage distribution of manufacturing attributes within the clusters.The clusters identified will be assigned as a process function.
The deliverable for this part of the work is a functional task abstraction.Based on the functional task abstraction, where specific manufacturing operation attributes have been allocated to specific centroids.
Before the results of the clusters can be finally demonstrated, the user must also determine whether the specific processes can be allocated in a specific cluster considering sequential criteria.If a process must be performed after another process (for instance grinding on different scales), sequential information must be provided.Therefore, the user is asked to answer for every process step, whether specific operations should be allocated in a stand-alone function due to a sequential importance or it should be kept in the existing cluster ("Keep", see Table 8).This step allows the tool to show the final process function considering sequential constraints.

Results
The proposed approach was tested on five industry processes, detailed descriptions of the cases studies can be found in the cited literature in Table 9.All of them are currently manual processes that are being considered for automation by the companies motivated by efficiency, skill shortage, flexibility and quality.The advanced automation solutions require sensors and intelligence to deal with process variabilities due to the highly skilled nature of the tasks.All studies were recorded and the HTA and IDEF0 representations were produced by the researchers for those processes.

IDEF0 results
First, the IDEF0 results are presented (see Fig. 5) before the results from the clustering algorithms are displayed in the next section.The IDEF0 have been produced by researchers   who studied the processes for automation, which is the current and widely adopted approach.

Clustering results
The clustering results are presented as a percentage distribution of individual attributes among different process functions for the operations identified in the HTA.In Tables 10, 11, 12, 13 and 14, the process functions (identified clusters) are displayed on the left-hand side and the manufacturing process attributes on top of the table.The manufacturing process attributes are as defined in Table 4, which could be expanded for customised attributes as appropriate to the process.

Analysis and validation
Table 15 compares the functional abstraction with IDEF0 produced by the experts, clustering results and with the actual solutions implemented in the original case studies.
The results for each case study are discussed in turn next.

Welding
The welding process was divided by the automation expert into 5 different functions (Fig. 5), whereas the clustering algorithm has identified 4 key functions (Table 10).The algorithm clustered welding (100%) and inspection (100% for texture and distance) into one function.A grinding process was identified as a function (Function 2) because in the manual process (and the HTA) the tip of the welding tool was ground by the operator.This step does not occur in the actual automation solution.Despite this, the 4 functions identified from clustering algorithm were accurate.The IDEF0 method divided the process functions in a repetitive pattern of tool preparation and setup but visual inspection and grinding the welding tip were neglected by the experts.

Grinding
The grinding process was manually abstracted into 5 different functions (Fig. 5), in contrast to 4 according to the clustering algorithm (Table 11).The clustering algorithm combines cutting with a geometrically undefined cutting edge (100%) with a tactile force perception (100%) and a tool changing attribute (20%).The second function contains a visual perception of the part surface/texture (100%).The turning of the workpiece in between the grinding processes was attributed to the pick and place function (100%).The remaining function is related to a tool changing attribute (80%).
The automation system implemented consists of a grinding application (abrasive belt) and a separate gripper containing the force/torque sensor element.In addition to that, a visual inspection system was suggested although not implemented.The tool changer was not needed as the actual solution does not require a change of the abrasive belt, due to a different type of component and it was only a prototype solution.The tactile feedback of force and torque was allocated to the gripping system since commercial grippers typically come with force and torque sensors.

Fig. 2
Fig. 2 Applied methodology informed the classification scheme.The first adapted part by Groover presents a categorisation of visual perception mechanisms for robotic automation.The second part incorporated the research by Lederman et al. focuses on the tactile perception of the humans.In accordance with their findings, a classification containing multiple perception attributes for a specific operation has been developed.The result is a combination of tactile and visual perception senses as a decision criterion to identify the required sensorial requirements (see Fig. 4).Examples of resulting operation attributes are Visual Perception Object Shape or Tactile Perception Temperature.

Fig. 4
Fig. 4 Tactile and visual perception senses as extension for DIN8580

Fig. 6
Fig.6Mapping from HTA to functions to requirements engineering

Table 1
Crystal and Ellington (2004)d empirical evidence in task analysis research derived and extended fromCrystal and Ellington (2004)

Table 2
Process representation models

Table 3
HTA example-welding case study reported by Sanchez-Salas Table 4 presents the detailed developed classification used for this work.As it can be seen it demonstrates both percep-

Table 4
Selection of classification categories based on standards around DIN8580

Table 5
Process attribute matrix for human operation analysis-welding

Table 8
Hierarchical task structure and allocated centroid-welding example

Table 10
Clustering results-welding

Table 15
Comparison of clustering and IDEF0-results summary