Introduction

Recent research in applied machine learning in manufacturing such as that of Kaji et al. (2023), Na and Yang (2023) are concerned with solving a specific application. The demand for specialists who can competently deal with information technology issues and data-driven analysis methods is increasing worldwide (Zhang et al., 2022). Data preparation using data mining (DM) and data science (DS) methods are essential for the successful application of machine learning (ML) techniques. Selecting the appropriate algorithm, constructing proper input features by experimenting with different parameters, and testing are challenging for non-experts (Filz et al., 2023). Most non-experts do not know which machine learning algorithms work optimally before trying them. Even experts often apply different ML algorithms when they deal with data that is not common in ML (Lee, K., Yoo, J. et al. 2019). Especially for the application of ML in the manufacturing industry, besides the information technology (IT) and programming knowledge as well as the mathematical-statistical knowledge, the process knowledge from the respective application area is of great importance in order to be able to use the full potential of the available database (Dehnbostel et al., 2021; Xames et al., 2023), see (Pokorni et al., 2021). In the future, a specialized AI-engineer will be needed to cover the extensive spectrum of competencies (Krüger et al., 2019). Wider acceptance of ML can be achieved, as with other computer technologies that have a large user base (e.g., email interfaces, web technologies), by improving the usability of tools for use. Due to the digitalization of the industrial working world and demographic change, digital assistance systems (DAS) are gaining strong importance in various corporate sectors across industries to accelerate the qualification and induction of a large number of skilled workers for new technologies such as ML (Apt et al., 2018b).

DAS are all systems that support employees in their actions and are embedded in a higher-level IT system (Mättig & Kretschmer, 2019). To classify DAS, Apt et al. introduce different expressions for the degree, objective and type of assistance (Apt et al., 2018a). For the degree of assistance, the categories low, medium, high, and variable are introduced. A low degree of assistance is shown for simple assembly activities, for instance. Examples for medium degree of assistance are rule-based planning processes of medium complexity. A high level of assistance is required for complex rule-based decision-making processes and open-ended, creative problem-solving processes. The degree of assistance can be variable in team structures with different levels of training and complexity of requirements. The objective of the assistance ranges from compensatory, preserving to skill-enhancing (Apt et al., 2018b). Compensatory and sustaining assistance systems offer the opportunity to balance individual, activity-related deficits and to exploit inclusion potentials along the diversity dimensions of age, disability, and ethnicity (Apt et al., 2018a).

In terms of the type of assistance, physical assistance systems are distinguished from DAS as execution assistance systems (exoskeletons; human–robot collaborations) (Busse et al., 2020). DAS supports humans in task performance in the manner of sensory perceptual support or cognitive decision making. In practice, the simplest example is the selective provision of information such as hints in the user interface of software, on the other hand, multi-step decision-making processes can be supported as well (Link & Hamann, 2019). The benefit of DAS follows from its ability to perform intelligent procedures that humans are inferior or fail to perform in the face of high levels of difficulty and system complexity due to their limited cognitive abilities (Blutner et al., 2007).

Bartschat et al. divide the user groups for computer–human interaction in DM tools related to ML into four groups: business applications (BA), applied research, algorithm development, and education (Bartschat et al., 2019):

  1. 1.

    BA: This group uses DM and ML as tools to solve commercially relevant BA, such as process optimization and automation. BA are covered by commercial tools that provide support for big data databases and deep integration with enterprise workflows.

  2. 2.

    applied research: a user group that applies DM and ML to research problems. Here, users are primarily interested in tools with proven methods, a GUI, and interfaces to domain-specific data formats or databases.

  3. 3.

    algorithm development: these users develop new ML algorithms and need tools to integrate their own methods as well as to compare them with competitive algorithms.

  4. 4.

    Education: for university education, ML and DM tools should be intuitive, have a convenient interactive user interface, and should be affordable.

In the context of this research work, a human-centered DAS was developed, which enables employees of the development and planning departments of industrial companies to apply advanced methods of DM, DS and especially ML to tabular data sets without developing lines of code. These employees generally bring mathematical-statistical as well as process knowledge from their university education, but due to shifting emphasis to IT and ML, they are usually well-versed with basic knowledge in the new areas (Apt et al., 2018b). Moreover, many users of the existing tools still fail to use them due to their complicated application. Even the operation of these tools requires extensive training. In addition, a selection of methods is provided without a suitability check whether all necessary domain requirements are covered. A DAS should provide users with step-by-step guidance and support for numerous decisions along the development process of an ML solution, with the goal of developing ML solutions in production within applied research. Typical challenges of real-world production data are nonstationary, high-dimensional datasets with unbalanced class ratios of good to bad parts. In ML Pro, a simple incorporation of typical coping strategies for the challenges in production datasets is provided for non-experts. In this work, the research question is investigated:

How must a digital assistance system be designed so that employees of development and planning departments of industrial companies can apply production-typical ML methods to tabular data sets without programming?

In this article, related work on existing solution approaches and requirements for DAS to apply ML (Chapter 2) is discussed. Subsequently, the structure, design characteristics, and utility of the developed DAS ML Pro are presented (Chapter 3) by demonstrating the functionality of ML Pro on a real-world, open-access production dataset. Chapter 4 discusses the contributions of ML Pro to research. Further research potential and key messages are noted as well.

Human–machine interaction in the implementation of machine learning techniques

The modes of interaction between humans and computers for the application of DM, which are transferable to the application of ML, can be divided into the following different categories (Bartschat et al., 2019):

  1. 1.

    Text-only interface with a programming language (command-line interface, CLI)—difficult to handle, but easy to automate,

  2. 2.

    Integrated development environment (IDE)—easy to use, but difficult to automate,

  3. 3.

    Graphical user interface (GUI); where the user selects function blocks or algorithms from a collection, defines parameters, places them in a workspace and connects them together to create complete ML solutions—a good compromise, but difficult to handle for large workflows,

  4. 4.

    Visualization interfaces such as dashboards with a collection of different charts to display different types of data and results.

Csiszar et al. emphasize the importance of a user-centered design (UCD) structure for communicating with ML solution generation tools for users with domain knowledge. The user group in manufacturing use applied research to develop ML applications. For domain experts to trust and accept newly introduced AI or ML applications, the integration of UCD into the ML development process and the incorporation of insights from interactive and interpretable ML research are recommended (Csiszar et al., 2020). An interactive system is the combination of humans, hardware, and software with which users interact to achieve specific goals (DIN 9241). The selected overall framework for computer–human interaction for applying ML using DAS is interactive machine learning (IML). In this chapter, IML and competing DAS based on IML for applying ML are discussed.

Interactive machine learning

The communication model of IML centers the human as the decision maker ("human-in-the-loop"). IML differs from classical machine learning (CML) in that it uses human intelligence by iteratively teaching and refining the model in a relatively tight loop of computation and subsequent verification. This contrasts with CML, where the workflow requires extensive reselection of training data and significant changes to the model per execution step. Amershi et al. (2015) discuss the difference between IML and CML and present several case studies that highlight the value of this approach. In IML, a mathematical model is established and refined through iterative cycles of input and verification by a group of users. The refinement of the model is done, for example, by providing and preparing selected data, applying the various operations for feature construction, or selecting algorithms as well as appropriate model parameters, so-called hyperparameters (HP), based on essential decisions made by the human, but prepared and executed by the machine in a supportive manner (Bernardo, 2020). Thus, it is possible for users to provide the system with additional information from the domain and process knowledge from the specific application. This does not mean that the computer has no influence on the development process or does not make independent decisions. For example, the computer can automatically select a subset of data and feed it into the algorithm. IML proposes to give users high-level control over the behavior of the system; without having to exercise this in every single interaction. From the interaction between computers and humans, the strengths of both interactors are synergized (Dudley & Kristensson, 2018).

Derivation of requirements for human–machine interaction

General requirements for the application of IML in a DAS are collected and derived. This is differentiated into tasks and competencies to be fulfilled by users (by Danyluk & Buck, 2019) and design rules (by Lee et al., 2019) for the development of such tools. The proposed framework for the platforms used to develop ML projects includes design rules that describe the functionalities with respect to task performance as well as design rules for the creation of the software are recommended, focusing on the characteristics and the structure.

General tasks of the users within a ML development process

Danyluk et al. elicited twelve essential tasks ((1)–(12)) required for the development process of an ML solution, presented in Table 1 (Danyluk & Buck, 2019):

Table 1 Tasks and competences of ML-Platform users. (Danyluk & Buck, 2019)
Full size table

Users' cognitive load can be reduced for task performance and decision making by providing unambiguous information in the proper place at an adequate level, as well as action strategies adapted to users' prior experiences (DIN 9241). Therefore, the essential decisions in development are addressed from problem description to model selection and configuration to result interpretation. The tasks and competencies of Danyluk et al. are also in line with more recent analyses of Rosemeyer et al (2022). which deal with the competence spectrum of ML from the perspective of shopfloor managers.

Design rules of the ML software in respect of functionality and task performance

The task-related and usage order-related structuring refers to the development process of an ML pipeline (DIN 9241). The following twelve design rules regarding functionality for the ML development are proposed by Lee et al. shown in Table 2 (Lee et al., 2019).

Table 2 Functions and tasks of ML software according to (Lee et al., 2019)
Full size table

Overall, the tasks and functions are listed in the order of the ML development process from data import to model creation to model adaptation to new data streams. Here, decisions must be made several times along the development process. A DAS supports the full decision process if the following three sub-processes are mapped:

  1. (1)

    decision preparation,

  2. (2)

    decision making as a choice between multiple alternatives, and

  3. (3)

    decision execution.

It follows that a DAS for decision support must be characterized by the following functions: Identification of a solution set, selection and evaluation of alternatives, and autonomous action (Blutner et al., 2007).

The data formats offered within DM and ML tools are based by the data structure of the raw data. Depending on the type of data and the goal of the DM process, the representation of the raw data is tailored to the task at hand. Moreover, the same raw data can be represented differently depending on the requirements of different tasks. The most widely used representation for a data set \(\mathcal{D}=\left\{\left({x}_{i},{y}_{i}\right)\in {\mathbb{R}}^{d}\times {\mathbb{R}}\right\}\) with data points of features \({x}_{i}\) and target variable \({y}_{i}\) with \(i\in \{1\dots N\}\) is the 2D feature table with N data point rows and d feature columns. Similarly, event logs and text data are two-dimensional, whereas time series data, data of sequences (genes, mass spectrogram), audio signals are 3D data. Video data and 3D images are 6D. 3D video files result in a 6D data base. The data formats of higher dimension than 2 can often be reduced to a 2D space by projection within the feature engineering (Bartschat et al., 2019).

Design rules of the ML software in respect of structure and characteristics

Lee et al. recommend general design rules for the software platform to perform ML, which are briefly introduced below (Lee et al., 2019). User management allows users to access their own projects. Project management allows users to create and manage multiple use cases in the DAS. An implemented resource management regulates the administration of created models and data in a folder system. Data processing is to be implemented to store and track the data processing steps that are executed on the models. The model executor is an implemented functionality that handles the creation of a model and the implementation of user inputs and configurations of the model and HP into the model. A wrapper manager is used to provide functionality from ML libraries such as sklearn or Tensorflow to users without having to program. The model manager maintains metadata on a project-specific basis for each model. This is information such as model name, creation time or specific information for training the model.

Overview of existing solutions of ML assistance systems for non-experts

This section provides an overview of existing solution building blocks and complete solutions of DAS to support non-experienced users and developers. The overarching logic is the division into generic tools for different domains and the specific tools for a solution. Many solutions converge supporting non-experienced ML users with automated ML (AutoML). With Vertex-AI, Google offers a commercial service with a web application for AutoML that allows developers with limited knowledge of ML to develop an ML model in the platform environment from import to output and interpretation of predictions for data in image, table, text, and video formats (Ng et al., 2022). Similar platforms are offered by Amazon, Microsoft, Apple, IBM, SAP, Tableau, MATLAB, H2O, SAS, and others. Prior research regarding AutoML has focused on the automatic determination of hyperparameters in machine algorithms. (Feurer et al., 2015; Moore, 2018; Thornton et al., 2012).

This work focuses on open-source solutions and libraries. This includes both application programming interface (API) and GUI. Several reviews have already addressed the solution of ML development in ML tools. Bartschat et al. give a descriptive and categorizing overview of commercial and academic solutions for AutoML, but do not bring any concrete design characteristics of an ideal tool for this purpose. Such a design is succinctly described in the comments of Lee et al. Bernardo shares design features for IML from the Rapid-Mix research project (Bernardo, 2020). Khalajzadeh et al. present an academic solution approach for cooperation and communication between domain experts and data specialists using the BiDaML tool to generate code templates based on diagrams to solve the use case (Khalajzadeh et al., 2020). Naik et al. compare well-known open-source tools with respect to the prediction accuracy of different algorithms for open-source datasets. No information is given about versioning or different implementation of the used algorithms in the tools (Naik & Samant, 2016). The same problem occurs in ogunleye et al. where the self-developed solution only rudimentarily fulfills the functions of a complete ML pipeline (Ogunleye et al., 2019). Slater et al. studied 40 tools commonly used for DM in education. They highlight the utility of the Python programming language and the package Jupyter Notebooks, (Slater et al., 2017). Santos-Pereira et al. provide a survey of DM tools suitable for medical datasets. Medical datasets often exhibit characteristics that are also known in manufacturing datasets: high-dimensional, time-dependent, and non-stationary data streams, decentralized storage, different data formats (numeric, text, image, graph, audio, text, video, etc.), imbalanced data distributions (Santos-Pereira et al., 2022). The characteristics are confirmed by Dogan et al. without dealing in detail with the tools for solving the ML use cases in production (Dogan & Birant, 2021). Across the board, all reviews highlight the utility of open-source libraries such as R, scikit-learn, NumPy, SciPy, Pandas, PyCaret, etc. According to Bernardo, Google Tensorflow (Abadi et al., 2016), Apple CoreML1, TuriCreate2, and Pytorch3 are gaining acceptance as ML development toolkits and APIs for developing deep learning solutions (Bernardo, 2020). Whereas most of these APIs are aimed at ML experts, some of them are also aimed at ML non-experts. However, many of these APIs remain difficult to use. A survey of the software industry highlights impairing factors for users: lack of obvious benefits, ML development experience, and learning time to adopt ML tools (Spain, 2017). Well-known open-source tools that aim to reduce the learning time of using ML by non-experts by providing a GUI are WEKA (Witten et al., 2016), RapidMiner (Mierswa et al., 2006), KNIME (Berthold, M., R. et al. 2009) and Orange (Demsar et al., 2013) (Bartschat et al., 2019; Jovic et al., 2014). Shen and Sun introduce the notion of GraphicalAI, whose graphical programming implementation is also applied in the previously listed well-known tools. Ready-made code modules interlock with each other based on linked graphical blocks with specific ports (Shen & Sun, 2021).

From the reviews, the following conclusions can be drawn regarding the listed tools. The tools are offered with basic algorithms like linear regression, but also more complex algorithms like support vector machines or tree-based methods. Modern algorithms like extreme gradient boosting (XGB), light gradient boosting (LightGBM) or artificial neural networks (ANN) (Mishra & Srivastava, 2014) are not implemented and must be added either by compatible libraries or manually. KNIME is easy to learn and use due to the already implemented algorithms. Different data types can be processed. The steps of reading in data, preprocessing, cleaning, analysis, and output as well as different possibilities for visualization are available. The scope of data cleaning and analysis is like that of WEKA and RapidMiner. However, KNIME provides few methods for error measurement and no wrapper methods for descriptor selection. Strengths of WEKA are the ease of use of a large collection of methods for classification, clustering, and data collection in a GUI. However, the CLI is significantly more powerful than the GUI. In addition, no features can be developed. RapidMiner also has low capabilities in feature development. However, it offers CV capabilities and a variety of evaluation metrics compared to other tools. Documentation and high cost if used commercially are major drawbacks with RapidMiner. Orange offers great clarity compared to the other tools, but it is only designed for smaller data sets and thus not suitable for all production applications (Slater et al., 2017).

In addition to the described tools, which were developed generically for the widest possible range of uses, a selection of existing special solutions are listed, which were developed specifically for a particular use case: Avramidis developed a GUI for the evaluation of texts (Avramidis, 2017). Bahiuddin et al. present an ML application in a GUI for the study of magnetorheological fluids (Bahiuddin et al., 2017). Nasoz and Shrestha developed a web-based GUI for which three ML algorithms linear regression, logistic regression, and backpropagation are implemented for the classification task of breast cancer (Yamamoto 2017). Carney et al. also proposed a web application for classification of image data for the use in education (Carney et al., 2020). Amashaa et al. present a Python-based GUI for clustering algorithm application (Amashaa et al., 2020). Klemm et al. have developed a tool to develop, train and test an ANN using a GUI (Klemm et al., 2018). doty et al. present a draft GUI for few-shot learning for classification of electron microscopy image data (Doty et al., 2022). Milde et al. present a GUI for the development of Convolutional Neural Networks for the analysis of image data (Milde et al., 2018). Bashir et al. present a MATLAB-based tool for outlier detection of 3D time series data from sensors (Bashir et al., 2022). Another MATLAB-based GUI for the application of multiblock data analysis of up to four sensors is proposed by Mishra et al. (Mishra et al., 2020).

Moreover, helpful solution modules are continuously published within the ML pipeline for open-source application on relevant platforms such as Kaggle or Github, which have not yet made their way into the tools offered. Zöller et al. highlight the importance for explainability of AutoML applications and develop a visualization tool to increase user confidence through explainable ML (Zöller et al., 2022). Another solution module for explainable ML is the API Shapash, which allows ML users to get an explanation for a single (local) prediction of an algorithm and thus contributes significantly to understanding within the ML development process.

ML Pro: A digital assistance system for the application of interactive machine learning in production

ML Pro is a DAS for the application of ML to feature tables of production use cases. The DAS was developed in a research cooperation between Bosch Rexroth AG in Homburg and the Chair of Production Systems at the Ruhr-University Bochum and is published as part of this publication. The DAS offers a guidance and an interactive framework for employees of development and planning departments of production companies to apply methods of supervised learning and to make forecasts for declared variables within the production. The software follows the recommendations from the previous chapters for developing a DAS. First, the software basis of the DAS is explained (chapter 3.1). Then, the structure and functionality of the DAS are described (chapter 3.2). In the last step, the functionality and the benefits are illustrated by means of an industrial use case (chapter 3.3).

Software basis of the digital assistance system ML Pro

The Intel Core i7-9850H CPU with 2.6 GHz and 64 GB RAM was used for the calculation of the model trainings and the application of the DAS. Weaker processors can lead to longer waiting times for the application of the methods. The assistance system ML Pro is developed in the Python programming language version 3.7.0. The list of various open-source packages of the virtual environment that gives the DAS its functionality in interaction is presented in Appendix A. In this section, the main packages are discussed. ML Pro is based on the GUI from the Pandas library (PandasGUI). The example in Fig. 1 illustrates the functional capabilities of the PandasGUI. Through the GUI, the basic functionality is given to process tabular data in Pandas-dataframe as matrix format. Accordingly, all mathematical operators and functions for processing the data set of Pandas (version 1.1.4) and NumPy (version 1.20.3) are available. Various analysis graphs can be generated via drag & drop operation in the PandasGUI.

Fig. 1
figure 1

Graphical user interface by Pandas as a software basis for ML Pro

Full size image

The column names of the current dataframe are shown in a list. In the example shown, a visual correlation analysis is performed with three features. The color scale distinguishes the two classes (0; 1) of the target variables to be classified. Here, the linear influences of the individual features on the target variables are visualized for the user. The example shows that the basic task of data exploration can be accomplished in a user-friendly way by this application of PandasGUI. Using ML algorithms, the multidimensional influences on the target variable can be learned and predictions in regression and classification models can be computed. By linking the Python APIs scikit-Learn, Pycaret, SDV, XGB, LightGBM, CatBoost, Tensorflow, Pytorch, DANet, etc., an extensive variety of DM, DS, ML, and deep learning methods of supervised learning are linked to the PandasGUI. There is a significant synergistic effect of the various Python packages in DAS ML Pro (in Sect. "Model executor, model manager and wrapper manager"). For data visualization, the standard graphs of the library matplotlib [80] and the interactive graphs of the library plotly are used.

Structure of ML Pro

The structure of the DAS is based on the design features of Lee et al. which were introduced in chapter 2.2.3: Project management and user management, resource management, data processing as well as model executor, model manager and wrapper manager (Lee et al., 2019).

Project management and user management

Lee et al. define various requirements for the software structure for user support in ML projects (chapter 2.2.3). Two software modules are project management and user management. The project management screenshot of ML Pro can be seen in Fig. 2. For the employees of the development and planning departments of the manufacturing industry, the application of machine learning methods should not cause any additional organizational effort. Accordingly, the software application starts with the main window of the project structure, which directly accesses the computer-internal folder structure, as shown in Fig. 2. This is where the progress and work results of the ML development processes are recorded. Higher-level, the Project window is activated when the DAS is started and remains permanently open. Project management allows users to create, manage, and control projects with different use cases. Centrally managed functions control the creation, management, and deletion of projects. If a project is created, a JavaScript Object Notation (Json) file is generated according to an implemented template, in which meta data of the project as well as project name and the user mode are stored. All information is always stored project specific. This makes it possible to clearly separate use cases and to delete all information about a project when it is no longer needed. Since ML Pro is a software that can be installed locally on a computer, user management is organized by PC access rights. Local users can open only those projects whose data and models are stored in accessible folders. Thus, an additional user management is not required.

Fig. 2
figure 2

Project management of ML Pro

Full size image

At each project design, the DAS prompts to import a tabular data set in one of the different data formats (csv.; xlsx.; db.; pkl.) via simple drag & drop application or alternatively in a folder call. At the start of the project, the target variable and the ML problem for supervised learning are defined in terms of regression and classification. In addition, one of the three user modes "beginner", "advanced" and "expert" is selected depending on the previous experience of the users. The user modes differ on the one hand in the scope of the instructions and on the other hand in the variety of functions. For the beginner mode the AutoML is provided. For the advanced mode and expert mode, the semi-automatic ML pipeline as well as the creation of ANN via buttons are provided. The expert mode focuses on time-efficient development compared to the advanced mode. The different user modes enable the knowledge transfer from ML experts to ML beginners, since established projects are saved with all development steps (data processing, feature development, training strategy, modeling) as ML models and can be called again in other projects.

Resource management

A resource management regulates the administration of created models and data in a folder system as a further required software component. All developed models, all imported and transformed data, as meta information about data processing steps, the training and all synthetically generated data are managed via a folder system and can be assigned to the respective project via the Json project file.

Data processing

Another requirement defined by Lee et al. is the data processing (Lee et al., 2019). A data processing management is to be implemented to be able to store and trace the data processing steps that are executed on the data set. The data processing steps are stored project-specifically and can be called up independently of the project. The data processing steps are stored in such a way that they can be undone at any time. For this purpose, each function for data processing and for feature engineering is stored centrally in a class and the associated meta data is stored. To be able to undo the data processing steps and the feature engineering functions, they are not immediately applied to the data set when selected by the users. Instead, the meta-data of the data processing function (e.g., deleting a column) or the feature engineering function (e.g., an Anova feature selection) is stored in an instance of a Python class. Each time a new function is added to this class, all the selected functions are executed again on the original imported dataset and this newly configured dataset is made available to the users. This has the benefit that users can deselect the implemented data processing functions and feature engineering functions at any time, and any functions still selected can be re-executed on the original dataset. This gives users a lot of flexibility and transparency as to which functions are applied to the data set.

Model executor, model manager and wrapper manager

In this research, the term ML model is used to refer to the combination of the processing steps of data preparation and feature engineering, and the selection of the algorithm with corresponding HP, as in previous studies of a hydraulic use case by Neunzig et al., (2022b, 2022c). In summary, the model executor, the model manager, and the wrapper manager are described as a defined requirement of the software. The model executor is a functionality that handles the creation of a model, the implementation of the inputs as well as the configurations of the algorithm. The model manager processes additional meta-data for each model on a project-specific basis. A wrapper manager is used to provide functionality from various libraries such as Scikit-Learn or Tensorflow to users without having to program. All models from the Python libraries Sci-kit-Learn, Pycaret, XGBoost, LightGBM, CatBoost, Tensorflow, Pytorch, DANet, etc. are managed in a central Python class. A challenge here is the different API of the packages. The models from different packages must be retrieved in different ways, but the associated data must be stored uniformly and made available to users in a uniform manner. Individual communication functions and storage functions must be written for all packages. The called models must all be storable in a uniform way and models stored as meta data in the Json file must be read in again from the Python libraries when the project is restarted. For both communication directions specific formatting functions were written, which consider the different data types of the packages.

In the following, the structure is described along the execution process in the DAS for the ML model development. The execution process of ML Pro is based on the widely used cross-domain development process CRISP-DM for data mining projects and ML models. The six phases of CRISP-DM are run through in the main windows Data Preparation, Feature Development, Modeling and Model Application. The DAS provides a guided workflow that enables an iterative development process with reversible mathematical operations as well as simultaneous adaptation and visualization of the database. The data table and data visualization can be viewed at any time. The workflow for the ML model development process in DAS is shown in Fig. 3.

Fig. 3
figure 3

Workflow for model creation within the assistance system ML Pro

Full size image

By inspecting the dataset after each computed operation, users can assess the effectiveness and correct implementation of each function on a short-cycle basis. It does not run a complete ML pipeline from start to finish. By disabling or repeatedly computing operations in an interactive table, it is possible for users to intermittently make and modify decisions along the ML development process. At any time, the current data set can be viewed in tabular form and visualized in various diagrams. For this purpose, a button is available in the upper right corner of each main window. The progress and the results can be exported to support the documentation of the project.

In the data preparation section, essential processing steps of DS and DM are applied to enhance the quality of the dataset through data cleaning and augmentation. For production, the curse of dimensionality and class imbalance are common challenges that are addressed in data cleansing and data augmentation. In data cleansing, various operations are performed to manipulate the columns and rows such as filtering by a criterion with the aim of synchronizing the data types, replacing, or eliminating invalid and implausible cell entries. In addition, users are encouraged to separately check and eliminate "data leakage". If a data set contains relevant data that is not available at the time of prediction of an ML model, so-called "data leakage" or "information leakage" is present. This leads to promising predictions in the training and possibly also in the validation data set, but to insufficient prediction accuracy for unseen data in real production operation due to lack of generalization. In the data augmentation section, various oversampling and synthetic data generation procedures can be applied to tabular data sets. This supports especially in compensating for unbalanced distribution and class distribution of a target variable, which are common in the stable processes of production datasets (Neunzig et al., 2022a) In a preliminary study, the suitable methods were compared and established in the DAS: tabular variational autoencoder (Kingma & Welling, 2019), conditional tabular generative adversarial networks (Han et al., 2021), synthetic minority oversampling technique (Chawla et al., 2002). The complicated partitioning of the different subsets of data sets and folds for the CV is performed automatically in the background.

The feature engineering section aims to use feature construction, feature selection, and feature extraction functions to significantly increase the quality of the final dataset for input to the ML algorithm. Feature construction is intended to transform or recreate features in a data set. For this purpose, mathematical formulas can be incorporated into the dataset by linking existing features so that additional process and domain knowledge can be considered (Neunzig et al., 2021). Some algorithms require certain transformations of the features prior to inputting the data. For example, scaling numerical features to ranges of values between 0 and 1 in many cases improves prediction accuracy and leads to a reduction in computation time (Nawi et al., 2013). In addition, some algorithms cannot handle categorical features, so they must be transformed into numerical classes. Within feature selection and feature extraction, the aim is to increase predictive power and computational efficiency by eliminating noise using dimensionality reduction.

For the feature selection a statistical filter method (Anova), a wrapper method (recursive feature elimination) and an embedded method (XGB) are implemented. By marking the reducing features and entering a target number, the dimension can be applied by clicking buttons without programming. In feature extraction, principal component analysis (PCA), independent component analysis (ICA) and linear discriminant analysis (LDA) are implemented. Just like the feature selection methods, entering numerical values, marking features, and activating buttons are sufficient to apply the advanced methods to the current data set. For these methods, useful pre-selections for the respective parameters have already been made for the users. The parameters can still be varied to achieve a better result if necessary. In any case, it should be determined to how many features the included data columns should be reduced. The feature extraction procedures are implemented in the program in such a way that they can be executed multiple times on the data set and the results of the different procedures can be combined. It is also possible to undo the procedures and run them again later.

In the modeling area, the different user modes are distinguished. The model development process is completely automated (AutoML) for the "Beginner" user mode to simplify the use of the program for inexperienced users. Users can select the ML algorithms to be considered for training. Training of the algorithms defaults to fivefold CV to avoid overfitting the models. After the models are trained, users have the option to have the HP of the best model improved with respect to the validation metric using a stand-alone hyperparameter optimization (HPO). For this purpose, the "Tuning" function of Pycaret is called for the HPO.

For the user modes "Advanced" and "Expert", a semi-automatic ML pipeline is implemented in contrast to the user mode "Beginner". The users can bring their own experience into the project with additional functionality and setting options, as shown in Fig. 4. Here, both random search and grid search can be applied. Depending on the algorithm, the matching HPs are loaded directly into the designated window by the wrapper manager. For each HP a certain data type is expected as input. Depending on the data type of the HP, the user can enter either alphanumeric or categorical data. Depending on the grid or random search, it may be necessary to specify a step size. Activating the "Modell erstellen" button adds the model to the model manager. After calculating the predictive metrics, the algorithms can be recalled individually with HP and their range of values can be adjusted. Users are given the opportunity to make iterative adjustments to the model development process through instant feedback. Experienced users are given the opportunity to provide additional input for the training strategy and the HPO strategy. The training strategy is model-agnostic and centrally managed to ensure comparability of results between models. In each case, the data is split into training and testing data. The ratio for splitting the data sets can be defined. In addition, a CV procedure with different split variants can be applied.

Fig. 4
figure 4

Hyperparameter optimization using random or grid search

Full size image

For the development of ANN models, a separate model executor is provided, in which via buttons, layers can be added, adapted, deleted, and visualized in the overall architecture. For the user, an ANN is automatically loaded with sensible default settings. The model builder window for ANN is shown in Fig. 5.

Fig. 5
figure 5

Model executor for artificial neural networks

Full size image

For example, the first layer is automatically added to the model based on the number of features in the dataset. For a binary classification, a sigmoid activation function is preset in the last layer, whereas for a classification with more than two classes, a "softmax" activation function is suitable. For regression problems, a linear activation function is preset in the last layer. The wrapper-manager automatically imports all HP according to their algorithms from the respective libraries. The HP "learning rate" of an ANN expects an input of type "float", i.e. a floating point number. Based on what type of data a model expects for a HP, the user interface for entering the HP is customized for the user. For example, for a floating point number, users are presented with a cell to enter numbers. If a Boolean value is expected, a checkbox is displayed to set the value to True or False. If all models are defined and the HPO strategy and the training strategy are fixed, all models in the model manager in Table 3 are trained when the button "Modelle berechnen" is activated.

Table 3 Model manager: exemplary overview of models
Full size table

The trained models are then displayed in the results table of the model manager in Table 4. The algorithm of the models whose HP are determined by HPO with the best hyperparameter combination, is displayed here. The models can then be evaluated in different ways. Regression or classification specific metrics are displayed for each model. Metrics are displayed for both the training dataset and the unseen test dataset to allow users to detect overfitting. For classification problems, results are presented in a confusion matrix. Results of regressions are visualized in a scatter plot.

Table 4 Model manager: Exemplary model results for a classification
Full size table

In Fig. 6 is the development of ML models and the model evaluation with the explaining text sections in ML Pro displayed. The user language is German.

Fig. 6
figure 6

Model development in ML Pro

Full size image

In the model application area, predictions are generated on newly added data sets. After training, the models are saved so that they can be recalled in the model application. For this purpose, the new data on which the forecast is to be performed is imported into ML Pro. If it is a regression, the tolerance limits of the specification for the target variable are set by the users, so that an exceeding of the tolerance limits can be indicated. On the newly imported dataset, all processing steps of data preparation and feature development are automatically performed according to the above model definition, which were saved and performed in the project. Accordingly, it is important that both data sets have the same format. If necessary, the processing steps can be deselected and new processing steps can be performed. The data set should then be in the same format with the difference that this one does not have the target variable, but only the characteristics. The predictions of the target variable are presented per identification number in a table on the top left and in a scatter plot on the bottom left. In addition, the results are analytically processed locally for each individual prediction to make the results explainable with explainable ML (EML) approaches. For this explainability ML prediction analysis, the Shapash library is used to determine which feature contributes to the prediction of the target variable and to what extent. In addition, the feature importance of all features can be calculated for each model. Feature importance can only be determined after applying a predictive model. Since the calculation of the feature importance requires high calculation times depending on the algorithm, the evaluations of the feature importance for each model must be requested individually by the users. The feature importance is displayed in the program with descending predictive power. Based on the feature importance, users can make further decisions to edit the model or the original database. For example, eliminating features with low predictive power could remove data noise and thus improve the generalizability of the model. Results table also shows which features have the greatest impact on a specific prediction. Activating a data series displays a graph for that data series. This graph shows which feature has the greatest prediction impact locally on the prediction of the target variable from the selected data series.

Applying ML Pro to an industrial use case

ML Pro is applied to an open-source Kaggle production dataset for predicting employee productivity in three iterations to demonstrate the utility and functionality of DAS ML Pro on real data (IshaDS, 2021). The industrial use case has already been studied by Al Imran et al. as a regression problem (Al Imran et al., 2019). Table 5 lists the 14 features and the productivity to be predicted. With respect to the statistical analysis, the following conclusions can be drawn: First, the feature "WIP" contains only 691 values, the rest of the rows are empty. Here, a decision is necessary how to deal with the missing entries. In this case, the function for replacing the invalid columns with the median value of the column provided in the data cleaning section is used. After applying the function, this column contains 1197 values. Second, the maximum of the actual productivity column is greater than one, although the study states that it should be between zero and one. Users have the option to visualize the problem graphically. From the visualization in a histogram, as shown in the data visualization of ML Pro, a small portion of the value range is above productivity level 1. Since the target variable does not necessarily have to be between zero and one, the target variable is not scaled. The characteristics "Quarter", "Department", "Day" and "Date" are not numerical but categorical characteristics. These are converted to numeric features using the ordinal encoder function provided in the feature construction section. In the first iteration, the dimension is not reduced, and no synthetic data is generated.

Table 5 Variables of the industrial use case in ML Pro
Full size table

The data were collected in the period from January 2015 to March 2015. Table 6 shows the statistics as they are presented in the program in the main window data preparation.

Table 6 Statistical analysis in ML Pro
Full size table

In the "Beginner" user mode, some algorithms are initialized with default settings. For the model development process in ML Pro, the following range of algorithms with low computation time in the first iteration has been considered: Linear Regression (LR), Lasso Regression (LSR), Ridge Regression (RR), Support Vector Regression (SVR), Random Forest Regressor (RFR), Extreme gradient Boosting (XGB) Regression, and Light gradient boosting Machine (LGBM) Regression. There is no other temporal dependence within the data that needs to be considered for the training strategy. In this respect, a fivefold cross validation with standard training to test data set split is set at 80/20. Table 7 presents the coefficient of determination (R2 score) for assessing accuracy on unseen data for all three iterations of the model development process. From the first iteration, it is clear from negative coefficients of determination that these algorithms are unsuitable for this use case. The estimator performs worse than the estimator to the mean. In the second iteration, synthetic data are included in the prediction using tabular variational autoencoders. All three tree-based procedures improve with synthetic data generation. In an automatic HPO of the tree-based procedures, the coefficient of determination can be increased to 0.454. At this point, the model could be improved with respect to the bias-variance trade-off, but this is not done here.

Table 7 Overview of the prediction results for the unseen data in ML Pro
Full size table

In the model application, 198 rows were taken from the original data set that were previously unseen. These are further considered as unseen "new" data on which the trained RFR model is run. After the data import, all previously described processing steps are automatically executed on the new data set. At this point, the global and local feature weights can be calculated. As an example, a local feature weighting is shown to demonstrate another possibility of investigation. Figure 7 shows the feature weighting of row 7 of the new data set.

Fig. 7
figure 7

Local feature importance of the data points of row 7 for the industrial use case in ML Pro

Full size image

It is evident that the top characteristic "smv" influenced the forecast value of productivity for the target value in row 7 most likely to turn out high. In contrast, the feature "incentive" has influenced the prognosis value to turn out low for row 7. The main goal of this application of ML Pro is to show the functionality and usefulness of the program on real production data.

Discussion

This research work aims to present the implementation of a human-centered DAS for the application of IML to universal use on production tabular datasets for employees from development and planning departments of industrial companies. This work has several unique contributions and implications, which are listed below:

  1. (1)

    It is generically focused on all tabular production data, i.e., other data formats must first be converted to the 2D feature table format. Unlike many other tools, ML Pro is not exclusively tailored to one use case, but is designed to be generic and extensible. Additional Python packages can be integrated into the existing structure. The DAS can also be extended with additional main windows and subcategories to reduce the dimensionality to the feature table in ML Pro.

  2. (2)

    Both the application of conventional flat ML algorithms like decision trees, gradient trees (XGB, LightGBM, Catboost), support vector machines and the development of self-developed deep learning algorithms (ANN) are enabled in a GUI. New algorithms as well as extensions of existing algorithms can be integrated into the DAS. Furthermore, the processing of image data is already prepared, which can be implemented by integrating additional layer types from Keras into the existing structure. By handling the model manager, multiple models with the same training and optimization strategy can be applied to the data and compared using evaluation metrics.

  3. (3)

    Different levels of experience and previous knowledge of the user group are considered by implementing three different user modes. The recall of models enables the transfer of knowledge from ML experts to ML advanced and novice users. Furthermore, documentation and traceability are supported.

  4. (4)

    The complete ML solution development process from importing data from database or other tabular data formats to applying ML models, visualizing, and analyzing predictions is covered by DAS ML Pro. Thus, ML Pro matches the functionality of existing approaches and structures them in an alternative way. Furthermore, models of existing solutions are extended by ML explanatory approaches. Current and advanced methods are provided for the various data preparation and feature engineering steps, and their computation can be reversibly activated and deactivated. After each operation, the data set can be visualized and exported. A demo version for the implementation of the model in real operation is also provided.

  5. (5)

    By activating buttons and entering parameters the autonomous processes of the DAS are started. In this respect, ML Pro distinguishes itself from graphical approaches that bypass the programming of the ML pipeline by linking function blocks to mathematical operators (GraphicalAI).

  6. (6)

    In contrast to existing solutions, ML Pro additionally offers the development and integration of methods of synthetic data generation and oversampling procedures for a feature table. The complicated compliance and partitioning of data subsets for the training, validation and testing dataset is calculated in the background of the tool for each dataset.

  7. (7)

    Explainable machine learning models support confidence in the solution and improve understanding of how the prediction came about (Xie et al., 2023). Therefore, the Shapash module has been incorporated as an explanatory model in ML Pro, as it is not available in any of the available tools.

As any software, ML Pro has certain limitations, which are listed below:

  1. (1)

    DAS ML Pro is limited to the different tabular formats. The data formats with dimension d > 2 must be reduced by projection onto the feature table. For this purpose, it is possible to extend the DAS for further research. This has the advantages that, for example, image and time series data can also be handled with the assistance system if the features have been examined and extracted in the previous step. In this respect, the DAS still has considerable potential for further development by adding such processing steps to the existing system. ML Pro is intentionally designed generically to handle application-independent problems in manufacturing with supervised methods. In the applications we examined, the working memory of 64 GB was by far sufficient. If data sets larger than 1 million lines are processed, the computer system should be designed accordingly strong. The accuracies of the prediction results are identical to those of console computation and have been validated accordingly. The calculation time of the predictions in ML Pro was comparable on average and was, depending on the method considered, about 10 percent in advantage or disadvantage compared to a computation in a Jupyter Notebook.

  2. (2)

    Supervised learning is the most common-used application in production (Krauss, 2022). Those algorithms of supervised learning are implemented which are listed in Table 9 of Appendix B. Nevertheless, unsupervised learning is very relevant. Oversampling and synthetic data generation methods as well as the feature extraction methods, that are implemented in ML Pro, are mostly based on unsupervised learning. ML Pro should also be extended for the unsupervised and reinforcement learning methods.

  3. (3)

    Transfer Learning forms an increasingly important role in the domain of production (Wang et al., 2023; Zhao et al., 2022). Series and parallel connection of models is only possible in ML Pro by clever exporting and importing of datasets, and thus not intended in the sense of use. The nesting and majority decision turns out to be very helpful in some use cases. In this respect, an extension of ML Pro is recommended in the future, but the incorporation was not part of the focus for enabling non-expert to apply machine learning in production as a first step.

  4. (4)

    Approaches of human-centered tools are provided for a user-individual adaptation of the DAS in terms of context sensitivity. This has not been the subject of current research and is recommended for further research. For example, in the case of incorrect use, the DAS could identify the underlying issue and suggest the respective user accordingly. This meta-level of software should be addressed as a separate focus. Both user input and meta-information of the input like speed, degree of logic as well as sensory features like eye movement, or facial expressions could be analyzed.

  5. (5)

    In the context of this DAS, the ML development process was focused. For ML non-experts, the question is which use cases should be addressed first. There is an opportunity to advise and assist industry users in selecting and assessing use cases. Here, monetary assessment methods of ML solutions have a special value for industry, but there is still a need for research in this area. Furthermore, there is the possibility to provide industry users with further assistance in monitoring ML models. Accordingly, ML-Pro can be extended before and after the ML development process.

In summary, ML Pro contributes to the application of IML for typical production data sets with the following characteristics: high-dimensional, non-stationary, decentralized storage, different data formats, unbalanced data distribution. The DAS is intended for the transfer of knowledge from ML experts to ML novices from development and planning departments of industrial companies, whose backlog of coding skills in the application of IML is overcome by a step-by-step guidance in the procedure model of ML Pro. ML Pro matches the functionality of existing solutions and offers added value through additional functionalities. Furthermore, an alternative approach to GraphicalAI for ML development without coding is presented.