Data Glasses for Picking Workplaces

Abstract

In the field of logistics the interest in wearable computing devices is constantly growing. In earlier studies it has been shown that data glasses can be a powerful tool for optimising warehouse management processes. However, those studies exclusively dealt with the impact of smart glasses on the efficiency of labour whereas our approach focuses on investigating the influence of data glasses on physical workloads. For this purpose we have designed a simplified picking workplace that enables us to carry out motion analyses and the concurrent assessment of physiological parameters under lab conditions. In this article we present the key features of our picking machine and the tailor-made measurement protocols together with the adequate evaluation methods that are used to quantify the effects of smart glasses on the physical workloads imposed on the study subjects.

Keywords

1 Introduction

Research on the utilisation of data or smart glasses in the field of logistics started some years ago [1, 2]. Meanwhile smart glasses have been tested under real working conditions [3, 4]. These pilot studies have shown that in the near future smart glasses can become a powerful tool to improve warehouse management processes. The logistics personnel of a German car manufacturer for instance used these glasses for order picking. The objective was to improve process security in production. All glass users automatically received all the necessary information such as storage locations or part numbers directly in their field of vision. According to the authors of this study it turned out that one of the main benefits of using smart glasses for picking tasks is that the users have both their hands free while they are working. The camera in the glasses is also used to detect picking errors by reading the product barcodes. Another pilot project to investigate the usefulness of smart glasses has been carried out by a German logistics company in cooperation with a warehouse in the Netherlands and an expert for wearable computing solutions. Here, the technology was used to implement ‘vision picking’ in warehousing operations. The company’s staff was guided through the warehouse by graphics displayed on the smart glass to speed up the picking process and to reduce errors. Both studies have proven that logistics can profit from the tools of augmented reality resulting in a significant efficiency increase during the picking process. However, these pilot studies exclusively dealt with the impact of smart glasses on the efficiency of labour. There are few studies that investigated the impact of smart glasses on physical workloads and they focus mainly on assembly workplaces [5] and not on picking workplaces. Consequently, our approach is to design a realistic model of a picking workplace, where picking tasks can be performed with and without data glasses. Our laboratory mock-up workplace (see Fig. 1) enables us to measure biomechanical and physiological parameters on subjects performing typical consignment tasks. In the following section we focus on the design requirements that are to be imposed on a laboratory mock-up for the assessment of physical workloads at a commissioning workplace.

Fig. 1.

CAD model of our picking workplace

2 A Picking Workplace Model

Figure 1 depicts the CAD model of our picking machine mock-up. Our laboratory workplace is based on a real industrial machine for fast commissioning. It has the same dimensions as the original but is simplified with respect to the product sorting mechanics and the objects that have to be picked. Our key requirement is to design the workplace interface for the subjects in such a way that it is as close as possible to the working environment of the real machine. In order to standardize the picking tasks we have chosen balls of different weight, diameter and color as picking objects instead of real products. The parameters of our fake products (i.e. the balls) have been derived from the original products. A single picking task is defined as follows: Our machine delivers an incoming box with a certain number of a certain product type to our subject. According to the information on the monitor (see Fig. 2) that is mounted on the rack of the mock-up the products have to be correctly sorted into four boxes (two to the left and two to the right of the incoming central box). For our study a comparative analysis of displaying picking instructions either on the display of a PC monitor or by smart glasses is planned. After having finished all picking tasks for one outgoing box the subject has to push a button thus indicating that this box is ready to be sent away. In our mock-up workplace this step is realised by mechanically removing all the balls from the completed box. But before the balls are fed back into the sorting and distributing mechanism of our machine an error analysis is performed. This task is realised by automated image processing. An image of the balls in the outgoing box is taken by a small webcam. The number and the properties of the balls are then automatically detected and instantly compared to those defined in the picking task displayed on the monitor. All errors are saved in a log file. By analysing the performance of real workers we have estimated that our picking machine has to contain several hundred balls in total. The balls having been processed by the picking subject are lifted to the mechanical sorting system by a fully automatic conveyor belt. This way, the picking objects are permanently circulating without human intervention. The methods for measuring the biomechanical and physiological parameters of the subjects performing picking tasks at our lab workplace and for assessing the corresponding workload are described in the following section.

Fig. 2.

Lateral view: typical working posture when looking at the monitor

3 Workload Assessment

3.1 Introduction

Our workplace-in-the-lab enables us to completely control the factors (i.e. the independent variables) that influence the outcome of our study. Environmental factors such as light and noise will be held constant in the laboratory workplace. As our machine is designed to deliver pre-defined and exactly repeatable picking tasks for each subject the complexity of the assigned tasks will also be kept constant. This allows us to evaluate the statistical significance of a single factor under the premise that all other factors are unchanged. If we alter for instance the way the information “where to put what” is delivered to the subjects (i.e. PC monitor vs. data glasses) we will be able to quantify the effects of these conditions on the physical workload that our participants are exposed to. This leads to the question how the dependent variable “physical workload” can be operationalised. The first part of the answer to this question can be given by making use of the following toolbox of methods and measurements:

Fig. 3.

The CUELA system

3.2 Kinematics

Body postures and movements are recorded in the laboratory study by means of the CUELA system [6]. This ambulatory measuring system is based on the IMU (inertial measurement unit) technology and includes a miniature data storage unit with a flash memory card, which can be attached to the subject’s clothing. Figure 3 (left picture) shows the attachment of the measuring system on the subjects. From the measured signals, the following body/joint angles and positions and their corresponding degrees of freedom will be calculated.

• Head: sagittal and lateral inclination

• Upper limb: shoulder joint and elbow (flexion/extension), hand position

• Cervical spine: flexion/extension

• Thoracic spine: sagittal and lateral inclination at Th3

• Lumbar spine: sagittal and lateral inclination at L5

• Thigh right/left: spatial position

• Lower leg right/left: spatial position

The trunk inclination angle will be calculated from the averaged Th3 and L5 sagittal inclination angles. The trunk flexion angle is defined as the angular difference between the Th3 and L5 sagittal inclination. The trunk lateral flexion angle is defined as the angular difference between the Th3 and L5 lateral inclination.

The data logger of the CUELA measuring system permits synchronous recording of all measured data together with physiological parameters, at a sampling rate of 50 Hz. The CUELA software enables these data to be displayed together with the digitized video recording of the workplace situation and a 3D animated figure.

Besides investigating human motion with IMU devices expensive and accurate optical marker based systems have been the state of the art until very recently. However, markerless low-cost systems have always been a desideratum in the field of biomechanics and sports science. Due to increasing computer chip power and the progress in image processing techniques the realization of such a system has become feasible. With the advent of the Microsoft Kinect sensor in 2010 a flexible low-cost tool has entered the computer game market enabling markerless tracking of human motion [7]. In 2014 an improved Kinect 2.0 version with higher tracking accuracy and reduced sensor noise has been launched. Our first pre-tests show that the 3D tracking abilities of the Kinect 2.0 are appropriate to analyze the posture of the upper body of a subject performing a picking task. Hence, for our application the Kinect will be used as an additional calibration tool in order to ameliorate the outcome of the CUELA measurements.

3.3 External Forces

The synchronous registration of ground reaction forces is realized using foot pressure sensitive insoles. Each insole consists of 24 piezo-resistive hydro cells. Based on the ground reaction forces, it is possible to detect the handled load weights by using a biomechanical model.

3.4 Muscle Activity

Electromyography (EMG) is a well-established tool in the field of physiology. EMG with surface electrodes (sEMG) is widely used as a diagnosis instrument for the assessment of electrical activity of skeletal muscles. Though it is difficult to find a simple relationship between EMG signal amplitudes and muscle force production it is generally accepted that a shift of the EMG signal frequencies is an indicator for muscle fatigue. For our main target (i.e. the analysis of the change in posture caused by the use of smart glasses for picking workplaces) sEMG is applied for monitoring the activity of the neck and trapezius muscles. Here, four differential pre-amplified (gain: 1000, band-pass filter: 5e1000 Hz) active Ag/AgCl surface electrodes (Ambu_ Blue Sensor, Denmark) will be used to measure EMG activity. The signals are converted from analog to digital (12-bit) at a sampling frequency of 1024 Hz and stored on flash memory cards in the mobile CUELA EMG signal processor for long-term analysis.

The electrodes will be placed in accordance with the SENIAM recommendations [8]. For assessment of the EMG signals, the root mean square (RMS) values will be calculated from the raw EMG data over consecutive time windows (0.3 s). The RMS values are normalized by the performance of maximum reference activities at the beginning of each measurement; all muscle activities are therefore relative to a maximum voluntary contraction (% MVC).

We suspect that the movement patterns during picking using a conventional PC monitor are an unfavorable combination of flexion and extension, lateral bending and rotation of the head to the left and the right. These complex head movement patterns are accompanied by a high activity of the arms. We use sEMG electrodes attached to all the relevant muscles that move the head during picking to quantify the corresponding muscle activity and possible fatigue.

3.5 Physiological and Psychological Aspects

Additional emotion-related physiological parameters such as heart rate, skin temperature and conductivity are recorded using wearable computer systems. Robust wearable computer systems are available that are designed for users who want to gather information on the emotional state of their subjects without being forced to take care of measurement artefacts, filter chains, or the irritating effects of wires imposing restrictions on the on the subject’s mobility [9, 10]. The emotion-related parameters add a psychological component to our dependent variable that we originally introduced as physical workload. The psychological aspect of the workload concept is further extended by using a standardized questionnaire to elicit subjective viewpoints of the study participants after having finished their “picking shift”. The Nordic Questionnaire [11] lets the subjects rank the musculoskeletal discomfort experienced during the task as well as the wearing comfort of the computer devices.

3.6 Defining Physical Workload

The second part of the answer to the question of how to measure and quantify the variable “physical workload” is the interpretation and integration of all the data collected with the modalities described above. As a first step existing guidelines for the classification of body postures are taken into account. The software tool WIDAAN that is part of the CUELA system uses the ISO 11226 norm to automatically classify posture angles according to the lights of a traffic signal (see Fig. 4).

Fig. 4.

Classification of upper body postures

After measurement it is possible to mark any actions or situations to highlight certain work activities and have them evaluated. The CUELA software automatically issues a series of statistical evaluations to give a quick impression of the quantified risk factors. Body angles and postures are analyzed with reference to the literature and some relevant standards:

• Extreme body angle positions, asymmetrical posture patterns [12]

• Static postures (assessed in accordance with European standards)

• Repetitive movements according to [13, 14]

For each measurement, it is possible to have an OWAS (Ovako Working Posture Analysing System [15] ergonomic analysis carried out. The software automatically identifies work postures classified in accordance with OWAS in connection with the handled weights and evaluates them statistically. As a result, the user receives a list of priorities that distinguishes between four risk classes (action category/class of measures). For the biomechanical assessment of manual load handling and to estimate the associated load on the spine, the measured data can be entered as input data into biomechanical human models [16]. Apart from the measured body/joint movements and forces, the model also requires the subject’s data, e.g. body height, length of limbs and body weight, as input variables. From this, force and torque vectors are calculated at the model’s joints. For estimation of the loading on the lumbar spine, an interface to the biomechanical model “The Dortmunder” [17] exists.

Finally it is planned to carry out a field study at the real picking workplace of our partner company in order to validate the results achieved under lab conditions.

4 Study Design, Hypotheses and Statistics

As mentioned earlier it is our main goal to quantify the effects of using smart glasses during picking tasks on the physical workloads of the subject. A first study capable of yielding the desired result should have the following general design. Two groups of subjects are chosen randomly out of a huge pool of possible subjects. In our case the population is defined by all the students of our campus (>2.500 persons). Two unbiased samples of this population are then drawn by using a random matriculation number generator. Students that have been selected by this random process are motivated to take part in the study by offering them a financial compensation. The size of the groups is determined by a statistical power analysis. Each subject is assigned an identical set of picking tasks although the order of the tasks will be randomised. Members of the first group (i.e. the control group) receive their picking information on the conventional PC monitor whilst the members of the other group use smart glasses. After having tested several smart glasses available on the market (e.g. Epson, Google) we have decided to choose the Vuzix smart glasses because it turned out that the Vuzix offers a good compromise between wearing comfort and reliability. The first version of Google Glass was not robust enough to be taken into account for our study. However, a new version of Google Glass has been announced. As soon as it will be available on the market we will test the usefulness of the new Google Glass version for our application.

The null hypothesis H0 and its undirected alternative H1 for this study design are formulated as follows:

• H0: The use of smart glasses during picking tasks has no effect on the physical workloads imposed on the subjects.

• H1: The use of smart glasses during picking tasks has an effect (positive or negative) on the physical workloads imposed on the subjects.

Statistical hypotheses testing methods are then applied to answer the question whether H0 can be rejected (and if H1 can be accepted).

5 Expectations and Outlook

Currently, our laboratory workplace is under construction. The set of sensors listed above is at our disposal. A set of pre-studies has already been designed in order to quantify external factors such as aspects of software ergonomics, the influence of the subject’s psychological status and the distortion of the results caused by working with a full body sensor suit. The mock-up is expected to run by the end of May 2016. A first pilot study becomes feasible in May/June 2016. Preliminary results of this pilot study could then be presented at the conference in July 2016.

In the near future, smart glasses will become a popular tool in many working environments. So far, the impact on working efficiency has been the main focus of current research. However, smart technology might as well be beneficial for the health of working people as it improves posture and likewise reduces the risk of work-induced degeneration of muscles and joints. Smart technology will then assist in establishing “physiological movement patterns” at the workplace and help to reduce costs for the health care system. However, in order to test this hypothesis it is crucial to first setup an artificial test environment and to quantify the effects. Thus, monitoring the change of movement patterns caused by introducing smart glasses using sophisticated sensor technology is a key factor for predicting long term work-related health issues.