Smartphone Nexus 5 was applied (behavioral and contextual data collection, provision of current stress level self-assessments) for the purpose of developing a solution for stress determination in an indirect way (noninvasive). No additional wearable sensors were applied. The following sections describe the proposed approach for data collection and analysis.
Behavioral and contextual data
Application based on framework Funf [18] gathered behavioral and context data. Then, these data were uploaded to a server in SQLite format upon availability of a WiFi connection. Once all the data were collected, it was imported to the database MySQL in separate tables, and the titles of the tables corresponded to the means of the collected information. The field “time stamp” was a primary key for every table, unique within a single table and corresponded to the time of occurrence of the event in the Unix format in 1-s increments. The schema of the main database storing information obtained from a variety of built-in smartphone sensors is depicted in Fig. 2.
Brief description and collected data properties are listed below.
-
(A)
Assess: current self-assessment of stress level based on a seven-scale questionnaire Nasa-TLX [19]. An Android application “Stress Tracker” was developed for this purpose, the interface of which is depicted in Fig. 3. This application displayed a request to evaluate the current stress level of a participant from 09:00 to 19:00 every hour at a random time. The time interval between the requests could not be less than 30 min.
We used a binary classification; consequently, evaluation marked with “1, 2, 3” was classified as “low stress,” while the levels “4, 5, 6, 7” were classified as “high stress.” Self-reports about current stress level were used as ground truth to label the collected data into two above-mentioned classes. After the labeling process, the data from the “assess” table were submitted to the main database table “base_weka.” Following the initial preprocessing, the data from other tables were also submitted to the database table “base_weka.”
-
(B)
Audio: Ambient noise data were collected every 300 s. Time interval between measuring of ambient noise (and other parameters below) was chose with the condition that a smartphone could hold a charge for at least one full day, as the smartphone was planned to be simultaneously used for usual daily activities. The data included (1) MFCC (mel-frequency cepstral coefficients) which characterize the signal based on its spectrum and amplitude; (2) l1norm (least absolute deviations) and l2norm (least squares); (3) frequencybands, i.e., power spectral density across frequency bands; and (4) diffSecs, i.e., difference in seconds between records up to the twelfth decimal place; this parameter was not involved in the computation, but it was used to test the accuracy of event detection. After completion of the data collection, average values for 3-, 5-, and 10-min intervals were calculated for all the audio features in order to consider the short-term and medium-term impact of changes in audio features on the stress level.
-
(C)
Gyro: Gyroscope readings on X-, Y-, and Z-axes were recorded every 120 s. Then, average values for 10- and 15-min intervals were calculated.
-
(D)
Screen: Every case of changing the screen mode (on/off) was counted. Information on the number of the phone screen switching on and off for the last 10 and 20 min was submitted to the main database table.
-
(E)
Light: The values of the ambient light were detected every 300 s. The light values for the last 10, 20, and 30 min were submitted to the main database table.
-
(F)
Probe: Based on the data received from ACC, the activity level (none, low, or high) was determined. Values of the activity level for the last 30, 60, and 120 min were submitted to the main database table.
-
(G)
Activity: We developed an Android application for classification of the current activity (walking, standing still, on car, tilting), which is compatible with Google activity recognition API [20]. The “tilting” activity corresponds to a sedentary or standing state of the user. This application shows the type of activity, confidence, date, and time in which the event was detected. The application interface is presented in Fig. 4. The current activity was recorded every minute for the purpose of receiving more detailed information. The detected activity was recognized as true if the confidence was more than 50 %. The data were further exported to.csv format, and then, it was submitted to the MySQL database.
The above data and its properties were selected for the stress analysis based on previous experience (according to Table 1).
Activity recognition
We decided to develop an application based on the Google activity recognition (GAR) for several reasons. Firstly, there is no need in Internet and GPS access for determination of the current activity, so such condition significantly reduces battery consumption. Secondly, this is a non-obtrusive method which does not require wearing any additional sensors by the user or any other interference due to the fact that only the data from smartphone ACC are used for the activity recognition. Thirdly, preliminary tests showed that the accuracy of determination of the mentioned activity types by this method was sufficiently high.
Nevertheless, it was necessary to interpret the resulting data on daily activities in the right way. It has been observed that in the process of data collection, there can appear several activities different from the activities block of one and the same type, not being a result of an error of the activity determination algorithm. For example, if a group of “car” blocks is followed by 1–2 units of “standing,” and then, the group of “car” blocks resumes the chain, it can be a result of stopping due to a traffic light or a traffic jam. Consequently, we could replace those 1–2 units of “standing” activity with “car” activity units, because while waiting for the enabling signal to turn on, the person is still in the car, and then continues movement. An example of such a situation is illustrated in Fig. 5 in row 999, where the fragment of the activity recognition application log is presented.
Thus, we will assume that the “car” activity ends if there are at least 3 units with activities other than “car” at the moment of its last detection. However, “standing”, “tilting” (the device angle relative to gravity changed significantly. This often occurs when a device is picked up from a desk or a user who is sitting stands up [21]), “walking” activities may have blocks of 1 and 2 units of an activity. An example of this can be a case of working on the computer followed by a short coffee break. To avoid motion artifacts that affect the data collected, it was important to understand whether the user is in a fixed position, which corresponds to the “standing” type of activity.
According to [22], to use stress models with physiological responses of body in practice, we must be able to correctly detect stress even during other activities that affect one’s physiology. In this study, we were interested in understanding the possibility of application of the behavioral pattern and contextual data to accurately detect stress even during various daily activities. And there also was a question to be answered, “Will information on the current user activity help build a more accurate classifier?” The question is reasonable because according to Table 1, the field of stress determination using the data received from a smartphone as the only and sufficient source with consideration of different types of activities is not fully understood yet.
A collection of algorithms for machine learning WEKA [23] was applied for data analysis. After collection and import of all the data to the software package WEKA, three models of classification were built:
-
1.
Initial model based on the analysis of behavioral and contextual data derived from a smartphone.
-
2.
Second model: The data on the activity type at a certain point of time were added to the initial model.
-
3.
Finally, a separate model of classification for the “standing” activity type was built. We wanted to check that can we get advantages due to building separate models for certain type of daily activity.