1 Introduction

Increased longevity of the population is a worldwide phenomenon with profound implications for society. The ageing process, which is only partially related with chronological age, is a heterogeneous group of health and functional states experienced by older people and determined by a range of genetic, biologic, social environmental, psychological and cultural factors [4]. Although with a significant diversity, the aging process is characterized by a gradual impairment in many body systems and an increased risk for disease. The Central Nervous System is affected by changes in neurotransmitter levels and neuronal function, brain atrophy, reduction in oxygenation and cerebral blood flow among others [1]. Some deterioration in memory, information processing speed, inductive reasoning, numerical abilities as well as impairment in motor and visuoperceptive functions are commonly found in older persons. Yet, age associated cognitive changes are not irreversible and improvements can be achieved with adequate training [1].

Thus, cognitive stimulation is a component of “active aging”, a term that was adopted by the World Health Organization (WHO) to promote a better quality of life and improved autonomy and independence of older people. There is evidence that regular engagement in moderate physical and cognitive activity can delay functional decline and the onset of chronic disorders in older subjects [19]. This not only stimulates neuronal plasticity [12] but also makes use of the “cognitive reserve” as additional brain regions are recruited during the task to compensate the reduced functional capacity [2, 5]. Optimizing cognitive function is an important objective since cognitive decline is associated with adverse outcomes in mental and physical health as well as in longevity [8, 18]. Stimulation and monitoring of cognitive performance can potentially be implemented with Serious Games, a class of games often simulating practical daily-life situations for professional training in critical conditions as well as for educational purposes targeting a diversity of users [20]. The increase utilization of Serious Games in immersive environments and the adoption of non-conventional devices has strengthened the relation with Digital Games. The possibility of generating virtual scenarios can increase motivation of users during the learning process. Previous research demonstrated that Digital Games are beneficial to old age users [10] specifically in visual perception [9], spatial orientation, reaction time [3], eyehand coordination and quality of life [11]. Green and Bavalier [14] suggested that these benefits are related to increased dopamine levels in the brain, (which decline with age,) that, when elicited by digital games, have an important role in cognitive performance following the training session.

Serious Games share some characteristics with other methods of cognitive stimulation, as defined by Franco and Orihuela [9]. These games involve a task which is continuous, systematic, stimulating and providing reinforcement, avoiding the routine and repetition. The task can also stimulate specific cognitive domains, especially attention-concentration, facilitating the stimulation of other domains. In addition, Serious Games fulfil the criteria proposed by Thompson and Foth [17] for cognitive stimulation in older persons: easy accessibility, not too expensive and user-friendly.

The aim of this study was to develop and test a serious game (Counting Sheep) to be applied in cognitive stimulation of older subjects, while providing, at the same time, a useful indicator about the cognitive performance of the player, which could be useful for healthcare professionals. Thus, our main question is: which game play indicators and metrics could be used as a good proxy for predicting (a correlate) evaluation with a rapid screening instrument, such as that obtained with Montreal Cognitive Assessment (MoCA)?

In the second section of this study, we will present the idea of the Serious Game and its functioning in comparison to the traditional methods of cognitive assessment. In the third section, we will present the methodology and the samples used in the research. Next we will present the results of testing this game with the sample population, together with the results of a MoCA. The analyses of the research are outlined in the fifth section. Finally, we will present the conclusions of this paper and future works.

2 Design Proposal

2.1 The Counting Sheep Game

Since a significant proportion of older subjects have mobility problems this game was developed for mobile technology (Smartphones or Tablets), to make the solution easily available everywhere. Thus, even the user with mobility problems can have access to the game on a variety of places including in hospitals, nursing homes or their own residence. Thus the choice of mobile technology was considered convenient to meet availability needs of the players [15, 16]. In this game the player will be asked to identify the number of Sheep that will appear in the Enclosure.

During the game, sheep and wolve representations are presented during short periods (Fig. 1). To identify the number of sheep, the player needs to focus on the screen and process the visual stimuli. To increase stimulation and challenge attention wolves will also be randomly generated, and the player will be asked to identify the number of these animals [7]. The operation of the game goes through the following steps: (1) When the game starts, the player will have one second to view the number of Sheep and/or Wolves in the Enclosure; (2) Sheep and/or Wolves will be moving; (3) After one second, the player will have to answer with the number of Sheep and/or Wolves in the enclosure; (4) The player will be asked randomly about the number of Wolves; (5) In case the answer about the number of Wolves is not correct the player will not be penalized; (6) If the player provides a correct answer the difficulty level of the game will rise; (7) If the player gives a wrong answer, the difficulty level of the game will be reduced and the player can only miss two more times, i.e., the player will only be entitled to three attempts.

Fig. 1.
figure 1

The counting sheep game challenge and response interface.

Full size image

Counting Sheep is culturally adapted to the target population (with a rural scenario and familiar animals) which is important to obtain reliable performances and assessments. The game variables are described in Table 1.

Table 1. Variables of the Game Counting Sheep.
Full size table

The game level was designed such that, when the player hits the correct number of sheep, the game gets harder, and when there is an error on the response on the number of sheep, there is a decrease in the number of sheep and wolves. Therefore, two stacks were implemented in order to store the number of sheep and wolves in each level, and both randomly draw a value between zero and three to add to the existing number of sheep and wolves. The same happens when the player misses the quantity of sheep, although the random draw will instead reduce the number of wolves and sheep. The total stack of wolves and sheep is limited to a minimum of one and a maximum of fifteen.

By means of experimental analysis, we wanted to compare the performance of the players in the game with the results obtained in the MoCA and thus, validate if the game could serve as a rapid cognitive stimulation and evaluation instrument. With this purpose, we wanted to collect gameplay data that could help us identify possible game performance variables that could serve as proxy for the MoCA evaluations.

2.2 The Montreal Cognitive Assessment (MoCA)

This is a brief screening test for cognitive function which can discriminate between normal and cognitive impairment in older adults [9]. This test has a one-page protocol and takes 10 to 20 min to apply. It doesn’t have adaptations for education level, sensorial impairment and it lacks a ludic component. It is not particularly sensitive to assess some cognitive domains.

Executive functions are assessed by the part B of Trail Making Test (the subject has to link letters and numbers in alternate order); Phonemic Verbal Fluency (also included in Language); Verbal Abstraction (also included in Abstraction). Visuo-spatial assessment consists in the Clock Drawing Test (circle, numbers and hands) and copying a cube. “Attention, Concentration and Working Memory” are assessed with digit span test in direct and reverse orders. A Sustained Attention task (target detection) consists in identifying the letter “A” during the pronunciation of a series of letters. Finally, in the Serial Sevens subtractions the subject has to consecutively sub-tract 7 starting from 100. Language tasks consist in naming 3 animals, repetition of 2 sentences with complex syntax, and phonemic verbal fluency in which the subject is asked to generate as many words starting with “p” as possible (excluding proper names). For verbal abstraction the subject must think and verbalize the similarity between two objects (e.g. banana and orange being fruits). Differed memory recall is tested 5 min after retention of 5 words (short term memory). The subject has two trials to recall the words after completing other tasks (attention, language and abstraction). The Orientation domain is assessed questioning the subject about the time (e.g. date, month) and location. Each task has a score as follows: Executive Function/Visio-spatial (5 points); Naming (3 points); Attention (6 points); Abstraction (2 points); Memory (5 points) and Orientation (6 points).

The sum of each individual score provides a total score (max. 30 points) which can be compared with standardized values according to age and educational level.

2.3 Relation Between MoCA Domains and Counting Sheep

Table 2 presents the expected overlap of cognitive competences, in a comparison between the cognitive domains assessed by MoCA, and the demands of the Counting Sheep game, from the perspective of a trained Psychiatrist. The level of overlap is classified as strong (+++), moderate (++), weak (+) and none (−).

Table 2. Expected relation between MoCA evaluations and Counting Sheep overall test.
Full size table

From Table 2 we would expect, as our initial conjecture, to find an empirical relation between the performance within the current Counting Sheep game and a cognitive assessment, especially relevant along the visuospatial, attention-concentration, and memory functions.

3 Methodology

Group 1 was composed with subjects with high cognitive performance attending a cultural Academy (Academia de Convívio e Cultura da Casa Cor de Rosa) and a senior University (Universidade Sênior Nova Acrópole). Group 2 consisted in subjects with cognitive impairment recruited in the Old Age Psychiatric Unit of Centro Hospital Universitário de Coimbra. All subjects with age ≥ 50 years were eligible to enter the study. The final sample consisted of 55 subjects in Group 1 and 51 subjects in Group 2 (Table 3).

Table 3. Group 1 and Group 2.
Full size table

After explaining the purpose and details of the study to each participant, all participants gave consent and agree to participate in the study; the research was conducted with the following steps: (1) Baseline cognitive assessment (MoCA); (2) Demonstration of the game (rules and how to play); (3) Game play during 10 min; and (4) Administering post-test questionnaire.

4 Results

4.1 Cognitive Assessment and Game Performance

This section aims to analyze the results found in groups in both the MoCA test in the Counting Sheep game. By applying the nonparametric Mann-Whitney test to the MoCA results belonging to samples “Group 1 and 2” (Table 4), we can conclude that the samples showed different results in the variables (Executive Function, Naming, Attention, Language, Abstraction, Memory, Orientation and Total Score). The reason for the choice of two distinct groups was to analyze the behavior and performance of the game as a mechanism of stimulation and evaluation in aged people who receive hospital follow-up and in aged people that, in principle, did not need hospital follow-up, that is, they were two possibly disjoint samples or population regions from which we would expect clearly diverse results.

Table 4. MoCA evaluation results with Group 1 and Group 2.
Full size table

The sample “Group 1” (Table 5) achieved higher levels in comparison with “Group 2” (Table 5), since the average number of levels completed by “Group 1” was 11.67, while “Group 2” completion average was of 5.61. It is known that the maximum amount of errors allowed in the indication of number of sheep is three, but if the player hits the right quantity of wolves he recovers a lost attempt, so sample “Group 1” committed an average of 6.69 errors in reporting the number of sheep, while “Group 2” made an average number of 4.65 errors.

Table 5. Performance variables in Game selected from 127 logged items, based on the correlations identified - Groups 1 and 2.
Full size table

By applying the nonparametric Mann-Whitney test to the game results belonging to samples “Group 1” and “Group 2” (Table 5), we can conclude that different results were achieved in the variables “Maximum Level”, “Number Scores Wolves” and “Number Errors Sheep”. Variables “Time spent in the game opening” and “Time spent in the game tutorial” showed similar results in both samples. This is relevant to identify which performance variables could distinguish between the groups’ performance and be possible candidates for approximating the MoCA based evaluations.

We concluded that, by the results achieved in MoCA, there is a higher incidence of people with cognitive impairment in sample “Group 2”, compared to sample “Group 1”. “Group 1” presented a more skillful behavior in the game in regards to the behavior shown by group “Group 2”. Finally, “Group 1” is characterized as a better qualified sample in what regards the cognitive context, which is shown by its participants achieving a higher score in both the game and MoCA compared with those of the sample of “Group 2”.

4.2 Correlations Between Game and MoCA

This section is intended to explain the choice of the correlation method used and verify if there was a correlation between the MoCA assessment and the performance in the game. To do so, the correlation method adopted was Spearman’s ®, because of its characterization as a nonparametric correlation measure [17]. Once r, the sample correlation coefficient, is calculated, it is necessary to determine whether there is sufficient evidence to decide whether the population correlation coefficient ρ (significance) is conclusive in a designated level of significance α [14].

Present below are the results of the correlations between the MoCA and the Counting Sheep game, used in both samples. By the performance of players of “Group 1” and “Group 2” with the MoCA (Table 6), we get (C: correlation; S: significance and N: sample):

Table 6. Correlation MoCA evaluations and game performance variables for Group 1 and 2 (showing only meaningful variables out of 127 logged and calculated game play metrics).
Full size table

The performance of the players of “Group 1” and “Group 2” with the variables “Education” (the values adopted were the following: 0 – No formal education, 1 – Until 12th year and 5 – Technical Expertise/Higher Education), “Age” and “Occupation” (the values adopted were the following: 1 – Active Worker, 2 – Retired and 3 – Domestic Worker) (Table 7) showed that (C: correlation; S: significance and N: sample):

Table 7. Correlation Group 1 (N = 55) and Group 2 (N = 51) with selected game play indicators.
Full size table

4.3 Feedback from Users

As mentioned beforehand, after participating in the experimental phase of the research, players were asked to answer a questionnaire to evaluate their views and level of satisfaction with Counting Sheep game.

In Table 8, we show the results of “Group 1” and “Group 2”. It is noticeable that in both samples, players understood the game (rules and gameplay) (96.36% “Group 1” and 96.08% “Group 2”). 96.36% of “Group 1” and 94.12% of “Group 2” found the game as a valid practice as a daily exercise at home. 98.18% of “Group 1” and 98.04% of “Group 2” had no difficulty in recognizing and viewing the animals and the buttons of the game (1.96% of the sample of “Group 2” had vision problems such as cataracts). Finally, both groups used the finger to perform the operations (100% “Group 1” and 100% “Group 2”).

Table 8. Reception of the Game by Groups 1 and 2.
Full size table

5 Discussion

For “Group 1”, the “Attention” was the cognitive domain that presented the highest correlation coefficient with the maximum level that the player obtained and with the other variables of correctness or error in the quantity response of the animals. This is a game that demands more attention from the player to observe the quantity of animals exposed during the attempts and, consequently, to answer the quantity of the respective animals correctly.

The cognitive domain “Language” is activated in the context of understanding the rules and gameplay of the game. It is also possible to notice a pattern with time-related variables of the game. Players from the “Group 1” sample with high scores in the “Language” domain tend to spend less time answering questions in the game. The greatest amount of correlations occurred in the declaration of wolf numbers. Unfortunately, it was not possible to understand why the higher incidence of correlation in the response time of the wolves. We emphasize that, with the increase of the player’s attempts to answer the number of the respective animals, the correlation strength increases. For the variable “Total Score”, there is a correlation because players that present high performance in MoCA, that is, absence of cognitive deficit, tend to reach higher levels.

For the “Group 1” sample, the cognitive domain of “Naming” is activated to respond to the number of sheep (goal of the game). With the increase of the player’s attempts to answer the number of the respective animals, the correlation strength increases. Therefore, the level of perception to name the animals might be important for the players during the attempts to answer the challenge.

The “Orientation” domain (Group 1) was correlated during the response time (not shown in the table) more often present in the response time for the number of wolfs. Unfortunately, it was not possible to know why the higher incidence of correlations in the response time of the wolves. With the increase of the player’s attempts to answer the number of the respective animals, the correlation strength increased. For the players in “Group 2”, “Attention” was correlated to the activity of responding to the number of sheep (goal of the game) and the time used to respond to the animals, thus, this cognitive domain was important for response performance and, the higher the player’s attention scoring in the MoCA, the less time the player took to respond. We emphasize that, with the increase in the player’s attempts to answer the number of the respective animals, the correlation coefficient increases.

In “Group 2”, the cognitive domain “Naming” exhibited a similar behavior to that of “Executive Function”, that is, the level of perception for naming animals seems important for players during answering attempts. Throughout the results, across game levels, we noticed that the correlation coefficients between these cognitive domains and the answering variable remain constant.

Thus, for the “Group 2” sample, some domains are important for a player’s best performance, such as “Language” (understanding the rules and gameplay of the game), “Naming” (perceiving which is the animal, processing the question and answering, e.g. “three wolves” and “two sheep”), “Orientation” (spatial orientation of how many animals are exposed), “Executive Function” (in answering the questions of the quantity of each animal). The “Attention” is the cognitive domain that presented the highest correlation coefficient with the Maximum Level that the player obtained. Therefore, this is a game that demands a greater attention from the player to observe the amount of the animals exposed in the attempts. For the variable “Total Score”, there is a correlation because players who perform high at MoCA. That is, lack of cognitive deficit, tend to reach higher levels and make more attempts to inform the number of the respective animals.

The “Education” of the players of both samples presented a similar behavior in the correlation coefficient. “Group 1”, is a sample composed of people with schooling and, in “Group 2” there were some members who did not have schooling. So, results of “Group 1”, presented a correlation, also common in the results of “Group 2”. The players who presented better performances had a higher level of schooling. It can be concluded that, for the Game, a minimum is required in the concept of schooling (counting animals and number recognition).

The age of the players in both samples allowed to conclude that players with less age presented the greater amount of correct answers to inform the number of wolves, compatible with a generally healthier situation. This makes it possible for the younger group to miss the number of sheep more often and thus not reaching the amount of errors that the game allows. As expected, another characteristic factor for player profile is that the higher the age, the lower the probability of reaching higher levels (inversely proportional correlation). These are important aspects relevant for the calibration of the game as a screening instrument for target groups.

Finally, the Occupation of the players did not present significant correlations, that is, the occupation of the players was not an important factor for the performance in the Game, or the characterization of occupation was not adequate for this purpose. Through results in Table 9 it was possible to notice that the game environment did not present major difficulties in the context of usability for the target population. There was encouragement of the players in particular of those in “Group 2”, to play the game.

Table 9. Evaluation of the players in the sample Group 1.
Full size table

As a tool to help health professionals interpret players’ results, normative tables of player performance have been constructed (Tables 9 and 10). In the context of cognitive assessment, Table 9 shows the performance “Group 1” in the game and makes it possible to evaluate the performance of the sample players against their population profile. The table was assembled with “Highest Level”, the most efficient variable in the evaluation criterion of player performance and the other variables chosen are profiles such as “Age” (important variable to indicate cognitive decline) and “Education level” of the player (Second important variable to indicate cognitive decline) [6].

Table 10. Evaluation of the players of the sample Group 2.
Full size table

Similar to the table above presented in the context of cognitive assessment, Table 10 exposes the “Group 2” profile in the game.

The results of the investigation also allowed us to conclude that the game presents a challenge better calibrated to screen the performance of “Group 2” than for “Group 1”. This is because “Group 2” contains players with cognitive limitations (some with cognitive pathology), the opposite occurs in “Group 1” (people without diagnosed cognitive pathologies or significant deficits). This calibration is confirmed by the clear number of correlations between the MoCA and several game play indicators, where “Group 2” presented a significant amount of correlations compared to “Group 1”.

When we put together both sample groups, still not a representative random sample, we gain a perspective that we can notice a promising trend line correlating performance in the MoCA screening test with game performance (Max Level achieved), thus justifying the need and usefulness of further investment in testing this approach (Fig. 2).

Fig. 2.
figure 2

Total Score (MoCA) and Maximum Level (Game) by Group 1 and 2.

Full size image

6 Conclusions

Our research had the goal of preparing a preliminary case study on the development of a serious game, which aims to cognitively stimulate its users and allow further study a possible relation between gameplay metrics and the assessment of cognitive abilities with the MoCA instrument. The prototype was tested with two groups that met the sample inclusion criteria of the senior target population. In order to contribute to the development of applications that could evaluate the cognitive performance of the senior public against a model for the target population profile, we carried out a preliminary study, with the aim to find game play indicator variables that could present a relationship between the cognitive stimulation and the cognitive assessment of the player. The research used a cognitive screening mechanism validated in the local population (MoCA), in order to enable a comparison with performance gameplay metrics recorded by the game, seeking a possible correlation with the cognitive domains presumably being activated.

The particular game design has shown to be better adjusted (rules, gameplay and difficulty) to the sample of “Group 2” (a clinical sample). This fact can be confirmed by the number of correlations that existed between the game and the MoCA for sample “Group 2”. For “Group 1” the progressive increase in the level of difficulty of the game seems to have been accommodated. The maximum Level achieved variable showed to be a promising proxy correlated with the total MoCA evaluation from a screening test. Normalized performance tables for age and education profiles were presented that showed the plausibility of discriminating the performance in the game for each profile.

In the analysis of the game and results it was possible to conclude that the game in the current conceptual form can be used as a mechanism of evaluation with elderly public (with or without cognitive pathology). However, as a mechanism of cognitive exercise, It can be quite demotivating for cognitively healthy people because as mentioned, the game does not appear to be challenging enough for cognitively healthy people.

We found that such a game device could become an easy engagement tool that would allow health professionals to work with cognitive impairment indicators of their patients, and implement game-based stimulation strategies aimed at ensuring a better quality of life to the elderly population. As a preliminary case study, this exemplifies the development of a game for cognitive screening, which can be used independently by players repeatedly as cognitive exercise, i.e. games are relevant tools that can be used as a stimulus supplement (exercises), assessment of the players and may offer useful information to the professionals involved in the care of the patients.

In order to improve the prototype, and therefore obtain data to better address the object of study, other studies must be carried out, such as: (1) Create a variable to store the amount of each animal, even in case of hits or errors. This will benefit the analysis of the results reached by each player. (2) In order to avoid the infinite “loop” of errors and/or hits of the players, a new rule will be added: When a player commits three errors in a row when reporting the amount of each animal, the game will end, even if there are attempts still available (to solve the cases of players that only remark one kind of animal). (3) Deal with lack of skill in handling the device, nervousness, or anxiety that can result of a cognitive disease or a dislike for gaming.