Keywords

Introduction

In the current digital era, leisure activities are increasingly intertwined with information and communication technology, especially for children and young people who have come of age immersed in the digital realm. Today’s youth have increased exposure and familiarity with digital technology from a very young age, having grown up in a world where technology is ubiquitous and an integral part of their daily lives (Bennett et al., 2008; Oblinger & Oblinger, 2005). Children and young people rely greatly on digital technology for retrieving information and interacting with others (Prensky, 2001a, b; Tapscott, 1999). Thus, researching the complex patterns of digital socialisation during leisure time has become increasingly important in today’s technologically interconnected society. Especially for children and young people, there is a clear need to thoroughly understand the patterns of digital socialisation during leisure time and explore the factors that shape them. The focus of this chapter is therefore to present a comprehensive research design that considers the role of multimodality in researching digital socialisation during leisure, as well as the agency of the research participants as co-researchers. By highlighting specific examples of combined elaborations from different means of data collection, such as interviews with children, communication diaries, and game observations, this chapter contributes to a multimodal and multi-method approach to the question of children’s digital leisure time and the use of digital technology.

Multimodality is employed as a means of perceiving the transformational and interactive processes in children and young people’s digital lives as it can provide a more nuanced understanding of how children create, interpret, and navigate digital content, as well as how they construct their identities and social relationships in digital environments. It allows us to examine the interplay between different modes and how they contribute to shaping the meaning and significance of children’s digital experiences. Multimodality has been developed over the years to systematically address much-debated questions about changes in society, for instance, in relation to new media and technologies (Kress & Selander, 2012). It offers a great contribution to research methods, specifically for researching digital habits and attitudes and collecting and analysing digital data and environments within social research (Jewitt, 2013). It also provides tools for mapping and analysing the visual, embodied, and spatial features of interaction with digital technology, as well as the analysis of games, music, and other new media (Burn, 2009, 2016; Snee et al., 2016; Caple & Knox, 2015; Jones, 2013; Adami, 2009; Knox, 2007). This study contributes to the ongoing discussion on the topic, advancing knowledge in the understanding and interpretation of leisure patterns. Additionally, it initiates the development of advanced digital tools for data collection, description, organisation, and analysis, further enhancing research in this field. More specifically, a state-of-the-art methodology of integrating and interpreting information across a wide range of modalities is proposed, namely a platform for semantic data integration, which allows for a unified analysis of multimodal data, thereby fostering a deeper and more comprehensive understanding of the data collected. Semantic data integration involves the systematic analysis and synthesis of different types of data to uncover meaningful patterns and relationships. To facilitate semantic data integration of the collected data an ontology of children and young people’s digital leisure time has been developed. In the context of semantic data integration, ontology is a formal, explicit specification of a shared conceptualisation of a domain. Therefore, developing an ontology is a significant step towards studying and analysing multimodal data in a structured and systematic manner. Despite the recognition of the importance of multimodal research, there are no studies to our knowledge that employ semantic data integration techniques to explore the nuanced patterns of digital socialisation during leisure among children and young people. Some methods of formal knowledge description have been proposed to model different aspects of adolescents’ life that mainly relate to health issues. In Tacyildiz et al. (2018), a mobile health application aimed at reducing obesity in children and adolescents is introduced. To achieve this goal, an ontology was developed to simulate obesity-related disorders and symptoms. Similarly, Jung et al. (2016) present an ontology for adolescents’ depression, which is utilised to annotate diverse data sources such as social media posts, counselling records, and narratives. Likewise, Jung et al. (2017) propose an ontology for modelling adolescent depression, providing a semantic basis for analysing social media data related to this phenomenon.

In the present chapter, the utilised modes of data collection and tools are subjected to a summative evaluation, providing recommendations for methods that can be employed in researching patterns of digital socialisation having children and young people play an active role throughout the process. More specifically, a combination of methods is presented and assessed to address the need for a more holistic approach to complex issues, such as leisure, socialisation, and the interpretations that children and young people attribute to these issues while involving them not only as participants but also as co-researchers. According to Kleine et al. (2016), participatory methods are increasingly being used in research that focuses on the online behaviour of children, indicating a growing trend in the field.

The study was carried out as a part of DigiGen, a European research project funded by Horizon2020, which focused on investigating the effects of technological transformations on the generation that has grown up with digital technology. DigiGen had four main focus areas, one of which is leisure. Within this domain, the primary objective was to determine how young people engage in social activities during their free time.

Methods, Measures, Ethical Considerations, and Limitations

The study adopts a methodological approach consistent with the traditions of the social sciences. However, it goes beyond the conventional norms of knowledge production, pushing the boundaries of academic conventions. When it comes to children and young people, we might encounter situations where the traditional methodological frameworks of the social sciences seem inadequate in capturing human action and behaviour (Summanen & Uski, 2015). Our research was conducted at a time when the field recognises that flexibility and the adept combination of previously unrelated methodologies and approaches are broadly acknowledged as the new norm in comprehending complex and ever-changing phenomena (Tiidenberg & Allaste, 2015), especially for those presented by technological transformations. However, it is also crucial to note that the drastic measures imposed due to the COVID-19 pandemic in almost all European countries created an unprecedented situation regarding the use of digital means in many aspects of everyday life. The issue of children’s leisure time, particularly in times of massive mobility restrictions and lockdowns of educational institutions, has been closely linked to the increased use of digital devices and applications (Kerekes et al., 2021). This study unintentionally but also inevitably investigated the impact of coronavirus-linked restrictions on children’s use of digital devices and applications and tried to capture the generated repercussions in terms of both extent and intensity.

Children and young people’s experiences and interactions with digital media are complex and multifaceted and cannot be fully understood using traditional research methods that focus solely on verbal or written responses. Thus, research inquiries aiming at conceptualising digital leisure can benefit from incorporating a multimodal research design. More specifically, in the present study fieldwork was carried out using semi-structured interviews, online participant observation conducted via ethnographic research during collective gaming sessions with children playing Minecraft, and digital communication diaries operated through a smartphone application developed specifically for this research with children aged between 9 and 15 years (Hyggen et al., 2020). The research was conducted in five countries, namely, Austria, Greece, Norway, Romania, and the UK, using common instruments for comparability purposes. The primary goal of the online communication diaries was to involve children and youth as active co-researchers in the research process, while the game observations went further to engage them as co-designers and co-creators of content for a potential new game. Under the United Nations Convention on the Rights of the Child (UN, 1989, Article 12), children have the right to express their views on all matters affecting them and to have those views given due weight. This right also applies in the context of research; however, examples of young children being engaged as co-researchers remain rare (Lundy et al., 2011). There is also a lack of participatory methods used to research young people’s digital habits. Moreover, a pressing need exists for novel approaches that enable a deeper comprehension of young people’s lives from their perspective. The idea of involving young people as co-researchers in the implementation stemmed primarily from an increasing body of research indicating that adopting a participative research approach improves the quality and significance of the findings (Smith et al., 2002). Examples of children’s innate curiosity, inclination to explore, and creativity have led Kellett (2005) to assert that they can be trained to develop the skills required to act as researchers. This is especially true in research covering habits, preferences, skills, and competencies of the digital generation.

The need to provide an integrated framework for combining diverse types of produced data was covered through semantic data integration, a process of interrelating information from diverse sources and consolidating it into meaningful and valuable information. An online tool was developed specifically for this task. In this manner, disparate data of different forms were classified and used by researchers coherently and comparably. As per the GDPR requirements of safeguarding sensitive data and ethical restrictions, all raw data related to it was stored on the Service for Sensitive Data (TSD 2.0)Footnote 1 at OsloMet University.

Ethical issues were particularly important as the research involved children and young people under the age of 18 as subjects as well as co-researchers and active participants. These issues concerned the procedures followed for the protection of personal data, the details on the material—including personal data—which were imported to/exported from/to the EU (e.g. uploading and downloading data to the TSD server), and information concerning the participants (e.g. recruitment procedures, signed informed assent-consent forms, procedures for incidental findings handling, procedures to ensure that the participants’ rights were safeguarded, etc.). Supplementary issues that had to be addressed during the implementation of the fieldwork include, for example, online recordings, online signing of informed consents, as well as issues concerning power and pressure and the importance of the children remaining at the centre of attention as active agents. Therefore, the challenge has been to develop research strategies, especially in conducting participatory research, that was fair, appropriate, and respectful to our research participants, paying full attention to the ways of analysing and interpreting data, disseminating findings, and protecting the research participants. Extreme precautions were taken during the conduct of the research, where children were actively involved as co-researchers to ensure that the children felt no stress to participate, were not worried, confused, misled by the research and/or patronised and that the participation was experienced as meaningful (Spriggs & Gillam, 2019).

A Multimodal Research Design

Technology and, in particular, digital media, have changed communication patterns and access to information in many ways. It is easier for children and young people today to find factual information but also fake news, elaborate their views on issues and be in constant connection with each other using digital media for small talk (chatting). In their leisure time, children and young people are also able to produce their own created content, such as photos, videos, and music, as well as films and reportages, and share these with the world via social platforms such as Snapchat, YouTube, and Instagram (McRoberts et al., 2019; Lenhart, 2015; Chau, 2010). These changes require new ways of thinking in terms of research and data collection. To this end, we have used various methods of data collection, some of which are participatory, and then semantically integrated them, providing an easy-to-use organisation and taxonomy of information of different forms, such as discourse and visual content.

Using multimodal research methods to collect data from various sources, such as videos, images, and other visual products, and analysing them to gain insights into children’s digital experiences, is particularly important because it allows researchers to capture the nuances of children’s digital lives, including their interactions with various forms of media, their use of technology to communicate and express themselves and how they create and consume digital content, rendering the information less biased by the researchers’ preconceptions and predefined categories.

Semi-Structured Interviews

Semi-structured interviews based on a common interview protocol developed collectively by all participating countries were conducted during the research. The main aim of the interviews was to investigate to what extent and in what ways everyday practices linked to leisure time are affected and/or transformed through digital technology usage. The age of the interviewees varied between 9 and 15 years.

The questions included in the interview protocol were related to the first digital divide (e.g., computer access, Internet access, digital device possession), communication patterns (e.g., means of communication, time spent online for communicating with friends), gaming practice (e.g., type of games, devices used, one-player gaming activity, collaborative online gaming activity), the intervention of virtual and physical space, negotiations amongst family members, and the impact of the coronavirus crisis. A total of 84 interviews were carried out across the five countries involved in the study.

The multiple COVID-19 restrictions imposed at the time of fieldwork and the unwillingness of parents and children to meet physically resulted in the semi-structured interviews being conducted online. Consequently, and in terms of sampling procedures, the children who were approached inevitably had relatively homogeneous characteristics. A section addressing specifically the pandemic’s impact on children’s digital leisure was also included in the interview protocol. In general, the children tended to provide short or one-word answers (yes, no, or maybe), particularly the youngest ones, during the interviews. This may be attributed to the fact that the questions were sometimes straightforward and did not encourage developed or enriched answers. In addition, the means of communication, using video conferencing tools, might have played a role as it did not provide much room for flexibility or detailed discussion. Moreover, in some cases, the children opted to turn off their cameras during the interviews, which prevented the researchers from observing their facial expressions. An additional challenge was the fact that many interviews, particularly with the youngest children, were conducted with parents taking part in the conversation/interview. This may have contributed to greater feelings of safety for the children (and their parents) but may also have reduced their willingness to speak freely about their digital everyday lives.

King’s (2004) and Miles and Huberman’s (1994) works provide valuable insights into the process of developing and applying coding techniques in template analysis, aiding in a deeper understanding of this methodology. In his recommendations, King (2004) suggested a halfway approach as one of the three possible starting positions, which involves having some initial coding (potentially from the interview protocol) and then refining it further through exploration of the data (Waring & Wainwright, 2008; King, 2004). In this study, the transcripts were analysed by developing an interview summary template that included the following thematic categories: digital capital, communication, gaming, virtual/physical space, negotiations with family, and the pandemic. Template analysis is a flexible data analytical technique ‘with fewer specified procedures, permitting researchers to tailor it to match their requirements’ (King, 2012, p. 428). The research conducted in this study presents a novel approach, as the thematic categories and coding were shaped to some extent by the interview protocol but were refined through the utilisation of artificial intelligence (AI) algorithmic procedure that identified significant terms and concepts from an extensive corpus of scientific data. After the identification of these concepts, an online open-source platform was developed to assist in data annotation, organisation, and analysis, namely, KGNotes.

Online Diaries, Snapshots, and Mini Surveys: Children as Co-Researchers

Experience Sampling Method (ESM) (Larson & Csikszentmihalyi, 1983) was one of the methods employed to explore children and young people’s everyday digital activities. With ESM, data is collected through in situ self-reports for systematic in-context data collection provided by participants who are proactively triggered at various points throughout the day to provide data (van Berkel et al., 2017). More specifically, a tool used to involve children in the research was an online communication diary downloaded by our young co-researchers as an app to their mobile phones. The app ‘Nettskjema Bilde’ was developed specifically for this purpose (i.e., to involve children as co-researchers and experts). Apart from the data collected via online communication diaries to collect evidence on how, how often, and for what purposes adolescents use digital media in their everyday life, the app could facilitate two additional modes of data collection: flexible mini surveys focusing on specific topics and the collection of information on children’s modes of expression through the creation of visual and/or audio-visual content. The children were the ones who selected what content they delivered through the app. The application served as one of the approaches employed to establish robust participatory methodologies, allowing children and young people to actively engage as co-researchers in the study. It allowed explorative research in a field marked by a preponderance of quantitative research focusing mainly on negative factors and a lack of involvement of study participants in the design of research (Pérez-Sanagustín et al., 2017; Livingstone & Smith, 2014).

The main objective of the communication diaries was to gather information on the use of digital media by children and young people, and more specifically about which devices children and young people use in their everyday digital life, how often they use them, and for which purposes.

Questions in the mini-surveys were mainly related to background information, digital capital, and the impacts of COVID-19 restrictions on their everyday lives. Despite its positive potential as a means for involving children and young people as co-researchers and collecting multimodal data exploring the digital lives of children and young people, some important challenges emerged. Despite great efforts on behalf of the research team, getting young co-researchers to start using the app was harder than expected. One main reason for this was the relatively complex procedure needed to ensure parental consent for the youngest children. To adhere to the GDPR guidelines parents had to send an email to the research team for an access code and personal ID number as well as provide a signed consent form. One way to overcome this challenge was to physically meet with the co-researchers and their parents to provide instructions and access codes. An additional challenge was drop-out. During the fieldwork period, the amount of time children spent on screens substantially increased because of the pandemic. This circumstance might have contributed to their reluctance or unwillingness to participate or drop out from the study before completing the 10-day period.

The children and young people who participated were instructed to dedicate a few minutes every day for approximately 10 days to work on their reports. Daily reports included brief survey questions and the opportunity to upload images or screenshots containing examples of their digital activities with the opportunity to tag the images with descriptions such as ‘gaming’, ‘communication’, and ‘entertainment’, and give a brief written description of the activities. In total, 50 children and young people participated as co-researchers from Austria, Norway, and the UK contributing 273 diary entries. The average number of entries made by boys was six, while girls made an average of five entries. We found no systematic variations regarding the number of entries made and the age of our co-researchers. Indeed, the contribution of our co-researchers resulted in substantial volumes of data, screenshots, and images, concerning their daily digital activities. By examining these images, we were able to observe that children and young individuals engage with digital devices and platforms across various daily activities. They utilise these devices as tools, for entertainment, learning, creative expression, and even as a means of passing time. Digital devices are employed for purposes such as tracking physical activity, acting as a digital bookshelf, shopping, collaborating on homework with siblings, and watching other gamers for entertainment or educational purposes, among a range of other activities.

Video Game Observation

The methodology of the video game observation implemented in the study falls into the category of participant observation. According to DeWalt and DeWalt (2002), ‘the goal for the design of research using participant observation as a method is to develop a holistic understanding of the phenomena under study that is as objective and accurate as possible given the limitations of the method’ (p. 92). Moreover, DeWalt and DeWalt (2002) suggested that participant observation can be used as a way to increase the validity of the study, as observations may help the researcher have a better understanding of the context and phenomenon under study. Barendregt et al. (2006) conducted game observations to uncover issues related to usability and enjoyment in a computer game named ‘Milo and the Magical Stones’ (MediaMix, 2002). Similarly, Bekker et al. (2008) used observations of gaming sessions, having participants engage in 30-minute sessions playing a specific computer game. Bird and Edwards (2015) investigated the learning process of 27 children using various technologies through play and more specifically utilising observation techniques of children using digital technology. In a more recent study, Behnamnia et al. (2020) described how teachers observed children’s interactions with the screen, employing nine different series of game applications.

Our fieldwork research was conducted using Minecraft as a tool in which to observe children and young people in gaming sessions. The children recruited for the interviews and/or online communication diaries were also asked whether they were interested in participating in the gaming session where the researchers would be observing them as they played. The research entailed online participant observation during different sessions at different moments while the gamer was playing online with his/her friends. Moreover, the participants were asked to provide permission to the researcher to record parts or all sessions. This kind of observation method is valuable to researchers in various ways. It provides researchers with ways to investigate nonverbal expressions of feelings, determine who interacts with whom, and seize how participants communicate with each other. Participant observation allows researchers to examine definitions of terms that the participants use in interviews, understand the used terminology, observe the events that the informants may be unable or unwilling to share when participating in an interview, and detect the situations the informants have described in interviews, making them aware of the distortions or inaccuracies in the description provided by those informants. More precisely, the participant observation of gameplay produced primary raw data based on the actual verification of digital skills and competencies as they were performed by players or on actual perceptions and aspirations of children and adolescents regarding their ‘inhabitancy in digital spaces’ (Jacobs & Cooper, 2018; Booth, 2010), i.e. their active participation, interaction, and immersing in the virtual context, through the use of digital devices. In this sense, the participant observation of gameplay provided feedback by testing and measuring the selected hypotheses developed within the interviews and communication diaries. During game sessions, researchers also examined various aspects such as organisational modalities, recurrence, stability, and contingency within the gaming context. They also studied group dynamics, online identities, patterns of socialisation that emerged during gameplay, and negotiations that took place between the players and explored how gaming activities were influenced by these interactions.

It is important to note that challenges were experienced with regard to the gameplay sessions. Some of the children interviewed did not engage in playing Minecraft, and among those who did, not all were eager to participate in this part of the study. Moreover, parents showed restricted enthusiasm regarding their children’s involvement in the gaming sessions. In addition, the impact of the COVID-19 pandemic might have led to an overwhelming feeling of ‘digital obligations’, which consequently led to a kind of rejection or unwillingness on the part of the children to participate, which can be described as a feeling of digital fatigue or digital burnout. As a result of these factors and the shift of game observation research to an online format, the level of participation was lower than initially anticipated. In total, 22 children and young people in Austria, Greece, and Norway participated in the gaming sessions with the researchers. During these sessions, that had a strong experimental character, the players described their actions during gameplay and provided some additional—and specific to Minecraft—information about gaming. The type of data that can be collected in the gamers’ natural environment, e.g., real-time reactions, strategies, decision-making processes, and social interactions, along with the amount of rich and detailed data provided, proves that participant observation emerges as a valuable method for enhancing research focused on gaming.

Dealing with Different Data Types: Semantic Integration of Multimodal Data

Handling data from diverse forms and sources, including audio and text from interviews, communication diaries, game sessions, mini-surveys, and images, can be challenging. To help researchers cope with these challenges and make sense of the data, a unified storage and access system should be used. This system can assist in organising and analysing the data and allows for easier comparison and combination of information from all sources. The aim was to develop an online system that supported the semantic data integration coming from all these different forms and sources. In the context of semantic integration of data, ontologies are used to formally represent knowledge or concepts that exist within a specific domain. It provides a structured vocabulary of terms and their relationships, as well as rules for their usage, to support consistent interpretation and sharing of data across different applications and systems. Ontologies play a critical role in semantic data integration because they help establish a common understanding of the meaning of data elements, allowing data from different sources to be combined and queried in a coherent and meaningful way. The ability to easily import and harmonise heterogeneous data from multiple sources and interlink is essential for knowledge extraction from research data (Filandrianos et al., 2022; Kazani et al., 2023; Parsanoglou et al., 2022). The created ontology was coded in Web Ontology Language (OWL) and developed using the ProtégéFootnote 2 ontology editor. Domain experts (i.e. social scientists and computer engineers) developed the current ontology in a semi-automatic manner with the assistance of an automatic algorithm called KGExtraction. This algorithm was specifically designed to identify scientific terms within approximately 100 scientific papers that were identified in a scoping review, save their origins and categorise them into the final concepts of ontology. Consequently, instead of reading all relevant papers to extract the respective terms and then cluster them to define concepts, the domain experts were only involved in reviewing the extracted terms and concepts to remove redundant or irrelevant ones. This procedure was extremely beneficial in terms of time and accuracy. For more information regarding KGExtraction, the reader can consult Filandrianos et al. (2022) and Parsanoglou et al. (2022). A part of the constructed ontology is depicted in Fig. 1.

Fig. 1
A network diagram of the constructed ontology excerpt has several interconnected elements. They include doing activity, owl thing, and communicating. Each further connect to doing activity alone, and with strangers, gaming, and video gaming board, and communicating with stranger.

Excerpt of the constructed ontology

Full size image

An example is now provided to clarify the logic underlying the semantic integration of data. In our research exploring digital technologies and their impact on leisure one particular area of interest and one of the extracted terms using KGExtraction is Online communication investigating for instance with whom children communicate what digital devices they use for this purpose and so on. If the category of interest is Online communication one can identify subcategories such as Online communication with family (e.g. Online communication with parents, siblings, or other members of the family). The initial stage of organising the data semantically is to develop formal ontological descriptions to detect and determine the features of the domain we are attempting to explain. In studying online communications for instance we have extracted concepts (classes), instances, and connections amongst them. For example a subclass of Online communication with siblings is a subclass of Online communication with family which is in turn a subclass of the class Online communication. Figure 2 provides an excerpt of the numerous classes, subclasses instances, and interconnections acquired from the ontology of adolescents’ digital leisure developed within the current research

Fig. 2
A chart of online communications has several elements. It has 3 main elements, communication with family, friends, and with schoolmates with interview hashtag 8 G R, interview hashtag 1 G R, and game session hash tag 10 G R, pointing towards each in order.

Excerpt of classes, subclass instances, and relations from the ontology of adolescents’ digital leisure time. Note: We use grey for the class Online Communication, blue for its subclasses Communication with Family, Communication with Friends, and Communication with Schoolmates, and green for its subclasses. Black lines denote the relation subclass-of (SO) and red lines are the instance-of (IO)

Full size image

If one uses the developed ontology to ask a query relating to data concerning Online communication with family the following instances should be brought forward:

  • Transcripts from Interview #8 (‘We have a group on Discord with my sister to communicate, for instance, arrange to watch a series on Netflix, exchange photos, etc.’ and ‘Once a week, I communicate via Skype with my grandmother who lives in another city and with my cousins who live abroad, usually during the weekend’) (Greek data)

  • Mini Survey #2 (‘I communicate with my parents every day through mobile messaging (SMS) and Viber’) (Greek data)

This way, researchers can identify and analyse all collected data that refer to this thematic focus (i.e. online communication with family) regardless of the form of the data.

To better utilise the advantages of such ordering and assist researchers in applying structured knowledge, we have created an open-source, user-friendly online platform named KGNotes, which can visualise the created ontology, load different types of files (text and videos), and annotate them using the knowledge concepts. The operation of this tool is based on the existence of a large, predefined ontology that has been developed in a semi-supervised way by the research team, and an algorithm to extract content from numerous scientific papers. In addition, for every different file type, the working environment of the tool (i.e. the tool’s interface and its elements such as menus, toolbars, and other visual and interactive components) automatically changes to be intuitively easier to use, and the annotations are stored in a unified way, regardless of the file type. Thus, KGNotes can search for instances of a query simultaneously for each file format. Figure 3 depicts the working interface of KGNotes for annotating text, whereas Fig. 4 depicts the one for annotating a video. Readers can consult Kazani et al. (2023), Parsanoglou et al. (2022), and Filandrianos et al. (2022) for a detailed description of KGNotes and the ontology describing adolescents’ digital leisure time that was constructed.

Fig. 3
2 screenshots of the K G Notes working interface for annotating text. The main screen has 2 segments. Left. An imported data list with several question-answer pairs. A few are highlighted and the window has a scroll bar. Right. An illustration of the ontology with various elements and sub elements.

Working interface of KGNotes for annotating text (English and Greek version). Note: The main screen of the tool is divided into two segments. On the left-hand side, the imported data are displayed, while the right-hand side depicts the ontology. Each ontology concept is clickable, and the user can expand it to study its subclasses. If the text is coloured (left-hand side) this means that these specific parts of the imported data have already been annotated by other users. The highlighted text for example here refers to the following phrases: ‘My mom doesn’t let me have a smartphone’, ‘All my friends talk on Discord’, ‘I spent 2-2.30 hours every day on the computer’, and ‘I have used a tablet since I was a baby, since I’ve learnt how to talk’

Full size image
Fig. 4
A screenshot of the working interface of K G Notes for annotating videos. The main has 2 segments. Left. A video still with 2 women seated, 5 editing options, a scrollbar enabled window. Right. An illustration of the ontology with various elements and sub elements.

The operational interface of KGNotes for video annotation. The primary screen of the tool is divided into two sections. On the right-hand side, the ontology is presented, with clickable ontology concepts that users can expand to explore their subclasses. On the left-hand side, the video annotation process is showcased

Full size image

The Greek data imported to KGNotes was used as a trial run to evaluate the suitability and usefulness of the method, as well as provide suggestions and recommendations for further improvement.

Possibilities, Challenges, and Conclusions

When examining the methodologies employed in this study, it becomes apparent that interview settings are relatively static when compared with the dynamic nature of conversations that occur, particularly during gameplay. Children tend to be more talkative during game observation sessions than in interviews. Through game observations, the researcher maintained a more discreet role than in interviews, and the observations themselves served as small focus group discussions. Apart from facilitating the observation of children interacting with each other, such as collaborating, dividing responsibilities, and discussing tactics, game observations also empowered children to take the lead, which was not the case in interviews due to the different dynamics. To put it differently, children appeared to feel valued as experts in game sessions and demonstrated greater enthusiasm in sharing their insights and perspectives. However, despite initial expectations, recruiting and retaining the engagement of children and young people as co-researchers through the app proved to be a more challenging task. Apart from the difficulties posed by the pandemic, it should not be assumed that a smartphone application is inherently captivating, appealing, or stimulating for children and young people who are highly accustomed to digital technology. This can clarify why, despite gentle reminders from parents, children and young people in the participating countries chose not to use the application, as they perceived it as an additional ‘digital task’ rather than a beneficial means of expressing themselves. Making children and young individuals co-researchers necessitates more participatory approaches that involve them in every stage of the research process, from designing and formulating research questions to implementing the research itself. This can entail engaging them as co-designers of the research questions and co-creators of content. Although our efforts to involve children as co-researchers were not entirely successful, we have managed to assess and highlight their perspectives. As such, this chapter holds value and practical implications for future research endeavours and policymaking.

The use of multiple modes of data collection can play a significant role in promoting the agency of children in research. Communication diaries provided children with a space to document their thoughts and feelings, enabling them to contribute their unique perspectives to the research process. Observing children playing games fostered an immersive atmosphere that motivated them to take an active part and exchange their experiences using their terminology and displaying their level of expertise. They were also able to propose suggestions for improvements and describe their ideal games. By incorporating these multimodal design elements, researchers can enhance the agency of children as research contributors and create more meaningful and impactful research that reflects the needs and experiences of children.

Furthermore, in the current chapter, we have proposed an ontology to represent the digital leisure activities of children and adolescents. This ontology serves the purpose of facilitating the semantic integration of the collected multimodal data. To construct this ontology, we have developed an artificially intelligent algorithm named KGExtraction, which automatically extracts relevant concepts from scientific papers closely related to the topic. By scanning scientific papers, it identifies domain-specific terms and compiles a comprehensive collection of non-redundant terms. The resulting ontology is then utilised to annotate various types of collected data, to capture patterns of digital socialisation among children and young people. To streamline the annotation process and facilitate data analysis, we have created an open-source online platform called KGNotes, which is a user-friendly tool that visualises the created ontology, loads different file types, and enables annotation using concepts from the knowledge base. By employing this comprehensive approach, utilising the KGExtraction algorithm, developing the ontology, and using the KGNotes platform for the analysis, we can unlock valuable insights and delve into the intricate dynamics of digital leisure and socialisation patterns among children and young people, ultimately enhancing our understanding of their digital experiences in a rapidly evolving technological landscape.