1 Introduction

Learning takes place in multiple contexts and valued practices of everyday lives across the life span [6]. Indeed, we can define different forms of learning, ranging from formal, non-formal and informal [38]. Formal learning includes those forms of education, which are normally delivered by trained teachers in a systematic and intentional way within a school, higher education or university [78]. Non-formal learning is referred to those structured learning situations which do not exploit typical elements of formal learning, such as curriculum, accreditation, or certification, but still it is recognized as more planned and organized than informal learning [74, 81]. In fact, informal learning is characterized for taking place outside educational institutions, arising from the learner’s involvement in activities that are not undertaken with a learning purpose in mind. Hence, it is considered as involuntary, naturally, and spontaneously, deriving from daily life activities [64]. While formal and non-formal learning are partially intentional and partially incidental, informal learning is totally incidental [49]. On the one side, intentional learning occurs when the learners consciously conduct and perform activities with an educational purpose, and it is a limited fraction in people life span [7]. On the other side, incidental learning is “unintentional or unplanned” as defined in [33], often referred as a synonymous of informal learning [69], and it represents the way in which learning mostly occur in a person’s lifetime [18].

In this context, the availability and the diffusion of mobile and wearable devices, together with network connectivity, have surely multiplied tools, moments, and chances to incidentally learn in a mobile and contextual way [26, 31, 38], in particular when they are applied to language learning [28]. Such a topic has been investigated from different points of view: from how Mobile Assisted Language Learning (MALL) can be effective in different learning activities [24], to how social network and instant messaging mobile apps can provide benefits in acquiring and improving language skills [10, 82].

In this paper, drawing on the results of the ILOCALAPP international Project [30], we reflect upon the efficacy of incidental learning, through a mobile application (the UniOn! app, [75]), which was developed and put to good use to support mobility students in daily life activities, while incidentally learning the Italian language and culture. In particular, the main goal behind the study presented in this manuscript is to answer at the following Research Question (as detailed in Section 2):

  • RQ: How effective is the incidental learning approach in improving a second language, when a mobile application is used as a supporting tool for the integration of mobility students?

In order to measure the efficacy of our approach, we have leveraged both -self-evaluation and self-assessment tools, by adopting a quasi-experimental design and involving 95 participants in 5 case studies, and a fine-tuned statistical analysis. In fact, a lack in the literature is related to testing and assessing the efficacy and effectiveness of incidental learning [71]: since it is naturally integrated with other activities and contextual elements, it is very difficult to isolate, identify, and measure its impact on learners’ skills improvements. Thus, one of the scientific outcomes of this paper is that of contributing to the intriguing discussion on how self-perception and self--interpretation sentiments about the improvements achieved by learners can be considered an interesting metric for a scientific assessment. The 5 different case studies have been chosen because they let us obtain and discuss results coming from different scenarios: involving different levels of knowledge of the goal language, coming from different countries, with different mother tongue languages, using the app for different periods, as detailed in Sections 4 and 5. The different characteristics of the case studies we have conducted let us focus on four distinct dimensions: target users, users’ language background, duration and frequency of use, context of use. We have analyzed the obtained results on the basis of such four dimensions, discussing and comparing the 5 case studies.

The rest of this paper is organized as follows. Section 2 presents some main background information on the incidental learning concept and the research question that has driven our study, as well as the context of this study: the ILOCALAPP project and the UniOn! app. Section 3 describes the methodology that we have applied to assess the incidental learning contribution in our case study, the UniOn! app with its theoretical underpinnings, and the questionnaire the participants filled in, together with the rationale behind its questions and answers. The assessment phase is illustrated in Section 4, which presents the five Case Studies. Section 5 reports the results that we have obtained, while the evidence by such results in response to our Research Questions is discussed in Section 6. Finally, Section 7 concludes the paper with final remarks and further work.

2 Background and research question

Incidental learning is by definition not intentional and not planned; as such, it may occur while pursuing other goals, or emerge while carrying out other tasks [25, 33].

Incidental learning is typically associated to childhood, whereas for adults it is embedded into the socioeconomic and cultural context [44], and it results from socialisation. Incidental learning is mediated by available means and tools, and by the preferred learning styles as well [36]. Moreover, incidental learning is based on prior knowledge and therefore it can be tackled by referring to Experiential Learning Theory [21]. This theory considers learning as the process enabling the creation of knowledge, which “results from the combination of grasping and transforming experience” [35]. When engaged in incidental learning, learners are in contact with factual information, and they interact with knowledge and experience [44].

The specific affordances offered by mobile devices are particularly suited to incidental learning, as they can provide a learning system with content that may prompt situation-specific learning activities [60]. In other words, mobile learning can help exploit daily experiences and activities for incidental learning, insomuch as learning opportunities can arise in any location and in any situation: it hinges upon the mobile device’s portability and affordability in order to allow learners to access computer-based learning anytime, anywhere [60, 72]. Mobile learning is situated, personalised, collaborative, ubiquitous and lifelong [11]. As a result, mobile learning provides opportunities for expanding traditional teaching methods, as well as informal and incidental learning.

If it is true that virtual and mobile technology fosters new kinds of learning, this is particularly relevant in the field of languages and cultures [68], for which it offers more opportunities to support learning and performance both inside and outside the classroom [48]. Research in Mobile Assisted Language Learning (MALL) has shown how mobile devices can be exploited to effectively adapt instructions to the physical environment of learners [8]. As Martin and Ertzberger [48] pointed out, situated learning requires knowledge to be presented in authentic contexts in order to be known and fully understood. The use of mobile devices for incidental learning relies in fact upon both place and personal space, as they are deployed to fit the users’ physical and temporal context and learning needs (Scanlon et al., 2014). MALL better suits bite-sized activities, sometimes also referred to as MLO (Mobile Learning Objects) and defined as “an information entity, digital, interactive, adaptable and reusable in different contexts, designed to support an educational objective through a mobile device” [12].

The learners’ needs represent a particular challenge when they are engaged in incidental situated learning as part of their everyday lives and specific software frameworks are required to enable developers to successfully identify suitable learning approaches and resources [40]. These specific frameworks are also beneficial in order to assess the users’ learning experience. Even if several reviews and studies about the usage of mobile technologies for language learning and about context-aware language learning [4] have been published, there is little factual evidence about the effectiveness of the usage of mobile devices for incidental learning.

How can we verify if incidental learning occurs? This is still a debated question, with no definite, validated methodology to be applied. Moreover, what is the users’ perception of learning when a mobile application is used as a tool to approach the experience of getting to live in a new reality? How can we assess improvements in skills and competences and how measurable are the results of such an approach? What kind of skills are particularly improved with such an approach? Is such an approach suitable for all language levels? Is such an approach useful for both language and cultural competence?

In this study, we have taken into account all these inquiries and we have formed the following Research Question (RQ):

  • RQ: How effective is the incidental learning approach in improving a second language, when a mobile application is used as a supporting tool for the integration of mobility students?

In the following sections, we present the methodology adopted in a specific case study, where a mobile app has been used as a tool to foster the incidental learning of Italian language and culture, and we discuss the evaluation campaigns that we have conducted with target users, with the aim of measuring the effectiveness of such an app.

2.1 Our context: the ILOCALAPP Project and the UniOn! App

UniOn! is a mobile application based on the principles of incidental learning and developed to cope with the specific needs of mobility students in terms of language acquisition and (inter) culture awareness [75]. The application offers the possibility to get acquainted with local language and culture in a smart, creative way [13]. UniOn! was developed within ILOCALAPP, a three-year (2015–2018) Erasmus+ project aimed at the production of a tool for the incidental learning of four cultures and languages: Finnish, Italian, Polish, and Portuguese [30]. The project was carried out by a transnational consortium (the Adam Mickiewicz University, Poznań, Poland; the University of Lapland, Rovaniemi, Finland; the Centre for Social Studies of the University of Coimbra, Portugal) coordinated by the University of Bologna in Italy. The Department of Computer Science and Engineering and the School of Languages and Literatures, Interpreting and Translation of the University of Bologna were in charge of the development of the application [13].

As student mobility requires innovative, creative, and at the same time simple, learning tools, the fact of presenting educational services and materials on mobile devices allows learning episodes to be incorporated into daily routines, and to be accessed at times and in places that suit learners best [25]. Moreover, the project teams decided to base the UniOn! app upon the features of incidental learning, thus conveying a quick comprehension of the cultural codes of the new place where the students happen to spend a study/training period, in a situated, location-based, and context-aware learning.

The UniOn! application developed by ILOCALAPP combines language and culture information, aiming at the provision of a complex system of cultural customs, values, and ways of thinking, feeling, and acting [7]. Moreover, UniOn! fosters the development of intercultural communicative competence as far as it enhances understanding and interaction [53]. The app and its content have been designed with beginners and intermediate students as target users in mind, with the intention of letting them exploit it while they are actually living in the hosting country, experiencing the language and the culture in their daily activities. It is worth mentioning that the design and development of the app is not the main goal of this manuscript, which focuses on the investigation done to answer to the Research Question (RQ) reported in the Introduction and in this Section of this paper. More details about the UniOn! application can be found on previous publications [13, 50, 77], including detailed description of how the app helps with incidental learning [17, 76].

3 Methodology

This section is devoted to outline and motivate the methodology that we have adopted. During the development phase of the UniOn! app, various activities with users were carried out, applying the user-centred design and the participatory methodology (for more details, see [50, 51, 76]. In particular, we involved target users collecting comments on the app interface and the interaction flow, on the provided content, on the app services and interactive functionalities. Target users were engaged in (i) focus groups and experience prototyping activities at the four project partners during the design phase, and in (ii) prototype tests and evaluations during the development phase. All these activities let us gather important and useful feedback, which we exploited in finishing and refining the mobile app.

After the development of UniOn!, we focused our attention on investigating how to assess and evaluate the effectiveness of the incidental learning approach, beyond their appreciation of the app. The assessment of such an approach is a difficult task, because several elements could influence the learning outcomes, making complex to isolate and identify the actual contribution of the input provided by the app itself [72]. Examples of these elements are [8]:

  • Learner’s motivation.

  • Duration and frequency of use during the day.

  • Learner’s daily activities (including jobs, courses, leisure, hobbies, sports).

  • Relationships with local people.

  • Guided or autonomous context of use.

Even though the formal testing is the most common way to measure achievements and proficiency in formal language learning [71], self-evaluation and self-assessment can be applied as non-traditional ways of judging students’ performance in non-formal and informal learning situations, characterized by incidental ways of learning [20, 46]. With a focus upon the user’s perception of learning, we can investigate the personal feeling of the users about a set of given assertions related to the progress in language proficiency, the knowledge of cultural aspects, and the integration within local reality. Hence, in this perspective, self-evaluation and self-assessment become indeed key factors in answering our Research Question.

It is also worth mentioning that, in such a context, the use of a control group was not considered feasible, as stated in [70], due to both the small size of the target users’ groups involved in the app testing phases and ethical considerations. In fact, control groups are still debated in learning situations: several studies consider such a methodology as unfair in language learning contexts, posing the question “Is it ethical to force the control group students to endure a substandard teaching methodology, compared to that offered to fellow students?” [57, 65].

For these reasons, we decided to adopt a methodology based on self-assessment surveys with the aim of evaluating the users’ awareness about the improvements they achieved thanks to the use of the mobile app. The questionnaire had been defined during the ILOCALAPP meetings, involving all the partners in structuring the questions and the related answers, based on previous studies [27, 73] and of standardized questionnaires for users experience evaluation [22, 62] with the aim of getting feedback on different items, ranging from the usability of the mobile app user interface to the language skills improvements. The whole questionnaire is structured in four main sections, focusing on the following points of attention [76]:

  1. (1)

    An overall assessment of the app concerning several aspects (topics, texts, lexicon, usability, comprehensibility, usefulness, effectiveness). The users were invited to express their agreement with 15 statements using a Likert scale running from 1 (totally disagree) to 5 (totally agree). This section of the questionnaire is mainly devoted to collecting feedback from the participants about their whole experience while using the mobile app. This section questions/statements have been defined taking into account standardized questionnaires for users experience evaluation, such as the User Experience Questionnaire [63]. Although they are compelling, the results obtained by analyzing this section of the questionnaire are not at the basis of the study presented and discussed in this paper.

  2. (2)

    The user’s own perception of learning, with a focus on the perceived change in language and culture competence due to the use of the app: again, the users were invited to use a 1–5 Likert scale running from 1 (totally disagree) to 5 (totally agree). Moreover, we included a non-mandatory open-ended question to collect users’ feedback. The rest of this paper is focused on the description of the case studies that we have exploited and on the results that we have obtained by analyzing the participants’ answers to this section questions/statements. Indeed, this section questions have been inspired by self-regulated questionnaires and strategies [54, 66]. Details about this section statements and the related feedback collected by the participants are presented and detailed in the following sections of this paper. The main statements on which we have based our studies are the following ones:

    1. a.

      I feel more confident when I read Italian texts after using the UniOn! app

    2. b.

      I feel more confident when I listen to Italian after using the UniOn! app

    3. c.

      I feel more confident when I speak Italian after using the UniOn! app

    4. d.

      I feel more confident when I write in Italian after using the UniOn! app

    5. e.

      I am more familiar with Italian culture

    6. f.

      I understand better the local environment.

    7. g.

      The app is useful to learn the language intuitively.

    8. h.

      The app was useful during my stay.

  3. (3)

    A reflection about some specific features (in particular, the “add notes” as a possibility to include the user’s own experience within the app). This section of the questionnaire is composed of some open questions related to these topics (and the analysis of the results is out of the scope of the study presented in this paper).

  4. (4)

    Some personal information to better define the profile of app users.

In particular, we have taken into account the dimensions reported in Table 1.

Table 1 Dimensions of investigation
Full size table

All this said, in a quasi-experimental design [2, 29], we have involved 95 students, who voluntarily participated and who corresponded to the target users of the UniOn! app: foreign students (attending master degrees or summer schools at the University of Bologna), coming from different countries, enrolled in different degrees (in terms of levels and in terms of field of studies), and with different Italian language starting level skills. The participants were grouped in 5 different case studies (according to the attended classes in Bologna), differing each other on be basis of the dimensions of investigation mentioned in Table 1.

During participants’ staying in Bologna, they were asked to participate in this study: once accepted, they have been invited to download and install the UniOn! app on their smartphone. At the end of their stay, they were asked to fill-in (on their own, without the support of any moderator or any other participant) a survey about their experience while using the app in Bologna. Due to the nature of the incidental learning and due to the lack of control during the use of the mobile app, it may not be possible to convincingly demonstrate a causal link between the treatment condition (use of the app) and observed outcomes (language skills and culture knowledge improvements).

The following SubSection is devoted to present the main characteristics of the UniOn! app at the basis of our investigation.

3.1 The UniOn! App: Our investigation

Our investigation has been conducted on the Italian language and culture version of the UniOn! app, taking place in Bologna. Figure 1 shows two screenshots taken from such a version. UniOn-Bologna contains 112 texts, organized in thematic categories and sub-categories. For each text, a colored flag indicates the language level for which it is more suitable (as shown in Fig. 1, on the right): green texts are indicated for beginners, yellow texts for intermediate and red texts for advanced users. Moreover, from each content it is also possible to switch to its English version.

Fig. 1
figure 1

Screenshots taken from UniOn! Bologna: the home page and a sub-category page

Full size image

This functionality was considered only as optional in the authors’ mind, who wanted to encourage the usage of Italian as the default language as much as possible in the perspective of enhancing the language incidental learning. In any case, UniOn-Bologna was meant to be used by all language levels and the colored flags were considered useful in order to let the users realize the expected level of difficulty. In addition, the specific items in the texts link to i) Lexicon sheets (explanatory notes for the vocabulary included) and ii) Talk sections (conversational examples with prototypical questions and answers) (Fig. 2). Each text also contains an image, and some of them also include audio, videos and links to external resources. In total, 105 places are geo-localized in UniOn-Bologna; the contents can be accessed browsing the categories, using the search function or following the push notifications related to geo-localized places when the user is in the near-by.

Fig. 2
figure 2

Screenshots of the Talk and Lexicon sections of UniOn-Bologna

Full size image

4 Assessment

In this Section, we present the results of five assessments (as case studies) involving a total of 95 users, who incidentally improve their Italian language skills, supported by the app. Each case study differed as far as the dimensions of investigation mentioned in the methodological table above are concerned, and these differences impacted upon the answer to our Research Question. It is worth mentioning that the participants have been involved in the case studies on a voluntary basis, with a voluntary sampling method. Table 2 summarizes the main characteristics of the participants involved in the 5 Case Studies and it anticipates the main obtained results, as reported in details in the following subsections and as described in the following Results Section.

Table 2 Methodological dimensions investigated (upper part) and perceived improvements in the four skills (lower part), for the five case studies
Full size table

4.1 Case study 1

As for dimensions one and two, i.e., users’ characteristics and language background, the first case study involved seven US American students participating in the “We Tell” International Summer School in Digital Storytelling, which took place in Bologna in June 2018. The students were all about 20 years old, they were attending media and journalism courses at their home institutions, and they were all beginners in the Italian language. As for dimensions three and four, i.e., duration and context of use, the students used UniOn! for about three weeks as an integration to regular Italian classes, which were included in the Summer School as an optional activity. At the beginning, the students were given a formal presentation about the app, its features and its functionalities, and they were supported by a tutor throughout the initial app usage.

The Case Study 1 participants’ perception of learning is reported in Fig. 3, based on the answers provided to questions of Section 2 of our survey, and it concerns the perceived improvement in the four language skills (reading, listening, writing, and speaking) after using the UniOn! app. In particular, 43% of the users involved agreed or strongly agreed with the sentence “I feel more confident when I read Italian texts after using the UniOn! app”, while 14% of them disagreed or strongly disagreed. Listening abilities were perceived as improved thanks to the use of UniOn! by 29% of Case Study 1 students, while 14% of them disagreed or strongly disagreed with the sentence “I feel more confident when I listen to Italian after using the UniOn! app”. As for oral skills, 14% of students claimed that the use of the app let them improve their speaking abilities, but the same percentage of respondents disagreed with the sentence “I feel more confident when I speak Italian after using the UniOn! app”. Finally, 14% of students claimed that the app let them improve their writing skills, while 29% disagreed.

Fig. 3
figure 3

Perceived Learning improvements, in terms of speaking, writing, listening, and reading skills – Case Study 1

Full size image

All in all, Case Study 1 participants showed a limited perception of improvement, in particular as for their productive skills, whereas the perceived improvement of receptive skills obtained better results. In contrast, this Case Study students perceived a deeper improvement of their understanding of Italian culture and of the local environment after using the app. In particular, 57% of them agreed with the sentence “I am more familiar with Italian culture” and 71% agreed with the sentence “I understand better the local environment”, after using the UniOn! App (as reported in Figs. 4 and 5). Finally, the students also reported a positive consideration of the UniOn! app, according to their answers to some of the questions in Section 1 of our survey. In particular, 43% respondents of Case Study 1 agreed or strongly agreed with the sentences “The app is useful to learn the language intuitively” and “The app was useful during my stay”.

Fig. 4
figure 4

Perceived Learning improvements, in terms of familiarity with Italian culture, for all the case studies

Full size image
Fig. 5
figure 5

Perceived Learning improvements, in terms of understanding of the local environment, for all the case studies

Full size image

4.2 Case Study 2

The second case study involved eight students attending the Italian Language and Culture Summer School 2018. The students came from different parts of the world, they had different ages and different positions but they were all beginners (dimensions one and two of Table 1). This second testing session lasted three weeks as the first one, however the context of use was different as the students were invited to use the app autonomously after attending a formal presentation about its features and functionalities, and its possibilities of usage as well (dimensions three and four of Table 1).

The results about Case Study 2 perceived learning improvements are depicted in Fig. 6. As it was for Case Study 1, also students of Case Study 2 reported a limited perception of improvement, and they showed more confidence with receptive skills rather than with productive ones, even if their percentage of appreciation is slightly lower if compared with Case Study 1. In particular, 38% of the involved students agreed or strongly agreed with the statement “I feel more confident when I read Italian texts after using the UniOn! app” (25% of them disagreed or strongly disagreed), while 25% claimed that they felt more confident with listening skills (and 25% disagreed/strongly disagreed). Finally, 13% of students claimed that the use of the app let them improve their speaking abilities, while 25% disagreed, and 13% of respondents also claimed that their writing skills were improved (38% disagreed, though).

Fig. 6
figure 6

Perceived Learning improvements, in terms of speaking, writing, listening, and reading skills – Case Study 2

Full size image

In contrast, again as it was for Case Study 1, Case Study 2 students also showed interesting results in terms of perceived improvements in their knowledge and understanding of the Italian culture and of the local environment. In particular, 63% of them agreed or strongly agreed with the statement “I am more familiar with Italian culture”, while 75% of them agreed/strongly agreed with “I understand better the local environment”, after using the UniOn! App (as reported in Figs. 4 and 5).

Finally, in their answers to the questions of Section 1 of the survey, also the students of Case Study 2 expressed an overall satisfaction with the usage of the UniOn! app. In particular, 50% of Case Study 2 agreed or strongly agreed with the sentence “The app is useful to learn the language intuitively” (while 38% of them agreed with “The app was useful during my stay”).

4.3 Case Study 3

The third Case Study involved a group of 40 international incoming students of the University of Bologna, with various provenience and different background, and also with diverse levels of language competence varied, even though the majority declared to be intermediate (dimensions one and two of Table 1). The students of Case Study 3 used the app for three months, voluntarily and in an autonomous context (dimensions three and four of Table 1).

Figure 7 reports the students’ perception about the improvement of their skills. As for reading, 48% of respondents agreed or strongly agreed with the sentence “I feel more confident when I read Italian texts after using the UniOn! app” (15% of them disagreed or strongly disagreed), while listening abilities were perceived as improved by 45% of Case Study 3 students (whereas 20% of them disagreed/strongly disagreed with the sentence “I feel more confident when I listen to Italian after using the UniOn! app”). As for writing 38% of Case Study 3 students agreed or strongly agreed with the statement “I feel more confident when I write in Italian after using the UniOn! app” (and 23% of them disagreed/strongly disagreed with it), finally, 40% of Case Study 3 students claimed that they felt more confident with their speaking skills after using UniOn! (while 18% of them disagreed/strongly disagreed).

Fig. 7
figure 7

Perceived Learning improvements, in terms of speaking, writing, listening, and reading skills – Case Study 3

Full size image

Generally, this Case Study participants reported better results in terms of self--perception of language skills improvement, confirming a higher trend for receptive skills in comparison with the productive ones. Case Study 3 students reported satisfying results also in terms of perceived improvements of their knowledge and understanding of the Italian culture and of the local environment. In particular, 68% of them agreed or strongly agreed with the statement “I am more familiar with Italian culture”, and 73% of them agreed/strongly agreed with “I understand better the local environment”, after using UniOn! app.

Again, also in this case, the students reported a general appreciation of the UniOn! app: 60% of Case Study 3 students agreed or strongly agreed with the sentence “The app is useful to learn the language intuitively” and 65% of them agreed/strongly agreed with the statement “The app was useful during my stay”. Moreover, 55% of these students affirmed that they would recommend the UniOn! app to other people.

4.4 Case Study 4

This forth Case Study involved ten US American students attending the Bologna Dickinson College for a.y. 2018/2019. They had diverse fields of studies, but they were all in their twenties and they all declared to have an intermediate level of Italian (dimensions one and two of Table 1). These students tested the app for 2 months, and they were supported by their Italian teacher, who arranged dedicated activities (dimensions three and four). In particular, Case Study 4 students were given thematic itineraries to explore through the UniOn! app (libraries; students’ meeting places; students’ eating places; music; parks) and they had to find specific places included in each itinerary, read the information about them in the app, then visit the places. Moreover, they had to answer some questions and write a short report about the places. Figure 8 shows a picture taken during one of the “students’ eating places” itinerary.

Fig. 8
figure 8

Pictures taken during a thematic itinerary (participants discussing about iii - students’ eating places) and during the gamified activity using the app

Full size image

Figure 9 reports the Case Study 4 students’ perception about the improvement of their skills (reading, listening, writing, and speaking) after using the UniOn! app. As for reading, 60% of the users involved agreed or strongly agreed with the sentence “I feel more confident when I read Italian texts after using the UniOn! app”, while 10% of them disagreed or strongly disagreed. Listening abilities were perceived as improved thanks to the use of UniOn! by 50% respondents of Case Study 4, while 20% of them disagreed or strongly disagreed with the sentence “I feel more confident when I listen to Italian after using the UniOn! app”. Speaking skills were perceived as improved by 40% of Case Study 4 students, while 30% of them disagreed/strongly disagreed with the sentence “I feel more confident when I speak Italian after using the UniOn! app”. Finally, 50% of Case Study 4 students had the perception of improved writing abilities after using the app, while 10% disagreed/strongly disagreed with the sentence “I feel more confident when I write in Italian after using the UniOn! app”.

Fig. 9
figure 9

Perceived Learning improvements, in terms of speaking, writing, listening and reading skills – Case Study 4

Full size image

Students of Case Study 4 showed extremely positive results in terms of perceived improvements of knowledge and understanding of Italian culture and local environment. In particular, 70% of them agreed or strongly agreed with the statement “I am more familiar with Italian culture”, while 80% agreed/strongly agreed with “I understand better the local environment”, after using UniOn! app (see Figs. 4 and 5).

Once more, the students involved in the testing session appreciated the UniOn! app. In particular, 60% of Case Study 4 students agreed or strongly agreed with the sentence “The app is useful to learn the language intuitively” and 70% agreed/strongly agreed with the statement “The app was useful during my stay”. Finally, 60% of these students claimed that they would recommend the UniOn! app to other people.

4.5 Case study 5

The fifth Case Study was composed by 30 students enrolled in the CLE (European Literatures) Erasmus Mundus Master Degree at the University of Bologna. They were in their twenties, came from all over the world and had an intermediate level of language proficiency (dimensions one and two of the methodological table). Students of Case Study 5 used the app for three months in a semi-guided context (dimensions three and four). To get started with the activity, they were offered both a formal presentation and a gamified activity (a treasure hunt with a photographic contest). Figure 8 shows a picture taken by one of the teams during such a gamified activity. After the presentation and the guided usage of the app, the students were invited to keep using the application during their daily routine in Bologna. Three months later, the 30 CLE-Master-Degree students gathered for a focus group during which the app was assessed.

Figure 10 shows the results about the students’ perception of improvements in terms of language skills. The students of this Case Study claimed that they perceived improvements in their reading and listening abilities after using the app: in fact, 63% of them agreed or strongly agreed with the statement “I feel more confident when I read Italian texts after using the UniOn! app” (while 7% of them disagreed/strongly disagreed), and 53% agreed or strongly agreed with the statement “I feel more confident when I listen to Italian after using the UniOn! app” (while 13% disagreed/strongly disagreed). Improvements in speaking abilities were acknowledged by 53% students of Case Study 5 as well, while 13% of the students declared that they did not improve in that particular skill. Finally, 47% of Case Study 5 students declared that they have perceived an improvement in their writing skills thanks to the app, while 13% felt that UniOn! did not provide them with any enhancement in writing.

Fig. 10
figure 10

Perceived Learning improvements, in terms of speaking, writing, listening, and reading skills – Case Study 5

Full size image

In addition, students of Case Study 5 reported high percentages of perceived improvements in their knowledge and understanding of Italian culture and of local environment (see Figs. 4 and 5): 70% of them agreed or strongly agreed with the statement “I am more familiar with Italian culture” and 77% of them agreed/strongly agreed with “I understand better the local environment”. Again, the students reported satisfying appreciations of the UniOn! app: 60% of Case Study 5 agreed or strongly agreed with the sentences “The app is useful to learn the language intuitively” and “The app was useful during my stay”. Moreover, 70% of these students claimed that they would recommend UniOn! to other people.

5 Results

Analyzing and comparing the results obtained within the five case studies, four main aspects emerge as particularly relevant for our Research Question and that can inform the use of mobile app for incidental learning.

The first and most evident aspect is that the use of a mobile application like UniOn! fosters the understanding of local environment and of local culture from the mobility students’ viewpoint. Figure 4 reports the perceived improvements after using the app in terms of familiarity with Italian culture, for the five case studies participants. The percentages of users agreeing with this statement are indeed high, about 60% of positive answers for Case Studies 1 and 2, and about 70% for Case Studies 3, 4 and 5. The percentages are even higher as for the perceived improvements in understanding the local environment, where more than 70% of respondents of Case Studies 1, 2 and 3 agreed or strongly agreed with the statement, and about 80% for Case Studies 4 and 5, as depicted in Fig. 5. This positive result is reinforced by the appreciation of four Italian students who were involved in the testing along with their international friends in Case Study 5: as people coming from different regions at their first experience in Bologna, they claimed that UniOn! was very useful for them to get to know the local cultural aspects and become familiar with the new environment.

The second relevant aspect of our study is about the perceived improvements in the four language skills (reading, listening, writing and speaking). Respondents from all the five groups confirmed that the receptive skills (listening and reading) of the target language were more likely to be improved if compared to productive skills (speaking and writing), as reported in Table 2. The improvement of productive skills was generally perceived as medium (about 40%/50% for both writing and speaking in groups 3, 4 and 5) or even low (13% in writing and speaking for group 2, and 14% for group 1), whereas the receptive skills showed higher percentages (from 25% to 53% for listening in the five groups and from 38% to 63% for reading). In particular, reading seemed the skill more likely to be improved thanks to the app usage and which is surprising considering that the majority of the input provided to the users is textual.

A third relevant aspect showed in Table 2 is that students with initial intermediate levels (B1-B2) of language proficiency seem to be more likely to improve their language competence if compared to the beginners (A1-A2). Students of Case Studies 3, 4 and 5 declared to have either a B1 or a B2 level of language proficiency when they started to use the app, and their perception of improvement of the four skills was higher than that reported by students belonging to Case Studies 1 and 2, who were beginners and who needed to frequently switch to the English versions of the texts because of the difficulty of using directly the Italian one. Students of Case Studies 1 and 2 also declared that learning with the app for them was not so intuitive, as the input understood was not enough. Students of Case Study 2 in particular considered this task too demanding, whereas students of Case Study 1 who were assisted by a tutor could more easily overcome the difficulties encountered.

A fourth relevant aspect that emerged during our testing is that, for all language levels, the usage was more effective when the mobile application was used for a longer period and in a guided context. In this regard, some weeks of usage were not enough to fully exploit the app potential, as confirmed by the users of Case Studies 1 and 2 who tested the app for three weeks and affirmed that they could not fully benefit from the app due to time constraints. Also, their motivation was lower for this reason. Students of Case Study 1, however, stated that their engagement with the application was facilitated by their tutor, and this was confirmed by users of Case Studies 4 and 5 as well, for whom the exploitation was more effective thanks to the guidance received.

The obtained results have provided a rich amount of evidence in response to our Research Question (RQ), here reported: How effective is the incidental learning approach in improving a second language, when a mobile application is used as a supporting tool for integration of mobility students?

We found out that to answer our RQ and, so, to measure the effectiveness of our approach, the four aspects mentioned in the Results section need to be carefully considered. In the next subsections, a discussion about each aspect is provided in relation to previous research and their contributions to the field of incidental learning and language learning with mobile applications. Finally, in the last subsection, we discuss the limitations of our approach.

5.1 Understanding of local environment and of local culture

From the collected data, it emerges that the majority of the participants agree on the fact that our approach is effective in improving the (perceived) understanding of the local culture and the local environment, as depicted in Figs. 4 and 5.

As maintained by scholars such as Risager and Kramsch, the relationship between language and culture is intrinsic, with language expresses, embodies and symbolizes cultural reality [37, 56]. Moreover, Duranti’s ethnographic examples expand the concept of embodied space by integrating language, body movement, spatial orientation, in-habited space, and distant homelands as expressions of cultural connectedness and socialization [23, 47]. In this scenario, living in an unfamiliar city presents a multitude of learning opportunities, as well as challenges [40]. The learning opportunities related to the urban environment have been investigated with the term “linguistic landscape” [14, 42]. In a few words, the linguistic landscape can be defined as all the language items (including visual and printed texts) that are visible in the public space. As claimed by [14], such items play a relevant role in second language acquisition and can be successfully integrated with mobile learning to developing intercultural competences [80].

In accordance with these studies, our participants’ outcome confirms that UniOn! is able to augment the linguistic landscape through mobile learning [78], exploiting location-based and context-aware contents. Such result echoes the main characteristic that an incidental learning experience should have: a learning activity that takes place in an authentic context-aware scenario, while other activities (such as observation, social interaction, and problem solving) are performed, as defined by [9, 33].

5.2 Perceived improvements in language skills

Considering all the participants at the five Case Studies, the language skill perceived more improved thanks to UniOn! is the one related to “read”, then we can find “listen”, followed by “speak”, and, lastly, “write”. Recent studies investigated the improvements in language skills in the use of smartphone applications, both using social applications, e.g., Telegram and WhatsApp [32], and language learning apps, e.g., Busuu [59]. In particular, the study of Kacetl and Klímová [32] presents an analysis of some studies focusing on the language skills practiced by means of mobile learning, concluding that there is a potential in the use of mobile apps. The specific improved skills depend of the kind of applications. For example, Telegram and WhatsApp was proved to be particularly effective in improving writing skills [1, 3]. In [59], the author confirms that autonomous language learning apps, such a Busuu, let users benefit, in particular, of vocabulary skill, as expected by considering how the mobile learning has been designed.

In our specific context, the fact that reading was the skill that users considered to have benefitted from most is not surprising. In fact, the app has been designed with the goal of mainly providing written texts and contents. Moreover, the texts and contents have been created by adopting techniques that facilitate long time memorization, forming meaningful association [15].

5.3 Preliminary competence with the new language

Our results let emerge that participants with initial intermediate levels (B1-B2) of language proficiency seem more likely to improve their language competence if compared to the beginners (A1-A2). This is contrasting with studies related to autonomous language learning apps that apply a non-formal learning approach. In fact, such apps can best support learners at a beginner level [59]. The motivation can be found in the goal of the application: autonomous language learning apps are designed to provide structured learning situations, by exploiting levels, starting from basic ones (basic grammar rules and vocabulary), progressing toward more complex, with the planned goal to learn a language in an intentional way. Instead, our system has been designed to facilitate and support incidental learning of the language, providing situated contents and dialogues. Thus, in our context, the skills improvement has been more self-perceived in those learners at an intermediate level than in the beginner ones.

5.4 Period of use and context of app usage

The period of use of the application (weeks vs months) as well as the context of use (autonomous, guided, semi-guided) are dimensions that impacted on the effectiveness of UniOn!. Considering the former dimension, as expected, the perceived effectiveness is directly correlated with the duration of the period of use (that varies from 3 weeks to 3 months). Previous studies on mobile learning, analyzing the frequency and the duration of the use, report that apps are used often, but for short interactions (i.e., [58, 59]). Moreover, Rosell-Aguilar [58] provides data confirming the fact that learners use apps mostly spontaneously, but a considerable portion also use them in planned study sessions. In both the cases, the use of mobile learning is devoted to increase the autonomous learning experience such as the user’s engagement [41]. In our context, the possibility of having guided or semi-guided sessions provides a further opportunity to exploit incidental learning possibilities through UniOn!, in a more social scenario. This explains why this dimension influenced in a positive way the efficacy of our application, increasing the frequency of use. It is worth mentioning that guided or semi-guided sessions of app usage were not intended as a formal or non-formal learning activities, but the language skills were acquired and improved in an incidental way.

5.5 Limitations

Since no previous research (to the best of our knowledge) reports on measuring the effectiveness of incidental learning in a mobile learning context, focusing on a similar scenario, it is not possible to directly compare our results to previous studies. Moreover, as explained in the paper, due to the nature of incidental learning, we opted for an evaluation exploiting self-evaluation and self-assessment, research methods that are, for their nature, subject to limitations [55], with specific regards to subjectivity. Subjectivity is surely present in self-evaluation and self-assessment-, and it is not possible to prevent it or to not consider it with such an approach. It could be interesting investigating on how to combine together self-evaluation/self-assessment with objective evaluations conducted by teachers (with level exams). Our mobile app has got a different approach to how to provide content on language and culture, improving related knowledge and skills. In particular, competences and levels mixed together, and it is worth mentioning that such levels are not corresponding to standard language levels (B1, B2, etc.). An interesting issue that we have tried to let emerge and evaluate is how much participants’ confidence in their language and culture skills has been increased after having used the app, how much participants feel themselves masters of everyday life situations after having used the app. Our aim was to measure such an issue, with a self-evaluation/self-assessment approach.

Another limitation of our study is represented by the limited number of involved participants, ranging from 7 (Case Study 1) to 40 (Case Study 3), with a total number of 95 students learning Italian. All of them participated to the case studies on a voluntary basis, hence we cannot exclude the presence of bias.

Finally, we analyzed data about the duration of the app usage, without considering its frequency, since we did not exploit the possibility to collect detailed logs: a further evaluation could be conducted by gathering information about the frequency and by integrating them with the other results. According to the literature [41, 58, 59], frequency is recognized as an important factor in second language acquisition, even if it is still not well investigated, in particular in terms of how it impacts on the language acquisition under incidental learning conditions, hence it will be strategic to better focus on this aspect.

6 Conclusion and future work

This paper presents some reflections about how effective the incidental learning approach can be in improving a second language, when a mobile application is used as a supporting tool for the integration of mobility students. We have defined a methodology that takes into consideration several dimensions and an assessment procedure which is based on the exploitation of the users’ perception of improvement, so as to identify the actual contribution of the incidental-learning-based mobile tool in learning a second language. We have conducted tests with participants at five Case Studies with a different background, in terms of geographical origin and initial language competence, and also with different contexts of app usage and duration.

Our findings confirm that the incidental learning approach is effective when supported by a mobile application with situated and context-aware learning content, and that it could be evaluated by exploiting self-assessment. The self-evaluation results show that the users involved in the experiments perceived more improvements in their language skills under certain conditions, for instance the app should be used for a period which is sufficiently long to guarantee a certain level of engagement while using it and the guided context of usage proved to be the right one to better appreciate the app features, more than the autonomous context. In addition, also the initial level of language competence of the users has shown to impact upon the obtained results in terms of learners’ better perceptions of language skills improvement.

Finally, similar studies could be applied to the other three versions of the UniOn! app, that is: i) the Finnish app geo-localised in Rovaniemi; ii) the Polish app geo-localised in Poznan and iii) the Portuguese app geo-localised in Coimbra. A comparison among the results collected for the four versions of the UniOn! app could be the object of a future, enlarged investigation not limited to the Italian language and culture but aiming at more general considerations about language-culture incidental learning.