Skip to main content
Log in

Technology-enhanced language learning with null and negative results since 2000: A systematic review based on the activity theory

Education and Information Technologies Aims and scope Submit manuscript

Abstract

Research on technology-enhanced language learning (TELL) has been rapidly growing since 2000, of which the attention is mostly on the statistically significant positive results. However, learning from TELL with null and negative results (NNR), especially its features and reasons, can develop knowledge and awareness of the nature and limitations of TELL and guide its future implementation and investigation. Hence, this study aims to explore TELL with NNR from the perspectives of features and reasons for NNR by systematically reviewing relevant studies. Forty-nine Web of Science and Scopus-indexed articles were screened based on predefined criteria and analysed step-by-step following the PRISMA framework. The findings revealed features of TELL studies with NNR concerning the seven factors of the activity theory: Outcomes, Targets, Learners, Technologies, Teacher and student engagement, Conditions, and Learning theories/Pedagogical models. We also identified reasons for NNR associated with the seven factors. Based on bottom-up analyses of the results, we constructed a 17-item checklist for analysing TELL studies with NNR from the perspectives of Conceptual framework, Artefacts, Design, Participants, Procedures and Evaluation. An example of analysing TELL studies with NNR was provided to illustrate the use of the proposed checklist. Future directions concern technology-enhanced grammar learning, technology with inclusive results, and the influence of sample sizes on TELL study results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

References

Reviewed articles

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Di Zou.

Ethics declarations

Conflict of interest

The authors reported no potential conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Outcomes, Targets, Learners, Technologies, Conditions, Teacher and student engagement, and Learning theories/Pedagogical models of TELL studies with NNR

Authors

Targets

Outcomes

Learners

Technologies

Conditions

Teacher engagement

and student engagement

Learning theories/ Pedagogical frameworks

Target aspects

Target language

Measures

Types of NNR

Sample sizes

Educational levels

Duration

Environments

Teacher engagement

Student engagement

Altunkaya and Topuzkanamis (2018)

Writing

Turkish as L1

Pre-post writing tests, pre-post questionnaires on writing anxiety, writing attitudes and writing self-efficacy

No treatment < TELL

96

Tertiary or higher educational levels

Social networking (i.e., Facebook)

14 weeks

Out-of-class

Building learning environments (i.e., Facebook groups);

Assigning learning tasks;

Providing learning resources;

Providing feedback

Individual plus collaborative

Learning process model

Andújar (2016)

Writing (syntactic and lexical complexity)

English as L2

Pre-post writing tests, app use log files

No treatment <  < TELL > CLL

80

Tertiary or higher educational levels

Chat (i.e., WhatsApp)

6 months

Out-of-class

Building learning environments (i.e., Whatsup groups);

Assigning learning tasks and materials;

Monitoring the learning process;

Answering learner questions

Individual

The Framework for the Rational Analysis of Mobile Education (FRAME);

Social development theory;

Noticing;

and collaborative learning

Andújar (2018)

Writing (syntactic and lexical complexity)

English as L2

Pre-post writing tests

No treatment <  < TELL > CLL

60

Tertiary or higher educational levels

Chat (i.e., WhatsApp)

N.A

Out-of-class

No engagement

individual

Social development theory;

Interaction analysis

Arnold (2007)

Communication apprehension

German as L2

post-treatment self-report, pre-post questionnaires

No treatment <  < TELL > CLL

56

Tertiary or higher educational levels

Course management system (i.e., Blackboard system)

1 semester

Out-of-class (for the asynchronous chat group) and in-class (for the synchronous chat group)

No engagement

Collaborative

N.A

Aslan et al. (2014)

Integrated/ Whole skill

English as L2

Pre-post language tests, post-questionnaire

No treatment <  < TELL > CLL

46

Tertiary or higher educational levels

Intelligent tutoring system (Student modelling system)

8 weeks

Out-of-class

No engagement

Individual

N.A

Aydın and Çakır (2022)

Integrated/ Whole skill

English as L2

Pre-post language tests, pre-post questionnaires, post interviews

No treatment <  < TELL > CLL

76

Tertiary or higher educational levels

Digital games

10 weeks

Blended

Encourage learner engagement in the TELL activities (i.e., talking to the students regularly about their game activities and reminding students

about the episode);

Organise in-class discussions after the TELL activities

Individual

Play Curricular activity Reflection Discussion model

Bahrani and Sim (2012)

Integrated/ Whole skill

English as L2

Pre-post language tests

No treatment < TELL

20

Tertiary or higher educational levels

Websites and digital resources (audiovisual news programs from CNN, BBC, Press T.V.,

and CBS)

10 weeks

In-class

Monitor the learning process;

Organise in-class discussions after the TELL activities

Individual

N.A

Baralt (2013)

Grammar (past subjunctive)

Spanish as L2

Pre-while-post grammar tests, post questionnaire

No treatment < TELL

84

Tertiary or higher educational levels

Chat

Four sessions

In-class

Collaborate with the learners in the TELL activities (i.e., carrying out story-retell tasks with the students)

Individual

Task-based language learning

Berry (2021)

Listening (listening comprehension), flow experience

English as L2

Pre-post listening tests, post-questionnaires, post interviews

No treatment <  < TELL > CLL

61

Tertiary or higher educational levels

Digital games

8 weeks

In-class

Build background knowledge about the TELL activities (i.e., the involved knowledge points)

Collaborative

Digital game-based learning;

The flow theory

Binhomran and Altalhab (2021)

Vocabulary

English as L2

Pre-post-delayed vocabulary tests, post interviews

No treatment <  < TELL > CLL

73

Elementary

Augmented Reality

One session

In-class

Provide technology training;

Support post-TELL knowledge consolidation (i.e., asking content-related questions)

Collaborative

N.A

Bowles (2004)

Vocabulary

Spanish as L2

Pre-post-delayed vocabulary tests

No treatment <  < TELL > CLL

50

Tertiary or higher educational levels

Electronic gloss or annotation

One session

In-class

No engagement

Individual

Noticing hypothesis

Chen (2016)

Writing motivation, writing anxiety, writing self-efficacy

English as L2

Pre-post questionnaires, while-activity journals

No treatment <  < TELL > CLL

44

Tertiary or higher educational levels

Social networking (i.e., Blog)

N.A

Out-of-class

Monitor the learning process;

Evaluate learner performances and give feedback;

Support post-TELL self-reflection (i.e., presenting learners’ errors and exemplary work)

Collaborative

Metalinguistic development

Chen and Chan (2019)

Vocabulary

English as L2

Pre-post vocabulary tests

No treatment <  < TELL > CLL

98

Pre-school

Augmented Reality

4 weeks

In-class

Monitor learning process

Individual

N.A

Chen and Zhang (2010)

Listening

English as L2

Pre-post listening tests, post questionnaire

No treatment <  < TELL > CLL

556

Tertiary or higher educational levels

Course management system (i.e., The New Era Interactive English)

4.5 months

Out-of-class

Monitor learning process

Individual

N.A

Choi et al. (2020)

Interest in language learning

English as L2

Post questionnaires

No treatment < TELL > CLL

298

Pre-school

Digital pens, e-books

N.A

Out-of-class

No engagement

Individual with parents

N.A

Dai and Wu (2021)

Speaking (pronunciation)

English as L2

Pre-post-delayed tests, post-questionnaires, post interviews

No treatment < TELL > CLL

84

Tertiary or higher educational levels

Automatic speech recognition

One session

Out-of-class

Building learning environments (i.e., group students);

Provide technology training;

Provide collaboration training

Individual or collaborative

Skill acquisition theory;

Sociocultural theory

Davie and Hilber (2015)

Vocabulary

English as L2

Pre-post tests

No treatment < TELL

55

Tertiary or higher educational levels

E-flashcards (i.e., Quizlet)

15 weeks

N.A

N.A

Individual

N.A

de Vries et al. (2015)

Grammar

Dutch as L2

Pre-post tests, log files of the technology, post questionnaire

No treatment <  < TELL > CLL

29

Tertiary or higher educational levels

Automatic speech recognition

Two sessions

In-class

No engagement

individual

Noticing hypothesis;

The Output

Hypothesis

Dehghan et al. (2017)

Vocabulary

English as L2

Pre-post vocabulary tests

No treatment <  < TELL > CLL

32

Secondary

Chat (i.e., WhatsApp)

3 months

Out-of-class

Building learning environments (i.e., Facebook groups);

Providing learning resources;

Providing feedback;

Organise content-related discussions

Individual

N.A

Geng and Yamada (2020)

Grammar (compound verbs and single verbs)

Japanese as L2

Pre-post-delayed tests, post questionnaires

No treatment <  < TELL > CLL

21

Tertiary or higher educational levels

Augmented Reality

One session

In-class

No engagement

Individual

Cognitive load theory;

The spatial continuity principle

Ghonsooly and Seyyedrezaie (2014)

Reading (reading comprehension)

English as L2

Pre-post reading tests, post questionnaire

No treatment < TELL <  < CLL

200

Tertiary or higher educational levels

Course management system (i.e., Nicenet)

5 weeks

Out-of-class

No engagement

Individual

N.A

Hong et al. (2022)

Motivation in language learning

English as L2

Pre-post tests, post questionnaire

No treatment <  < TELL > CLL

57

Secondary

Digital games

6 weeks

In-class

Support post-TELL knowledge consolidation

Individual

Digital game-based learning

Hsu et al. (2022)

Language self-efficacy

English as L2

Pre-post tests, post questionnaire

No treatment < TELL

131

Tertiary or higher educational levels

Virtual Reality, Google Glasses

10 weeks

In-class

No engagement

Individual

Social cognitive theory

Ironsi and Bostancı (2022)

Learning experiences, speaking, writing

English as L2

Post language tests, post interviews

No treatment < TELL

77

Tertiary or higher educational levels

Smartphones or cell phones, tablets

N.A

Out-of-class

Evaluate learner performances and give feedback;

Answering learner questions;

Assign learning tasks

Individual

Presentation- Production- Practice

Jalali and Dousti (2012)

Grammar, vocabulary

English as L2

Pre-post vocabulary and grammar tests

No treatment <  < TELL > CLL

58

Elementary

Digital games

4 weeks

In-class

No engagement

Individual

The flow theory

Jiang et al. (2022a)

Speaking (breakdown

fluency, fluency metrics and phonological

accuracy)

English as L2

Classroom observations

No treatment < TELL

160

Tertiary or higher educational levels

Automatic speech recognition

14 weeks

Blended

Provide technology training;

Organise non-technology learning activities

Individual plus collaborative

Flipped learning

Jiang et al. (2022b)

Vocabulary (lexical accuracy)

English as L2

Classroom observations

No treatment <  < TELL > CLL

63

Tertiary or higher educational levels

Automatic speech recognition

14 weeks

Blended

Provide technology training;

Organise non-technology learning activities

Individual plus collaborative

Flipped learning;

The Triadic Componential complexity- accuracy- fluency Framework;

The Trade-off Hypothesis

Kao (2020)

Grammar (article accuracy)

English as L2

Pre-post-delayed tests

No treatment <  < TELL > CLL

45

Tertiary or higher educational levels

Digital games

4 weeks

In-class

No engagement

individual

The theory of language-related episodes;

The Output

Hypothesis

Kuo et al. (2014)

Vocabulary

English as L2

Pre-post-delayed vocabulary tests, post questionnaires

No treatment <  < TELL > CLL

50

Elementary

The Embodiment- based total physical response learning system

One session

In-class

No engagement

individual

The theory of total physical response;

The theory of embodied cognition

Lai (2016)

Vocabulary

English as L2

Pre-post vocabulary tests, log files of the technology

No treatment < TELL > CLL

45

Secondary

Chat (i.e., WhatsApp)

3 months

Out-of-class

Monitor the learning process;

Encourage learner engagement;

Provide feedback

Individual

The input hypothesis;

The interaction hypothesis;

The output hypothesis;

The noticing hypothesis;

The mobile immersion

Lee et al. (2011)

Listening

English as L2

Pre-post listening tests, pre-post questionnaires

No treatment < TELL

21

Elementary

Robots

8 weeks

In-class

Monitor learning process

Individual

The Output

Hypothesis

Li and Li (2022)

Emotional engagement in learning

English as L2

Pre-post questionnaires, post-interviews

No treatment <  < TELL > CLL

69

Tertiary or higher educational levels

Course management system (i.e., The Cloud Classroom)

8 weeks

Blended

Providing learning resources;

Evaluate learner performance and give feedback;

Organise non-technology learning activities

Individual plus collaborative

Flipped learning;

The F-L-I-PTM Model

Li and Peng (2022)

Listening

English as L2

Pre-post listening tests, post questionnaire

No treatment <  < TELL > CLL

61

Tertiary or higher educational levels

Course management system (i.e., The English Listening Learning Platform)

16 weeks

Blended

Provide technology training;

Providing learning resources;

Monitor the learning process;

Organise non-technology learning activities;

Support post-TELL self-reflection

Individual plus collaborative

Flipped learning

Lim and Shen (2006)

Reading

English as L2

Pre-post reading tests, post questionnaires, post interviews (focus group), classroom observations

No treatment <  < TELL > CLL

74

Tertiary or higher educational levels

Websites and digital resources

One semester

In-class

N.A

Individual

N.A

Lin (2017)

Reading (reading comprehension)

English as L2

Pre-post vocabulary and reading tests, post questionnaires

No treatment <  < TELL > CLL

152

Secondary

Tablets

4 weeks

In-class

Provide technology training;

Organise non-technology learning activities

Individual

N.A

Moranski and Kim (2016)

Grammar (pronoun se)

Spanish as L2

Post-tests

No treatment <  < TELL > CLL

213

Tertiary or higher educational levels

Websites and digital resources

13 weeks

Blended

Receive pedagogical training;

Organise non-technology learning activities;

Answer learner questions;

Support knowledge consolidation after TELL

Individual

Flipped learning

Nakata (2008)

Vocabulary

English as L2

Pre-post-delayed vocabulary tests, post-questionnaires, post interviews

No treatment <  < TELL > CLL

226

Secondary

E-flashcards

One session

In-class

No engagement

Individual

List learning;

Cognitive Psychology for effective vocabulary learning

Nayak and Sylva (2013)

Reading (reading comprehension and reading accuracy)

English as L2

Pre-post reading tests

No treatment < TELL > CLL

205

Elementary

E-books

8 weeks

In-class

Provide technology training

Individual

Guided reading

Paris-Morris et al. (2013)

Reading (reading comprehension)

English as L1

Post tests, post questionnaires

TELL <  < CLL

86

Pre-school

E-books

One session

In-class

No engagement

Individual

Dialogic reading

Reynolds and Taylor (2020)

Vocabulary

English as L2

Pre-post vocabulary tests, post questionnaires, post interviews

No treatment < TELL > CLL

24

Tertiary or higher educational levels

Clicker (i.e., Kahoot!)

10 weeks

In-class

Assign learning tasks

Individual

N.A

Reynolds et al. (2021)

Vocabulary

English as L2

Pre-post vocabulary tests, post questionnaires

No treatment <  < TELL > CLL

139

Tertiary or higher educational levels

Clicker (i.e., Kahoot!)

10 weeks

In-class

Assign learning tasks

Individual

N.A

Şimşek and Işıkoğlu Erdoğan (2021)

Integrated/ Whole skills

Turkish as L1

Pre-post tests

No treatment < TELL <  < CLL

56

Pre-school

E-books

8 weeks

In-class

Assign learning tasks

Individual

Dialogic reading

Slim and Hafedh (2019)

Vocabulary

English as L2

Pre-post tests

No treatment <  < TELL > CLL

64

Tertiary or higher educational levels

Social networking (i.e., Facebook)

One semester

Out-of-class

Provide learning resources and assign learning tasks (by posting information on the Facebook page);

Answer learner questions;

Give feedback

Individual

Social-interactionist approach

Su (2017)

Grammar

English as L2

Pre-post grammar tests, post questionnaires, post interviews, log files of the technology

No treatment <  < TELL > CLL

100

Secondary

Course management system (i.e., English Grammar Learning Assistant System based on the tripartite dialogic structure)

3 weeks

In-class

No engagement

Individual

The tripartite dialogic model

Sun (2012)

Speaking (pronunciation, language complexity, fluency, or accuracy)

English as L2

Post questionnaires, the first and the last three activity outcomes

No treatment < TELL

46

Tertiary or higher educational levels

Social networking (i.e., Blogs)

18 weeks

Out-of-class

No engagement

Individual

Task-based learning

Sun et al. (2015)

Self-efficacy

English as L2

Pre-post tests, pre-post questionnaires, post interviews

No treatment < TELL

41

Tertiary or higher educational levels

Cell phones or smartphones, the GPS-based mobile context-aware learning system

One session

In-class

Provide technology training

Individual

Context-aware learning

Wood et al. (2018)

Speaking (expressive definitions)

English as L1

Post-tests

No treatment <  < TELL > CLL

287

Pre-school and elementary

Course management system, e-books

20 weeks

In-class

Monitor learning process

Individual

The Revised Hierarchical Model

Yeşilbağ and Korkmaz (2021)

Learning attitudes, listening,

reading, writing

English as L2

Pre-post tests, pre-post questionnaires

No treatment < TELL > CLL

60

Elementary

Course management system (i.e., Voki)

6 weeks

In-class

Monitor learning process

Individual

N.A

Yu (2016)

Integrated/ whole skills

English as L2

Pre-post tests, post questionnaires

No treatment <  < TELL > CLL

71

Tertiary or higher educational levels

Chat (i.e., Q.Q.)

One semester

Out-of-class

Provide learning resources;

Assign learning tasks;

Encourage learner engagement

Individual

Collaborative learning

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, R., Zou, D. & Cheng, G. Technology-enhanced language learning with null and negative results since 2000: A systematic review based on the activity theory. Educ Inf Technol (2023). https://doi.org/10.1007/s10639-023-11993-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10639-023-11993-1

Keywords

Navigation