Dimensions of scale: Connected Learning Initiative (CLIx)—a case study of educational technology initiative in India

Abstract

Developing countries around the world are scaling up education interventions. New educational technologies offer opportunities to develop new models to deliver quality education at scale. However, the literature suggests that defining scale is complex, especially in heterogeneous contexts. This paper provides a conceptualization of scale as a dynamic process with three key dimensions: ‘quantity’, ‘diffusion,’ and ‘quality’ through the case of a multi-state, multi-stakeholder program called Connected Learning Initiative in India. It also describes the implementation processes that involve teacher professional development, student engagement, technological developments, and efforts to improve classroom processes in a multifactor environment of stakeholder needs, context, and variance in resources. In conclusion, robust and flexible design approaches for scale are discussed, with implications for further research.

Appendices

Appendix A

Student module usage

According to CLIx (2020), out of the 12 student modules, module usage in CG, MZ, and RJ exceeded 4.5, whereas in TS, it was 2.19. There was also variation in module usage across subjects and states. In English, the highest usage was observed in MZ (52%), followed by CG (45%), with an overall average of 31%. For Mathematics, CG had the highest usage at 83%, followed by MZ (48%), with an overall usage rate of 33%. In the case of Science, CG and RJ had a usage rate of 41%, followed by MZ at 32%. Overall, module usage levels were significantly higher in MZ and CG, while lower in TS.

Table A1
State	Average of all the modules used/Total modules rolled out	% of English modules used per school (out of max 2 modules) (%)	% of Math modules used per school (out of max 3 modules) (%)	% of Science modules used per school (out of max 6 modules) (%)
CG	4.60/10	45	83	41
MZ	4.71/11	52	48	32
RJ	4.62/12	28	32	41
TS	2.19/12	25	16	15
All CLIx	3.73/12	31	33	29

1. Module usage by students between July, 2018 and March, 2019. Source. (CLIx, 2020)

Appendix B

Stakeholder/area	Time scale (Short term: 1–2 years Medium term: 3–4 years Long term: 5 years and above)	Impact indicators and targets
Students	Short term	70% students will show improved conceptual understanding in science and mathematics
	Short term	80% students will show improved proficiency in English communication
	Short term	95% students will demonstrate digital literacy skills
	Short term	70% students will show improved ability to use technology for learning and real-life situations
	Medium term	70% students will show improved life skills (communication skills, confidence, citizenship skills, group work, perseverance)
	Medium term	70% students will show increased interest in STEM and self-perception of learning
	Long term	Increase in number of students choosing the science stream after grade 10
	Long term	Increase in the percentage of students continuing education after grade 10
Teachers	Short/medium term	80% teachers become digitally literate
	Short/medium term	At least 50% of teachers use technology to improve teaching learning
	Short/medium term	At least 60% of teachers show improved subject knowledge
	Short/medium term	At least 60% of teachers show improvements in classroom practice by opportunities for students to talk, ask questions, share, discover, and analyze (as against rote or lecture-based methods)
	Short/medium term	At least 50% of teachers teaching English, Science, and Math will start using open resources/technology in classrooms
School	Short term	All project schools to have functional ICT infrastructure
	Medium term	Improved grade 9 and 10 examination results
	Medium term	At least 70% of schools use science labs to conduct experiments
	Medium term	Improved enrollment, attendance, and retention
Teacher educators	Short/medium term	Number of teacher educators with enhanced skills for supporting teachers and student teachers in Science, English, Math, and life skills
	Short/medium term	Number of teacher educators and student teachers from the implementation sites enrolling in courses related to educational technology

1. Appendix B. CLIx impact indicators and targets. (Source. CLIx Operational plan document)

Appendix C

Student learning outcomes

The learning outcome study (CLIx, 2020) was conducted in grade 9 using three subject modules of CLIx. English was studied in MZ, mathematics in CG, and science in RJ. Specific modules from each subject were implemented under the most favorable ICT infrastructure and classroom conditions. The study was spread over 3 to 4 weeks, which included teaching of CLIx modules following prescribed classroom experience, including computer lab-based work for digital components of the module. In the non-intervention schools, over the similar period, the related topic was taught by teachers using the prescribed textbook.

In the intervention schools, higher fidelity of module implementation was attributed to the extent to which CLIx teachers understood the objectives and design of the module and implemented them as per the intended or recommended design while incorporating feedback from the research group collaborating in the classrooms. Pre- and post-tests of students were conducted. For data analysis, paired t tests were conducted to determine if there were significant pre- and post-test gains in scores for students in each intervention and non-intervention school. Independent t tests were then conducted to compare pre-post gains between the intervention and non-intervention schools to determine if the gains were significantly different. The average scores of students and an analysis of the changes are presented in Table C1.

In English, for listening skills, students in the intervention schools showed increased improvement by 21.0 percentage points compared to their counterparts in the non-intervention schools. Students in the intervention schools showed significantly greater gains in listening skills compared to students in non-intervention schools. Whereas for speaking skills, students in non-intervention schools showed a significantly greater gain than those from intervention schools.

In math, the scores of the students in the intervention schools showed significantly higher gains from pre-test to post-test by 13.42 percentage points. The gain of students in the intervention schools from pre-test to post-test was statistically significant compared to their counterparts in the non-intervention schools. A gender analysis of student performance revealed that girls in the intervention group scored significantly higher than girls in the non-intervention group.

In science, the scores of students in the intervention schools improved significantly by 13.27 percentage points, whereas the scores of those in the non-intervention group did not improve. There was a significant difference between the post-test scores of students in the intervention (47%) and non-intervention groups (33%), as well as in their pre- and post-test gains, with the intervention group scoring higher in the post-test and showing greater improvement.

Table C1
Subject-topic	CLIx			Non-CLIx			Difference between CLIx and Non-CLIx	p value
	Pre-test	Post-test	Gain	Pre-test	Post-test	Gain
English—Listening (N: CLIx = 100, Non-CLIx = 75)	40.0	61.0	21.0	57.3	69.2	11.9	9.1	p = 0.003
English—Speaking (N: CLIx = 99; Non-CLIx = 77)	43.0	53.9	10.9	55.7	73.3	17.6	− 6.7	p = 0.03
Maths—Geometric Reasoning (N: CLIx = 466; Non-CLIx = 499)	29.7	43.1	13.4	33.9	36.7	2.81	10.6	p < 0.001
Math—Gender wise data (N: CLIx -F—251, M = 215 Non-CLIx F = 290, M = 209)	F = 29.8 M = 29.6	F = 39.4 M = 47.5	F = 9.6	F = 35.4 M = 38.4	F = 32.8 M = 35.4	F = 2.6	7	p = 0.002
Science—Astronomy (N: CLIx = 169; Non-CLIx = 118)	33.3	46.5	13.2	32.0	32.9	0.91	12.3	p < 0.001

1. Learning outcomes study results—pre and post-test average scores (in percentage). Source. (CLIx, 2020)