Factors influencing the accuracy of Canada’s large-scale assessment data: Policies and practices of exclusion, absenteeism, and social promotion

Abstract

The purpose of this study was to examine patterns and factors influencing exclusion rates and achievement on large-scale assessments in Canada. Data analysis employed a case study to examine policies and practices of exclusion, absenteeism, and social promotion related to large-scale assessments at the international, national, and provincial levels. In addition, information was solicited from assessment experts regarding exclusion rate practices. Findings revealed significant increases in student performance, which paralleled significant increases in exclusion rates. At the provincial level, the analysis led to the discovery of the relationship between social promotion policies and a document guiding assessment practices in Canada (i.e., Principles of Fair Assessment Practices). This relationship was the rationale given for excluding poor performing students, not learning or physically disabled students, from participating in large-scale assessments. Recommendations include an alignment of exclusion policies between the three levels of administration, documenting students who are unable to participate on large-scale assessment because they are operating too far below grade level as a result of being social promoted, and to over sample provinces and schools with high absenteeism on assessment days.

Appendices

Appendix

1.1 PEI Data Obtained via a Request for Information

1.1.1 Exemptions - Provincial Mathematics Assessments

Because the Department of Education, Early Learning and Culture is no longer responsible for the education of children in private and band schools, as well as children who are home schooled, those students are not included in this analysis. Therefore, there may be small discrepancies between the following information and information provided in some of the provincial presentations. Also, we do not know the exemption status of absent students.

Primary Mathematics Assessments 2011–2017
EXEMPTION STATUS	PMA 2011	PMA 2012	PMA 2013	PMA 2014	PMA 2015	PMA 2016	PMA 2017
Alternative Education				1		1
EAL	12	8	2		6	22	11
Exempt (extenuating circumstances)	6	2	1	2	5	4	5
Individual Education Plan (IEP)	29	43	48	39	42	28	28
Individual Education Plan and Modified			1
Learning Plan*
Modified	5	30	21	19	25	44	41
Parent Refusal			1
Total Exempt	52	83	74	61	78	99	85
Total Students Public School Branch (excluding absent)	1434	1526	1612	1416	1455	1493	1590
Exemption Rate	3.6%	5.4%	4.6%	4.3%	5.4%	6.6%	5.3%
Absent	10	8	5	5	5	7	0

1. * Learning Plans included in the Exemption Policy as of 2018

Elementary Mathematics Assessments 2013–2017
EXEMPTION STATUS	EMA 2013	EMA 2014	EMA 2015	EMA 2016	EMA 2017
Alternative Education				1
EAL	7	3	1	14	10
Exempt (extenuating circumstances)	2	6	7	9	6
Individual Education Plan (IEP)	44	28	49	45	40
Individual Education Plan and Modified
Learning Plan
Modified	31	38	49	53	62
Parent Refusal
Total Exempt	84	75	106	122	118
Total Students Public School Branch (excluding absent)	1350	1338	1553	1664	1471
Exemption Rate	6.2%	5.6%	6.8%	7.3%	8.0%
Absent	5	4	8	6	7

Intermediate Mathematics Assessments 2011–2017
EXEMPTION STATUS	IMA 2011	IMA 2012	IMA 2013	IMA 2014	IMA 2015	IMA 2016	IMA 2017
Alternative Education	8	14	24	15	12	11	9
EAL	1	2		1		1	1
Exempt (extenuating circumstances)		2	6	1	3	5	5
Individual Education Plan (IEP)	27	27	27	18	24	23	20
Individual Education Plan and Modified	4	3	4
Learning Plan
Modified	130	126	96	99	108	58	70
Parent Refusal			2	1
TAP (Transition Action Plan)	1
Total Exempt	171	172	159	135	147	98	105
Total Students Public School Branch (excluding absent)	1734	1664	1487	1402	1495	1387	1455
Exemption Rate	9.9%	10.3%	10.7%	9.6%	9.8%	7.1%	7.2%
Absent	55	42	21	16	16	18	22
Student Refusal					1

Department of Education, Early Learning and Culture Common Assessment Program -

2.1 Exemptions and Adaptations Definitions

2.1.1 IEP (Individualized Education Plan)

A written record that documents the collaborative process for the development of an individualized plan for a student with special educational needs. This planning is a continuous and integrated process of instruction, assessment, evaluation, decision-making and reporting. The IEP outlines support services and educational program adaptations and/or modifications.

(Department of Education, Early Learning and Culture, 2005, p.75)

2.1.2 Modified

A process which changes the prescribed curriculum to meet a student's special needs. Modified courses do not provide the same credit as a prescribed course. Details of the modified course must be documented and included in the student's file and the student's record or transcript should indicate that the course has been modified.

(Department of Education, Early Learning and Culture, 2005, p. 75)

2.1.3 Adapted

A documented process that allows a student with special needs to participate in a prescribed curriculum (course) with changes in format, instruction strategies and/or assessment procedures that retain the learning outcomes of the curriculum. This adaptation may include alternate formats, instructional strategies and/or assessment procedures. Full credit will be granted for such courses but adaptations used will be documented and kept on the student's file and/or transcript.

(Department of Education, Early Learning and Culture, 2005, p. 73}

2.2 Guidelines for Identifying Students Eligible for Total Exemptions, Partial Exemptions, Adaptations/Accommodations and EAL

2.2.1 Total Exemptions

Total exemptions from the assessment should be considered for those student s who have a cognitive deficit, multiple handicapping conditions or a learning disability to such a degree as would render the assessment inappropriate and/or emotionally harmful. Exemptions will be allowed for student s who have been identified with exceptionalities and have an Individual Education Plan (IEP) or Modified Program in place as of December 1, 2018. The entire booklet for those students must be returned to the Department of Education, Early Learning and Culture with documentation attached, i.e., a photocopy of the first two pages of the IEP/Modified Program.

2.3 Exemptions (Other)

Specific circumstances may prevent a student from participating in the assessment. Such circumstances may include:

• Long-term illness—student was unable to writ e the assessment during the designated time frame.

• Bereavement – student was not able to participate during the designated time frame, due to a significant loss.

• Other—student was unable to participate due to some other circumstance not listed above. The nature of the circumstance must be specified in written format and attached to the student’s assessment booklet before returning to the Department of Education, Early Learning and Culture.

Partial Exemptions

Partial exemptions from the assessment should be considered for those students currently identified with exceptionalities and have an Individual Education Plan (IEP) or Modified Program. These students would be identified as children who are able, with adaptations/accommodations, to attempt a specific component of the assessment. Adaptations used to support the student during instruction should be provided during the assessment, if such adaptations do not compromise or alter the validity of the assessment, as it is important that the results of the assessment accurately reflect the independent abilities of the student .

Appropriate documentation must be attached to the assessment booklet, i.e., a photocopy of the IEP/Modified Program and completed Adaptation/Accommodation form, before returning to the Department of Education, Early Learning and Culture.

Adaptations

Adaptations used to support the student during instruction should be provided during the assessment, if such adaptations do not compromise or alter the validity of the assessment, as it is important that the results of the assessment accurately reflect the independent abilities of the student. Photocopies of that documentation must be attached to the assessment booklet before returning to the Department of Education, Early Learning and Culture. Students currently working with adaptations are not eligible for exemption.

English Second Language (EAL)

EAL student s are expected to participate in the assessment. Appropriate adaptations/ accommodations, if required, should be provided and documentation of adaptations/ accommodations must be attached to the assessment booklet before returning to the Department of Education, Early Learning and Culture.

Note:Though it is expected that the English As An Additional Language student participate in the assessment, if the student's English language skills are not sufficiently developed, then a school-based decision involving, at minimum, the student, the student's parent or guardian, the student's teacher and principal at the school may be to exempt the student from participating in the assessment.

Acceptable Adaptations/ Accommodations

6.1 Additional Time

Students for whom ¹¹additional time ¹ is a documented adaptation may require more than the additional time given to the entire class during the assessment.

6.1.1 Verbatim Scribing

Teachers with students who will use a scribe to complete the assessment this year will need to complete an application form made available from the Department of Education, Early Learning and Culture's Achievement/Assessment Unit.

Please contact Linda MacDonald at lwmacdonald@edu.pe.ca

Verbatim Scribing is allowed for the reading comprehension component only.

Students with a visual or physical impairment, injury or learning disability and for whom ,,verbatim scribing" is a documented adaptation.

If a scribe is required, the scribe writes exactly what the student dictates. Scribes do not edit or proof-read student responses, nor do they advise, suggest or imply that changes are required. The student is required to indicate where punctuation and capitalization are to be inserted.

Verbatim Reading

Students who need instructions or prompts read to them by the teacher or communicated through sign language.

Only verbatim reading of directions or prompts should be used. Reading comprehension passages and questions accompanying them may not be read out loud to students.

Alternate Setting

Students for whom "alternate setting" is a documented adaptation.

Students placed in alternate settings must complete the assessment independently. Ideally, the assessment should be supervised by the student's teacher, even in the alternate setting.

However, if this is not possible, and someone other than the student's teacher is supervising the assessment, this supervisor must know the parameters of the assessment, and the integrity of the assessment must be maintained. The name and the role of the supervisor must be noted on the photocopy of the documented adaptation that is attached to the student' s assessment booklet when the booklet is returned to the Department of Education, Early Learning and Culture.

Assistive Technology

Students with a visual or physical impairment, injury or learning disability, for whom "assistive technology" is a documented adaptation.

Students may use assistive technology in the writing portions of the assessment, if such assistive technology does not compromise or alter the validity of the assessment . Such assistive technology, therefore, should not include such features as cueing systems or grammar or spell-checks, as these would compromise the validity of the assessment. The type of assistive technology (and software) used must be specified on the documented adaptation that is attached to the student's assessment booklet when the booklet is returned to the Department of Education, Early Learning and Culture .

Personal FM System

Students who require a personal FM system during a regular school day.

English/Foreign Language Dictionary

ESL students are permitted to use an EAL dictionary (paper or electronic).