Advertisement

Development and psychometric testing of an abridged version of Dundee Ready Educational Environment Measure (DREEM)

  • Kathiresan Jeyashree
  • Hemant Deepak Shewade
  • Soundappan Kathirvel
Open Access
Research article
  • 348 Downloads

Abstract

Background

Dundee Ready Educational Environment Measure (DREEM) is a 50-item tool to assess the educational environment of medical institutions as perceived by the students. This cross-sectional study developed and validated an abridged version of the DREEM-50 with an aim to have a less resource-intensive (time, manpower), yet valid and reliable, version of DREEM-50 while also avoiding respondent fatigue.

Methods

A methodology similar to that used in the development of WHO-BREF was adopted to develop the abridged version of DREEM. Medical students (n = 418) from a private teaching hospital in Madurai, India, were divided into two groups. Group I (n = 277) participated in the development of the abridged version. This was performed by domain-wise selection of items that had the highest item-total correlation. Group II (n = 141) participated in the testing of the abridged version for construct validity, internal consistency and test-retest reliability. Confirmatory factor analysis was performed to assess the construct validity of DREEM-12.

Results

The abridged version had 12 items (DREEM-12) spread over all five domains in DREEM-50. DREEM-12 explained 77.4% of the variance in DREEM-50 scores. Correlation between total scores of DREEM-50 and DREEM-12 was 0.88 (p < 0.001). Confirmatory factor analysis of DREEM-12 construct was statistically significant (LR test of model vs. saturated p = 0.0006). The internal consistency of DREEM-12 was 0.83. The test-retest reliability of DREEM-12 was 0.595, p < 0.001.

Conclusion

DREEM-12 is a valid and reliable tool for use in educational research. Future research using DREEM-12 will establish its validity and reliability across different settings.

Keywords

DREEM Abridged version Psychometric testing Confirmatory factor analysis Educational environment 

Abbreviations

DREEM

Dundee Ready Educational Environment Measure

M.B., B.S.

Bachelor of Medicine and Bachelor of Surgery

QOL

Quality of life

SASP

Students’ academic self-perception

SPoA

Students’ perception of atmosphere

SPoL

Students’ perception of learning

SPoT

Students’ perception of teachers

SSP

Students’ social self-perception

WHOQOL-BREF

World Health organisation Quality of Life- BREF

Background

Education environment refers to the diverse physical locations in which the students learn. The educational environment of an institution has direct and indirect influence(s) upon the knowledge gained by the students including the motivation to learn, personal safety and their well-being [1]. Dundee Ready Educational Environment Measure (DREEM) is one of the tools developed specifically to assess the educational environment of medical institutions as perceived by the students.

DREEM has 50 items grouped under five domains namely students’ perception of learning (SPoL; n = 12), students’ perception of teachers (SpoT; n = 11), students’ academic self-perception (SASP; n = 8), students’ perception of atmosphere (SPoA; n = 12), and students’ social self-perception (SSP; n = 7). It is a generic and culturally non-specific tool which has been translated and validated in at least eight languages and is used worldwide [2]. In addition to being used to diagnose the deficiencies of the current educational environment, DREEM has also been used to compare different groups, to monitor the same cohort over a period of time and to assess factors influencing the educational environment [3, 4]. DREEM was used among undergraduates [5] including nursing [6, 7] and dental students and various post graduate medical students [8, 9] to explore their perceptions of educational environment.

Though universally accepted and used, investigators followed their own analysis and reporting system of DREEM data [10]. Few studies questioned the five-factor structure and construct validity of the original DREEM tool and also tried to develop abridged versions of DREEM [11, 12, 13, 14]. The other studies have developed an abridged version but the properties of the abridged version have not been tested on another cohort. These abridged versions have not been validated nor have they been widely in use.

Our study developed and validated an abridged version of the DREEM-50 with an aim to have a less resource-intensive (time, manpower), yet valid and reliable, version of DREEM-50 while also avoiding respondent fatigue.

Methods

Study design and setting

A cross-sectional study was conducted for development of an abridged version of DREEM at a private, tertiary care teaching institute in Madurai, a city in Tamil Nadu state of India. It is one of the two major and the only private institution teaching medicine and allied health sciences in the city. Every year, 150 students are selected through a competitive examination for the undergraduate course in medicine—Bachelor of Medicine and Bachelor of Surgery (M.B, B.S.).

Study participants

A list of the 450 undergraduate medical students currently enrolled in the institute (third, fourth and sixth semester) as on August 2016 was obtained, and all students were invited to participate in the study. These students were divided into two groups group I (students of the fourth and seventh semester, n = 300) and group II (students of third semester, n = 150).

Data collection

The DREEM-50 item version was administered to both groups after obtaining written informed consent, and two separate datasets were maintained. The author was present during the consent process to ensure that clarifications were provided to queries, if any. Students who did not consent and those unavailable after three attempts of contact were excluded from the study.

Statistical analysis

Double data entry and validation was done using EpiData version 3.1 (EpiData Association, Odense, Denmark). Items that had to be reverse coded were done so before entry. The data was analysed using STATA (version 12.1, copyright 1985–2011 StataCorp LP USA, serial number: 30120504773). Number and proportion were calculated for sociodemographic characteristics like gender and semester. Mean and standard deviation were calculated for age and for score of each item, domain and total scale of DREEM-full and abridged version.

Development of an abridged version

For the development of the abridged version, the authors followed a procedure similar to that adopted in the development of WHOQOL-BREF, an abridged version of WHOQOL-100, which measures quality of life (QOL) [15]. Item-total correlation (r) and domain-total correlation (r) were calculated (group I dataset). Two items with the highest item-total correlation values were selected from each of the five domains of the DREEM questionnaire. When two items had the same correlation value, both were included. Thus, the authors arrived at a 12-item version of DREEM (DREEM-12). The face and content validity of this abridged version was assessed by the three authors, in their capacity as subject experts each with a minimum of 3 years’ experience in medical education.

Validity and reliability of DREEM-12

DREEM-50 and DREEM-12 scores were calculated on group II dataset. Internal consistency was measured using Cronbach’s alpha. The percentage of variance in DREEM-50 that is explained by the abridged version was calculated using R square. DREEM-12 alone was administered on group II to get the second set of scores of DREEM-12 to check for test-retest reliability (r).

Confirmatory factor analysis was performed to assess the construct validity of DREEM-12. A five-factor first-order model was fit to assess the DREEM-12 structure using the items selected for each DREEM-12 component (items), i.e. SPoT (two), SPoT (three), SASP (three), SPoE (two) and SSP (two).

Ethics

Ethics approval was obtained from institute ethics committee of Velammal medical college hospital and research institute, Madurai, Tamil Nadu. Written informed consent was taken from the students and the consent process was approved by the ethics committee.

Results

A total of 277 students from group I participated in the development of the abridged version and 141 students from group II in the validation and reliability of the abridged version. The age and gender distribution of the study population is given in Table 1.
Table 1

Profile of study participants - undergraduate medical students from Madurai, India in 2016 (n = 418)

Characteristics

Group I n = 277

Group II n = 141

Mean age (SD) in years

19.2 (0.8)

18.1 (0.64)

Gender (%)

Male

104 (37.5)

54 (38.3)

Female

173 (62.5)

87 (61.7)

Group I–fourth and sixth semester students; group II–third semester students

SD standard deviation

The item-total correlation and domain-total correlation of the items selected for inclusion in DREEM-12 are given in Table 2. Correlation between total scores of DREEM-50 and DREEM-12 was 0.88 (p < 0.001). Domain–domain correlation between DREEM-50 and DREEM-12 were 0.75 (SPoL), 0.7 (SPoT), 0.79 (SASP), 0.66 (SPoA) and 0.62 (SSP). Regression analysis revealed that DREEM-12 explained 77.4% of the variance in DREEM-50. The internal consistency of DREEM-12 was 0.83 as compared to 0.86 of DREEM-50.
Table 2

Domain-wise list of items in DREEM-50 that had the highest item-total and domain-total correlations

Domain

Item no

Item

Item-total correlation

Domain-total correlation

Students’ perception of learning

44

The teaching encourages me to be an active learner

0.646

0.711

22

The teaching helps to develop my confidence

0.556

Students’ perception of teachers

2

The course organisers are knowledgeable

0.510

0.611

37

The course organisers give clear examples

0.450

18

The course organisers have good communication skills with student

0.450

Students’ academic self-perception

21

I feel I am being well prepared for my profession

0.435

0.599

41

My problem solving skills are being well developed here

0.430

45

Much of what I have to learn seems relevant to a career in healthcare

0.430

Students’ perception of atmosphere

36

I am able to concentrate well

0.500

0.633

43

The atmosphere motivates me as a learner

0.487

Students’ social self-perception

3

There is a good support system for students who get stressed

0.419

0.556

19

My social life is good

0.416

The item wise and overall scoring system of DREEM-12 was devised in line with the DREEM-50 (Table 3). When calculated for group II, DREEM-50 and DREEM-12 scores yielded the same interpretations on the students’ perceptions about the educational environment: overall and at domain level (Table 4).The test-retest reliability of DREEM-12 was 0.595, p < 0.001.
Table 3

Items selected to form DREEM-12 and the applicable scoring system

Domain (Maximum score)

Item No.

Item

Scoring

Student’s Perception of Learning (8)

22

44

The teaching helps to develop my confidence

The teaching encourages me to be an active learner

0–2 Very Poor

3–4 Teaching is viewed negatively

5–6 A more positive perception

7–8 Teaching highly thought of

Student’s Perception of Teachers (12)

2

18

37

The course organizers are knowledgeable

The course organizers have good communication skills with student

The course organizers give clear examples

0–2 Abysmal

4–6 In need of some retraining

7–9 Moving in the right direction

10–12 Model course organizers

Students’ Academic Self-Perception (12)

21

41

45

I feel I am being well prepared for my profession

My problem solving skills are being well developed here

Much of what I have to learn seems relevant to a career in healthcare

0–3 Feelings of total failure

4–6 Many negative aspects

7–9 Feeling more on the positive side

10–12 Confident

Students’ Perceptions of Atmosphere (8)

36

43

I am able to concentrate well

The atmosphere motivates me as a learner

0–2 A terrible environment

3–4 There are many issues which need changing

5–6 A more positive attitude

7–8 A good feeling overall

Students’ Social Self Perceptions (8)

3

19

There is a good support system for students who get stressed My social life is good

0–2 Miserable

3–4 Not a nice place

5–6 Not too bad

7–8 Very good socially

Overall scoring for interpretation of educational environment:

 0–12 Very Poor

 13–24 Plenty of Problems

 25–36 More Positive than Negative

 37–48 Excellent

Table 4

Comparison of DREEM-50 and DREEM-12 scores and their interpretation calculated for group II

SN

Domain

DREEM-50

DREEM-12

Mean score (SD)

Interpretation

Mean score (SD)

Interpretation

1

Students’ perception of learning

30.5 (5.4)

A more positive perception

5.1 (1.6)

A more positive perception

2

Students’ perception of teachers

25 (4.8)

Moving in the right direction

8.4 (2.1)

Moving in the right direction

3

Students’ academic self-perception

21.1 (4.2)

Feeling more on the positive side

8.1 (1.9)

Feeling more on the positive side

4

Students’ perception of atmosphere

28.2 (5.9)

A more positive attitude

4.8 (1.6)

A more positive attitude

5

Students’ social self-perception

16.8 (3.6)

Not too bad

4.7 (1.7)

Not too bad

Total

121.7 (18)

More Positive than Negative

31.1 (6.3)

More Positive than Negative

SD standard deviation

Confirmatory factor analysis of DREEM-12 construct was statistically significant (LR test of model vs. saturated: χ2(44) = 80.88, Prob> χ2 = 0.0006) (Fig. 1).
Fig. 1

Confirmatory factor analysis of DREEM-12 construct. SSP-Social Self Perception, ASP-Academic Self perception, SPOT-Students’ Perception of Teachers, SPOTe-Students’ Perception of Teaching, POE- Perception of Environment three-3, ninet-19, twone-21, fone-41, ffive-45, twtwo-22, ffour-44, two-2, eightn-18, tseven-37, tsix-36, fthree-43. LR test of model vs. saturated: chi2(44)=80.88, Prob > chi2 = 0.0006

Discussion

This study developed and validated DREEM-12, an abridged version of DREEM-50, using two subsets of a population of undergraduate medical students and established good model fit of DREEM-12 by confirmatory factor analysis.

Our study, besides developing an abridged version of DREEM-50, has also confirmed its factor structure and established its validity and reliability. We have employed robust data quality management, through double data entry and validation, and statistical techniques to prove the same. Our study has enrolled students from different years of medical education and thus can be considered representative of the perceptions of students in different stages of medical education.

The study was limited to only students from a single teaching institution in India. However, ours being a study on educational environment, it is also necessary to point out that the quality standards of a medical college with respect to curriculum, teaching methodology, infrastructure, staffing and assessment methods are strictly governed to be uniform by the Medical Council of India. These standards are uniform for all the colleges countrywide. All colleges, government and private, are to abide by these standards, failing which the recognition of the institute is cancelled and the given institute will no more be able to admit candidates. Hence, it will be safe to say that the educational environment offered in colleges across the country are to be uniform. The students are admitted into the undergraduate medical course by a nationwide common competitive entrance examination—National eligibility entrance test. The students do come from different sociodemographic and cultural backgrounds in different parts of the country. But, we do not expect that to significantly affect their perception of their environment. Thus, while the internal validity of our study is commendable, further multi-centric research is required to establish the generalisability of the results outside the country.

The psychometric properties of original DREEM-50 were investigated in few studies. Confirmatory factor analysis found that the five-factor structure of DREEM-50 showed poor model fit in studies conducted in Malaysia and Australia [12, 14]. Both these studies came up with an abridged version of DREEM-50 with a good model fit (Table 5). The Malaysian study derived a 17-item abridged version with good model fit with each domain having three items except the students’ perception of atmosphere which had five items. Similarly, the Australian study derived 12-item abridged version with only four domains after removing student’s social self-perception. The difference in the items chosen in our study, Malaysian and Australian abridged versions might have been due to the following reasons- the methodology of choosing items into the abridged version was different in the three studies. In our study, we had followed the method used in development of WHO QOL BREF. We included items based on their item-total correlations. While the Malaysian version retained the five-domain structure of the original DREEM, the Australian version had done away with the social self-perception domain. Since our study revealed equally good domain-total correlation for all domains, similar to the Malaysian study, we chose to retain all domains in our abridged version. Eight (items 3, 19, 22, 37, 41, 43, 44 and 45) out of the 12 items selected in our study (Table 3) also figure in the Malaysian 17-item version [11, 12]. None of the items in the Australian abridged version are present in the current DREEM-12 except for item 22 [14]. Our abridged version is short like the Australian version and maintained the five-domain structure of the original version like the Malaysian version.
Table 5

Comparison of items in the abridged version of DREEM developed in our study with those in Malaysian and Australian versions

Domains

Item number(s)

India (current study)

Malaysia

Australia

Students’ perception of learning

22, 44

20, 22, 44

13, 22, 24, 48

Students’ perception of teachers

2, 18, 37

6, 37, 40

8, 29, 32, 39

Students’ academic self-perception

21, 41, 45

26, 41, 45

21

Students’ perception of atmosphere

36, 43

30, 33, 42, 43, 49

30, 34, 35

Students’ social self-perception

3, 19

3, 19, 46

Total items

12

17

12

A study from Ireland also mentioned the poor construct validity of the DREEM-50 and found some 17 items having poor fit indices (< 0.7). This study failed to propose an abridged version (33 items) mentioning two reasons, namely, the weakest items may not be same for different settings and fear of affecting the factor structure [13].

Reports on test-retest reliability of DREEM are variable. A study from Brazil reported moderate test-retest reliability of 0.43 [16]. Contrarily, a study validating the Greek translation of DREEM, reported 0.90 [17]. While a generally accepted cut-off of 0.7 is followed, there are no standards laid for test-retest reliability cut off for instruments measuring psychological parameters [18].

Conclusion

DREEM-12, developed as an abridged version of DREEM-50, was found to explain 77.5% of the variance of the scores of the original version and also retained its factor structure, validity and reliability. Future research using DREEM-12 will establish its validity and reliability across different settings.

Notes

Acknowledgements

We thank the students for participating in the study. We thank the Department for International Development (DFD), UK, for funding the Global Operational Research Fellowship Programme at the International Union Against Tuberculosis and Lung Disease (The Union), Paris, France, in which HDS works as an senior operational research fellow.

Funding

None.

Availability of data and materials

The complete double-entered data and validation report are available with the corresponding author. Please contact her for data requests.

Disclaimer

The contents of this paper do not necessarily reflect the views of the Velammal Medical College Hospital and Research Institute, PGIMER Chandigarh or The Union.

Authors’ contributions

KJ, HDS and SK conceived the study. KJ was involved in data collection and entry. KJ, HDS and SK were involved in data analysis and interpretation. KJ prepared the first draft. All authors critically reviewed the draft and gave approval for the final draft to be published.

Ethics approval and consent to participate

Ethics approval was obtained from the institute ethics committee of Velammal Medical College Hospital and Research Institute, Madurai, Tamil Nadu, India. Written informed consent was taken from the students, and the consent process was approved by the ethics committee.

Consent for publication

Not applicable. No individual person’s personal details, images or videos are being used in this study.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.
    Mayya S, Roff S. Students’ perceptions of educational environment: a comparison of academic achievers and under-achievers at kasturba medical college, India. Educ Health (Abingdon). 2004;17:280–91.  https://doi.org/10.1080/13576280400002445.CrossRefGoogle Scholar
  2. 2.
    Roff S, McAleer S, Harden RM, Al-Qahtani M, Ahmed AU, Deza H, et al. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teach. 1997;19:295–9.  https://doi.org/10.3109/01421599709034208.CrossRefGoogle Scholar
  3. 3.
    Whittle SR, Whelan B, Murdoch-Eaton DG. DREEM and beyond; studies of the educational environment as a means for its enhancement. Educ Health (Abingdon). 2007;20:7. http://www.ncbi.nlm.nih.gov/pubmed/17647175 Google Scholar
  4. 4.
    Till H. Identifying the perceived weaknesses of a new curriculum by means of the Dundee Ready Education Environment Measure (DREEM) inventory. Med Teach. 2004;26:39–45.  https://doi.org/10.1080/01421590310001642948.CrossRefPubMedGoogle Scholar
  5. 5.
    Varma R, Tiyagi E, Gupta JK, James PA, Kreiter CD, Shipengrover J, et al. Determining the quality of educational climate across multiple undergraduate teaching sites using the DREEM inventory. BMC Med Educ. 2005;5:8.  https://doi.org/10.1186/1472-6920-5-8.CrossRefPubMedPubMedCentralGoogle Scholar
  6. 6.
    Hamid B, Faroukh A, Mohammadhosein B. Nursing students’ perceptions of their educational environment based on DREEM model in an Iranian university. 2013. https://tspace.library.utoronto.ca/handle/1807/54827. Accessed 26 Oct 2016.Google Scholar
  7. 7.
    O’Brien a P, Chan TM, Cho M a. Investigating nursing students’ perceptions of the changes in a nursing curriculum by means of the Dundee Ready Education Environment Measure (DREEM) inventory: results of a cluster analysis. Int J Nurs Educ Sch. 2008;5:Article25.  https://doi.org/10.2202/1548-923X.1503.Google Scholar
  8. 8.
    Jeyashree K, Patro B. The potential use of DREEM in assessing the perceived educational environment of postgraduate public health students. Med Teach. 2013; http://www.tandfonline.com/doi/full/10.3109/0142159X.2012.737058. Accessed 20 Jan 2016
  9. 9.
    Khan AS, Akturk Z, Al-Megbil T. Evaluation of the learning environment for diploma in family medicine with the Dundee Ready Education Environment (DREEM) inventory. J Educ Eval Health Prof. 2010;7:2.  https://doi.org/10.3352/jeehp.2010.7.2.CrossRefPubMedPubMedCentralGoogle Scholar
  10. 10.
    Miles S, Swift L, Leinster SJ. The Dundee Ready Education Environment Measure (DREEM): a review of its adoption and use. Med Teach. 2012;34(9):e620–34.  https://doi.org/10.3109/0142159x.2012.668625.
  11. 11.
    Yusoff MSB. Psychometric properties of DREEM in a sample of Malaysian medical students. Med Teach. 2012;34:595–6.  https://doi.org/10.3109/0142159X.2012.675104.CrossRefPubMedGoogle Scholar
  12. 12.
    Yusoff MSB. The Dundee Ready Educational Environment Measure: a confirmatory factor analysis in a sample of Malaysian medical students. Int J Humanit Soc Sci. 2012;2:313–21.Google Scholar
  13. 13.
    Hammond SM, O’Rourke M, Kelly M, Bennett D, O’Flynn S. A psychometric appraisal of the DREEM. BMC Med Educ. 2012;12:2.  https://doi.org/10.1186/1472-6920-12-2.CrossRefPubMedPubMedCentralGoogle Scholar
  14. 14.
    Vaughan B, Mulcahy J, McLaughlin P, Miles S, Swift L, Leinster SJ, et al. The DREEM, part 2: psychometric properties in an osteopathic student population. BMC Med Educ. 2014;14:100.  https://doi.org/10.1186/1472-6920-14-100.CrossRefPubMedPubMedCentralGoogle Scholar
  15. 15.
    THE WHOQOL GROUP. Development of the World Health Organization WHOQOL-BREF quality of life assessment. The WHOQOL Group. Psychol Med. 1998;28:551–8.CrossRefGoogle Scholar
  16. 16.
    Filho G, Vieira J, Schonhorst L. Psychometric properties of the Dundee Ready Educational Environment Measure (DREEM) applied to medical residents. Med Teach. 2005;27:343–7. https://www.researchgate.net/profile/Joaquim_Vieira3/publication/7721219_Psychometric_properties_of_the_Dundee_Ready_Education_Environment_Measure_DREEM_applied_to_medical_residents/links/54c238f30cf219bbe4e64ad7/Psychometric-properties-of-the-Dundee-Read. Accessed 5 Mar 2018CrossRefGoogle Scholar
  17. 17.
    Dimoliatis IDK, Vasilaki E, Anastassopoulos P, Ioannidis JPA, Roff S. Validation of the Greek translation of the Dundee Ready Education Environment Measure (DREEM). Educ Heal. 2010;23:348. http://www.ncbi.nlm.nih.gov/pubmed/20589604. Accessed 5 Mar 2018Google Scholar
  18. 18.
    Aldridge VK, Dovey TM, Wade A. Assessing test-retest reliability of psychological measures. Eur Psychol. 2017;22:207–18.  https://doi.org/10.1027/1016-9040/a000298.CrossRefGoogle Scholar

Copyright information

© The Author(s). 2018

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors and Affiliations

  1. 1.Department of Community MedicineVelammal Medical College Hospital and Research InstituteMaduraiIndia
  2. 2.Department of Operational ResearchInternational Union Against Tuberculosis and Lung Disease (The Union), South-East Asia Office, C-6 Qutub Institutional AreaNew DelhiIndia
  3. 3.Department of Community Medicine, School of Public HealthPostgraduate Institute of Medical Education and Research (PGIMER)Chandigarh UTIndia

Personalised recommendations