In response to the modest use of technology in most classrooms without significant influence on learning, much research has been done on the design of classroom, but the research on assessing the technology-rich classroom has lagged behind considerably. This paper aimed to develop and validate a scale for evaluating technology-rich classroom. In this paper, a framework was proposed to evaluate the technology-rich classroom environment on the physical and psychosocial aspects. Then, we conducted a survey to confirm the structure and the internal consistency reliability of the scale based on the proposed framework. The results indicated that the scale was useful for evaluating technology-rich classroom environment, as the results of evaluation provided informative references about how to improve classroom environments to enhance engaged learning.
Since the 1990s, a massive amount of resources has been expended to create universal access to technology in schools. An underlying assumption fueling these investments is that use of technology in the classroom will transform teaching and learning (Mouza and Lavigne 2013). Despite the investment, researchers have consistently observed modest use of technology without significant influence on teaching and learning in most schools and classrooms (Cuban 2009; Ramsden 2011; JISC 2009). Acknowledging the challenge of technological innovation in schools, scholars have argued the need of shifting attention from technology and software to the design of learning environments and learning activities (Simsek 2005). Thus, the design of learning spaces emerged to focus on the rebuilding of formal and informal learning places in school by utilizing proper technology (Long and Ehrmann 2005).
The classroom is one of the most important learning spaces where most of the formal learning activities and teaching activities take place. Research on the classroom environment has shown that the physical arrangement can affect the behaviors of both students and teachers (Savage 1999), and that a well-structured classroom tends to improve student academic and behavioral outcomes (MacAulay 1990). However, there are several problems that exist in classroom environment when considering the learning needs of the new generation of students who were born and grown up with digital technology (Yang et al. 2013).
The new generation of students expects that technology is part of the learning environment and will have difficulty with environments that lack technology (Tapscott 2009). However, the history of pedagogical practice reveals that despite rapid developments in the outside world with information technology, classrooms have evolved very little over the years (Mäkitalo-Siegl et al. 2012). While our understanding of learning and favorable learning environments has substantially improved and technological innovations are offering a variety of new possibilities, it still seems that most of the today’s schools and universities remain more or less unaffected by these developments.
In recent years, with the rapid development of learning spaces, some research has been done on the design of classrooms for the improvement of the classroom environment and the construction of future classrooms.
The student-centered activities for large enrollment undergraduate programs (SCALE-UP) project was initiated in North Carolina State University, with the aim to establish a highly collaborative, hands-on, computer-rich, interactive learning environment for large, introductory college courses (Beichner et al. 2007). Kansas State in America initiated Technology Rich Classrooms project, and after the project, Ault and Niileksela (2009) found that including technology in a classroom, training teachers how to use the technology, and providing support for technology use might change many aspects of learning. One strand of the literature focused on classroom environment in a broader way of learning spaces, addressing the issues surrounding learning space design principles for new as well as repurposed learning spaces within an institution, or anticipating what the future of learning spaces may hold (Brown 2009; Francis and Raftery 2005; Jamieson et al. 2005; Johnson and Lomas 2005; Oblinger 2005).
Although these studies were able to demonstrate that the combination of the newly designed classrooms and active learning approaches contributed to improving student learning achievements, research on assessing these new technology-rich classrooms, however, lagged behind considerably, with very few empirical studies offering evidence of their impact on educational outcomes (Brooks 2012).
In response to the lack of standards for measuring technology-rich classrooms, this study focused on the method on how to evaluate the classroom environments from both physical and psychosocial aspects.
The next section describes the proposed framework for evaluating the classroom environment. “Method” section discusses the methodology used to evaluate the classroom environment in a school based on the proposed framework. “Results” section presents the results of data analysis. The last two sections conclude the paper by summarizing the main contributions of proposed work and presenting future directions.
Framework for evaluating technology-rich classrooms
Typically, most classroom layouts are “rows of seats and tables facing forward”; teachers fixed on the podium for managing the computer; and classes using PowerPoint presentations are no better or even worse than traditional classes (Yang 2015). All these problems in the classroom make the classroom a place of no attraction for digital native students who have a culture of connectivity and online creating and sharing (Ng 2012). The literature reveals that the design of learning space should consider the characteristics and behavior of learners (Oblinger 2006).
Today’s learners have grown up with digital technology and Internet. Net generation, digital natives, Millennials, and Generation Y were used to describe the today’s digital learners. Howe and Strauss (2000), Palfrey and Gasser (2008), Tapscott (2009), Prensky (2010), and other authors argued that today’s generation of learners behaved differently compared with the previous generation, because they were immersed in a world infused with digital technologies. It was claimed that they learned differently, they exhibited different social characteristics, and had different expectations about life and learning. These researchers argued that this digital generation of learners preferred active rather than passive learning and preferred using digital technologies and collaborating to finish the work.
Brown (2005) attempted to map from the net generation’s characteristics to learning theory, and then from theory to learning space applications, and the technology that might support those applications. Perkins (2009) proposed a PST-S framework as the principles to design learning space, which not only considered the convergence of pedagogy, space, and technology, but also placed the individual learner at the center of the teaching and learning processes. Given that classroom is the most important learning space, the characteristics of the digital generation of learners, pedagogy, and technology should be considered in its design and evaluation.
According to Fraser (1998a, b, p. 3), learning environment encompasses “social, physical, psychological, and pedagogical contexts in which learning occurs and affects student achievement and attitudes. As the pedagogy adopted by teachers was the most important factors influencing student’s psychological perception in a classroom, Zandvliet and Fraser (2005) proposed the physical, social, and psychological context as the three dimensions of evaluating classroom environment. But the social context and psychological context could not be separated and always combined with the psychosocial context; therefore, we decided to cover the physical context and psychosocial context in the classroom environment in this research.
Looking back the research about the physical context of the classroom environment, we found that the layout, lighting, and air quality were the main focus of research (Taylor et al. 2010). However, a few researchers performed research on the physical context of the classroom environment from the perspective of pedagogy, but different pedagogies demand different learning spaces (Radcliffe 2008). More importantly, the characteristics of the digital generation of learners were not considered while designing and evaluating the classroom environment. The advantages of more student-centered, engaging and reflective, interactive and collaborative learning environments “do not happen spontaneously unless the learning environments are thoughtfully designed” (Wang 2009).
Therefore, we proposed a framework for designing and evaluating physical classroom environment by considering the characteristics of digital students and various pedagogies adopted by teachers, as shown in Fig. 1. In order to ensure that the framework was theory based and practically oriented, Delphi survey techniques were used to collect and organize experts’ ideas on the meaning of each dimension and the items for each dimension. Finally, showing stands for the level of convenience teachers/students presenting and sharing instructional contents or learning outcomes; manageable stands for the level of convenience teachers/students utilizing the facilities, for instance, whether the layout of a classroom is flexible to change according to different pedagogies; accessible stands for the level of convenience teachers and students getting access to digital learning resources in classroom; tracking stands for the intelligent control of lighting, temperature, electricity, noise, etc., for instance, whether the lighting is suitable for reading; enhancement stands for the level of facilitation of learning and teaching by the technology used in classroom.
When considering the psychosocial aspect of the classroom environment, several scales and inventories have been developed and validated. “What Is Happening In this Class? (WIHIC)” is one of these most famous scales that have been used extensively all over the world (Fraser 1998a, b). WIHIC includes seven dimensions of student cohesiveness, teacher support, involvement, investigation, task orientation, cooperation, and equity, which provide a useful foundation since it identifies a comprehensive set of factors that could reveal the actual psychosocial environment and the relationship between the psychosocial environment and students’ learning outcome.
Equity and student cohesiveness were excluded in this study, as we mainly discuss the classroom environments from pedagogies and learning needs of the digital learners. Therefore, a framework for evaluating the psychosocial aspect of the classroom environment was developed, including five dimensions of teacher support, involvement, investigation, task orientation, and cooperation, as shown in Fig. 2. Teacher support stands for the degree to which the teacher helps, befriends, trusts, and is interested in students; involvement stands for the degree to which students have attentive interest, participate in discussions, do additional work, and enjoy the class; investigation stands for the degree to which the emphasis is on the skills and process of inquiry and their use in problem solving and investigation; task orientation stands for the degree to which it is important to complete activities planned and to stay on the subject matter; and cooperation stands for the degree to which students cooperate rather than compete with one another on learning tasks.
According to the elements of physical and psychosocial classroom environments, we proposed the framework for evaluating technology-rich classroom. The final framework is shown in Table 1.
The objective of this study was to develop a scale for evaluating the technology-rich classroom by utilizing the framework we proposed in the second section. Therefore, a survey was conducted with 283 students in a middle school to appraise both the physical and psychosocial classroom environments.
Based on the framework for designing and evaluating the classroom environment, we developed a questionnaire for evaluating the classroom environment named the Classroom Environment Evaluation Scale (CEES), which consisted of ten dimensions and five items for each dimension.
A key step in creating the questionnaire was item generation, in which we should ensure that the potential respondents could understand each item. For this purpose, a list of 50 items was presented to a group of 20 secondary (aged 13–19) students for their feedback. Specifically, these students explained what they thought each statement meant to them. After that, the items were revised for clarity, language, and parsimony in sentence. In the end, the refined 50 items were hypothesized to load on the above-mentioned ten dimensions and were measured on a five-point scale with one “strongly disagree,” and five “strongly agree.”
A sample of 283 students from a middle school in Beijing, China was selected for this study. The sample consisted of 87 grade 7 students, 73 grade 8 students, 70 grade 10 students, and 58 grade 11 students. The female subsample constituted 53.3 % of the full sample and 46.7 % male. The classrooms where these students lived mainly had some networked computers, laser projectors, with Internet access for teachers or students. For each classroom, a general profile of the learning environment was constructed by evaluating a number of psychosocial and physical factors.
Data were analyzed using confirmatory factor analyses, descriptive statistics, and correlative analyses between the physical environment and psychosocial environment scores. Confirmatory factor analysis (CFA) and scale reliability analysis were employed to substantiate the structure of the CEES. In this study, AMOS 7.0 was employed as the computer software for the CFA. Cronbach’s alpha coefficient was used as an estimate of the internal consistency reliability for CEES. Simple correlation and multiple linear regression analyses were computed to analyze the correlation between physical classroom environment variables and psychological classroom environment variables.
Descriptive statistics for the scale
Data were first analyzed by using descriptive statistics that estimated the item means and deviations, as shown in Table 2.
Overall, the study indicated that students viewed their learning environments positively, and the environments were characterized by relatively high levels in showing and task orientation. However, students rated the amount of accessible in these settings as very low, which indicated it was not convenient to access to the Internet and digital resources in the classroom.
Confirmatory factor analysis
Before conducting factor analysis, the KMO and BTS were applied to ensure that characteristics of the dataset were suitable for factor analysis. Factor loadings along with the KMO and BTS results are provided in Table 3.
As indicated above, CFA was performed on the data by means of AMOS 18.0 software to substantiate the structure of the 50-item CEES. Classroom environment (as assessed by the CEES) was indicated or assessed by ten latent variables, which were expected to explain the intercorrelations among the 50 observed indicators. Fit statistics for this model were RMSEA = 0.056, CFI = 0.914, and NFI = 0.928. These statistics indicate acceptable model fit to the data and confirm the 10-scale structure of the CEES (Hair et al. 2010). For the relationship between the ten CEES scale latent variables and the 50 observed indicators, loadings ranged from 0.610 to 0.943, as shown in Fig. 3.
Reliability coefficients (Cronbach coefficient alpha) were computed for each scale (see Table 4). Indices for each dimension ranged from 0.77 of tracking to 0.95 of the investigation, indicating that the internal consistency of the responses was acceptable for all the constructs.
Associations between physical and psychosocial environment factors
Simple correlation and multiple linear regression analyses were computed to explore possible associations between psychosocial environment scale variable and physical factors. In Table 5, a number of significant independent associations emerged between the physical environment variable and psychosocial variable. Further, significant independent associations emerged between the physical variable and psychosocial variable. These results suggested that these physical attributes of a learning space could influence the psychosocial environment.
Indicators for the classroom environment
We could calculate the mean value for each of the ten scales in CEES, and a large-scale sample survey could indicate the baseline for each scale of the questionnaire. In this study, we assumed that 0–0.4 stands for bad status, 0.4–0.8 stands for normal status, 0.8–1 stands for the idea status, which should be tested in future research. As shown in Fig. 4, the yellow line was the status of the physical classroom environment in the school. “A” was in the bad status, which indicated that accessing to technology in the classroom should be improved. In fact, this was proved by the interviews with students who expressed that although WIFI could be accessed in the classroom, it was forbidden for students.
The psychosocial environment could also be expressed in this way of the radar chart as shown in Fig. 5. “IN” and “IV” was not quite satisfied compared with the other three factors. Therefore, in order to improve the classroom environment, the investigation and involvement should be enhanced to make students more engaged in their learning process.
The aim of this research was to develop and validate a scale for evaluating technology-rich classroom. Analysis of data in this study revealed several important conceptual and substantive implications for learning environment researchers, administrators, and classroom practitioners.
First, this study proposed a framework for evaluating the technology-rich classroom environment from the pedagogical aspect in order to facilitate teaching and learning activities in class. SMATE model has been tested for its usefulness to design and evaluate the physical aspect of the classroom environment. The five dimensions of showing, manageable, accessible, tracking, and enhancement could be used as the indicators for the optimization of physical classroom environment, and the five dimensions of TS, IN, IV, TO, and CO could be seen as the guidance for the development of psychosocial environment.
Second, CEES was developed and validated to measure the physical and psychological environments in class. The CEES was validated with a limited number of samples, and a large-scale survey is needed to be done to validate the scale in a broader sense. Moreover, the CFA analysis was not the most appropriate method in the study as the assumption of independence of the observations for running CFA was violated. Because students from the same class shared the same technology-rich classroom environment, their responses to the items would not be independent. When a large-scale investigation is carried out in the near future, the multilevel CFA should be the appropriate method.
Third, both the five indicators of the physical classroom environment and the five dimensions of the psychosocial environment could be represented in a radar diagram, which could show clearly which dimension is strong and which is weak. It would be useful for teachers, school administrators, and policy makers to understand the status of technology-rich classroom environment in a class, a school or a region, and then accordingly adopt strategies to improve the situation.
Fourth, the association between physical classroom environment and psychosocial classroom environment indicated that the physical environment might influence the psychosocial environment. Therefore, different psychosocial climates demand different physical settings. For example, if the teacher wants to adopt collaborative learning strategy, the classroom layout may be several round tables with students working together instead of “rows of seats and tables facing forward.” Physical settings for showing, manageable, accessible, tracking, and enhancement should be flexible to change according to the different pedagogies.
The topic of “smart learning environment” is an emerging research field, focusing on the learning place or an activity space that can sense learning scenarios, identify the characteristics of learners, provide appropriate learning resources and convenient interactive tools, automatically record the learning process and evaluate learning outcomes in order to promote the effective learning. Smart classroom is one direction in smart learning environment research. In order to promote the full infusion of technology in classroom learning and teaching, the design and evaluation methods for the classroom environment become more and more important, which is one important part of smart classroom. This paper has invented a scale for evaluating technology-rich classroom environment, which is one part of smart classroom research. The structures of smart classroom, the learning experience in smart classrooms, the teaching and learning models in smart classroom, and the evaluation of smart classroom on a large scale should be investigated for promoting easy, effective, and engaged learning.
Effective teaching and learning are linked to sufficient fit between the person and classroom environment. With the development of information technology, classrooms are gradually equipped with various technologies to facilitate teaching and learning. However, the literature has revealed that many problems existed within the physical and psychosocial classroom environments. In order to solve these issues, we proposed a framework for designing and evaluating the classroom environments in both physical and psychosocial aspects. Based on the framework, we developed and validated a scale for evaluating the classroom environments in both physical and psychosocial aspects.
While the findings of this study seem to indicate that CEES was useful in evaluating classroom environment, more empirical investigations of the CEES in regards to students’ learning outcomes, such as attitudes, satisfaction, and achievements, should be carried out in the future to verify these findings. This study was just the tentative beginning in the big map of smart classroom. Empirical studies should also be conducted in the designing of the technology-rich classroom, the effects of pedagogies at different kinds of technology-rich classrooms, the relationship between the scores of CEES and student’s achievements, etc.
Ault, M., & Niileksela, C. (2009). Technology rich classrooms: Effect of the Kansas model. Paper presented at the National Meeting of the National Educational Computing Conference (NECC), Washington, DC.
Beichner, R., Saul, J., Abbott, D., Morse, J., Deardorff, D., Allain, R., et al. (2007). Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) project. In E. Redish & P. Cooney (Eds.), Research-based reform of university physics (pp. 1–42). College Park: American Association of Physics Teachers.
Brooks, D. C. (2012). Space and consequences: The impact of different formal learning spaces on instructor and student behavior. Journal of Learning Spaces, 1(2), 10. Retrieved from http://libjournal.uncg.edu/ojs/index.php/jls/article/viewArticle/285.
Brown, M. (2005). Learning space design theory and practice. Educause Review, 40(4), 1.
Brown, M. (2009). ‘Inversions’. Educause, 44(2), 65–70. http://www.educause.edu/EDUCAUSE%2BReview/ERVolume442009/EDUCAUSEReviewMagazineVolume44/163794.
Cuban, L. (2009). Oversold and underused: Computers in the classroom. Cambridge: Harvard University Press. Retrieved from http://books.google.com/books?hl=zh-CN&lr=&id=sdSutyVQfzYC&oi=fnd&pg=PP6&dq=Oversold+and+underused:+Computers+in+the+classroom&ots=VEZluu6tVV&sig=r6zwenZXNS46tdXyw5fslAMefj0.
Francis, R., & Raftery, J. (2005) Blended learning landscapes. Brookes eJournal of Learning and Teaching, 1(3), 1–5. http://bejlt.brookes.ac.uk/vol1/volume1issue3/perspective/francis_raftery.html.
Fraser, B. J. (1998a). Classroom environment instruments: Development, validity and applications. Learning Environments Research, 1(1), 7–34. Retrieved from http://www.springerlink.com/index/X318568887820545.pdf.
Fraser, B. J. (1998b). Science learning environments: Assessment, effects and determinants. In B. J. Fraser & K. G. Tobin (Eds.), International handbook of science education (pp. 527–564). Dordrecht: Kluwer.
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis: A global perspective (7th ed.). Upper Saddle River, NJ: Pearson Education.
Howe, N., & Strauss, W. (2000). Millennials rising: The next greatest generation. New York: Vintage Books.
Jamieson, P., Dane, J., & Lippman, P. (2005). Moving beyond the classroom: Accommodating the changing pedagogy of higher education’. Refereed Forum Proceedings of the Australian Association for Institutional Research (pp. 17–23). http://www.aair.org.au/app/webroot/media/pdf/AAIR%20Fora/Forum2005/Jamieson.pdf.
JISC. (2009). Designing spaces for effective learning: A guide to 21st century learning space design (pp. 1–33). Bristol: HEFCE. Retrieved from http://www.jisc.ac.uk/media/documents/publications/learningspaces.pdf.
Johnson, C., & Lomas, C. (2005). Design of the learning space: Learning and design principles. Educause Review, 40(4), 16–28.
Long, P. D., & Ehrmann, S. C. (2005). Future of the learning space: Breaking out of the box. Educause Review, 40(4), 42–58. Retrieved from http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume40/FutureoftheLearningSpaceBreaki/157992.
MacAulay, D. J. (1990). Classroom environment: A literature review. Educational Psychology, 10(3), 239–253.
Mäkitalo-Siegl, K., Zottmann, J., Kaplan, F., & Fischer, F. (Eds.). (2012). The classroom of the future: Orchestrating collaborative learning spaces (pp. 245–255). Rotterdam: Sense. Retrieved from http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Classroom+of+the+Future+Orchestrating+Collaborative+Spaces#1.
Mouza, C., & Lavigne, N. (2013). Introduction to emerging technologies for the classroom: A learning sciences perspective. In C. Mouza & N. Lavigne (Eds.), Emerging technologies for the classroom SE-1 (pp. 1–12). New York: Springer. doi:10.1007/978-1-4614-4696-5_1.
Ng, W. (2012). Can we teach digital natives digital literacy? Computers & Education, 59(3), 1065–1078. doi:10.1016/j.compedu.2012.04.016.
Oblinger, D. (2005). Leading the transition from classrooms to learning spaces. Educause Quarterly, 1, 7–12.
Oblinger, D.G. (2006). Space as a change agent. In D. G. Oblinger (Ed.), Learning spaces (pp. 1–444). Boulder, CO: EDUCAUSE. Retrieved from http://net.educause.edu/ir/library/pdf/PUB7102d.pdf.
Palfrey, J., & Gasser, U. (2008). Born digital: Understanding the first generation of digital natives. New York: Basoc Books.
Perkins, J. (2009). Enabling 21st century learning spaces [Web log message]. Retrieved from http://jperk30.edublogs.org/2009/11/07/enabling-21st-century-learning-spaces/.
Prensky, M. (2010). Teaching digital natives: Partnering for real learning. London: Sage.
Radcliffe, D. (2008). A pedagogy-space-technology (PST) framework for designing and evaluating learning places. Learning spaces in higher education: Positive outcomes by design. Retrieved from http://scholar.google.com/scholar?q=A+Pedagogy-Space-Technology+(PST)+Framework+for+Designing+and+Evaluating+Learning+Places&btnG=&hl=zh-CN&as_sdt=0,5#0.
Ramsden, B. (2011). Evaluating the impact of learning space. Reference Services Review, 39(3), 451–464.
Savage, T. V. (1999). Teaching self-control through management and discipline. Boston: Allyn and Bacon.
Simsek, N. (2005). Perceptions and opinions of educational technologists related to educational technology. Journal of Educational Technology & Society, 8(4), 178–190.
Tapscott, D. (2009). Grown up digital: How the net generation is changing your world. New York: McGraw-Hill.
Taylor, P., Zandvliet, D. B., & Straker, L. M. (2010). Physical and psychosocial aspects of the learning environment in information technology rich classrooms. doi:10.1080/0014013011006041.
Wang, Q. (2009). Designing a web-based constructivist learning environment. Interactive Learning Environments, 17(1), 1–13.
Yang, J. (2015). A method for evaluating technology-rich classroom environment. In G. Chen, V. Kumar, Kinshuk, R. Huang, & S. C. Kong (Eds.), Emerging Issues in Smart Learning SE - 4 (pp. 31–40). Berlin: Springer.
Yang, J., Huang, R., & Li, Y. (2013). Optimizing classroom environment to support technology enhanced learning. In A. Holzinger & G. Pasi (Eds.), Human-computer interaction and knowledge discovery in complex, unstructured, big data (LNCS 7947) (pp. 275–285). Berlin: Springer.
Zandvliet, D. B., & Fraser, B. J. (2005). Physical and psychosocial environments associated with networked classrooms. Learning Environments Research, 8(1), 1–17. doi:10.1007/s10984-005-7951-2.
This paper is supported by the 2012 National Key Project of 12th five-year plan for Educational Technology: “Strategies and models for promoting the faculty development (126220656).”
Appendix: Classroom Environment Evaluation Scale
Appendix: Classroom Environment Evaluation Scale
About this article
Cite this article
Yang, J., Huang, R. Development and validation of a scale for evaluating technology-rich classroom environment. J. Comput. Educ. 2, 145–162 (2015). https://doi.org/10.1007/s40692-015-0029-y
- Learning environment
- Technology-rich classroom