THE APPLICATION OF THE ANALYTIC HIERARCHY PROCESS FOR EVALUATING CREATIVE PRODUCTS IN SCIENCE CLASS AND ITS MODIFICATION FOR EDUCATIONAL EVALUATION
- 321 Downloads
To design and to manufacture in science learning are increasingly important in science education. Yet, evaluation techniques in school for evaluating students’ creative products are apparently left behind. With the aim of developing an evaluation method to evaluate creative products in science and technology class, this study constructed a set of criteria with data collected from teachers and students. The analytic hierarchy process (AHP), a multiple criteria decision-making tool for single rater, was selected for the purpose of weighting and evaluating students’ products. However, the traditional AHP used one rater’s pair-wise comparisons; its subjectivity and complexity limit its applications in school. For solving this problem, this study developed an advanced technique, called direct-rating AHP (DR-AHP), to extend the applicability of the traditional AHP. The DR-AHP is used to obtain weights or preferences for criteria/alternatives by a process of directly ranking criteria/alternatives by single/multi rater(s), checking consistency, and developing a rank vector matrix. The DR-AHP was implemented in obtaining criteria weights of a hierarchy framework for creative products evaluation by a group of science educators (N = 13) and field-tested in ranking creative products by another group of science teachers (N = 9). Results showed its superiority in objectivity and efficiency over traditional ways of evaluation. The results also demonstrate how the AHP and DR-AHP are capable of helping evaluators systematically construct criteria and/or to evaluate students’ creative products for classroom instruction as well as during many other activities.
KEY WORDSalternative AHP creative products evaluation method products evaluation scientific creativity
Unable to display preview. Download preview PDF.
- Besemer, S. P., & O’Quin, K. (1999). Confirming the three-factor creative product analysis matrix model in an american sample. Creativity Research Journal, 12(4), 287.Google Scholar
- Bodin, L. & Gass, S. (2004). Exercises for teaching the analytic hierarchy process. Informs Transactions on Education, 4(2).Google Scholar
- Besemer, S. P., & Treffinger, D. J. (1981). Analysis of creative products: Review and synthesis. Journal of Creative Behavior, 15(3), 158–178.Google Scholar
- Bottero, M., Comino, E. & Riggio, V. (2011). Application of the analytic hierarchy process and the analytic network process for the assessment of different wastewater treatment systems. Environmental Modelling and Software, 26(10), 1211–1224. doi: 10.1016/j.envsoft.2011.04.002.CrossRefGoogle Scholar
- Donaway, R. L., Drummey, K. W., & Mather, L. A. (2000). A comparison of rankings produced by summarization evaluation measures. Paper presented at the proceedings of the 2000 NAACL-ANLP workshop on automatic summarization-Volume 4.Google Scholar
- Figueira, J., Greco, S. & Ehrgott, M. (Eds.). (2005). The analytic hierarchy and analytic network processes for the measurement of intangible criteria and for decision-making. New York: Springer.Google Scholar
- Green, D. R. (1981). How probability pays. Mathematics in School, 10(2), 23–24.Google Scholar
- Hsu, S. L. (2004). The effect of context-based creative-thinking instruction on students' learning. Unpublished Master Thesis. National Taipei University of Education, Taipei.Google Scholar
- Joint Committee on Standards for Educational Evaluation (1994). The program evaluation standards: How to assess evaluations of educational program. Thousand Oaks: Sage.Google Scholar
- Lin, H. Y. (2004). To explore the effect of situated STS teaching on elementary students’ scientific creativity. Unpublished Master Thesis, National Taipei University of Education, Taipei.Google Scholar
- Ministry of Education (2001). The 1-9 grades science and life technology curriculum standards. Taipei: Ministry of Education.Google Scholar
- National Research Council, N. (2012). Conceptual shifts in the next generation science standards. http://nextgenscience.org/sites/ngss/files/Conceptual%20Shifts%20in%20the%20Next%20Generation%20Science%20Standards%20POST%20PUBLIC%20May%20Draft.pdf. Accessed 18 April 2013.
- National Taiwan Science Education Center. (2012). Products list of elementary life and applied sciences. http://activity.ntsec.gov.tw/activity/race-1/52/elementary06.htm. Accessed 10 July 2013.
- Patton, M. Q. (1987). How to use qualitative methods in evaluation. Newbury Park: Sage.Google Scholar
- Saaty, T. L. (1980). The analytic hierarchy process. New York: McGraw-Hill.Google Scholar
- Saaty, T. L. (1994). Fundamentals of decision making and priority theory with the analytic hierarchy process. Pittsburgh, PA: RWS.Google Scholar
- Trochim, W. M. K. (1998). An evaluation of Michael Scriven’s “Minimalist theory: The least theory that practice requires”. American Journal of Evaluation, 19, 243.Google Scholar
- Yu, J. R., Hsiao, Y. W. & Sheu, H. J. (2011). A multiplicative approach to derive weights in the interval analytic hierarchy process. International Journal of Fuzzy Systems, 13(3), 225–231.Google Scholar