Skip to main content
SpringerLink
Log in

Expanding the evidence of content validity for the Teacher Behavior Checklist using the IRAP

TBC’s content validity using the IRAP

  • Original Article
  • Published:
The Psychological Record Aims and scope Submit manuscript

Abstract

The Teacher Behavior Checklist (TBC) consists of 28 items mainly used to measure college teachers’ performance or to investigate the primary qualities of good teachers. Psychometric studies using the TBC have presented evidence of its validity and reliability in different cultures. However, TBC content validity is exclusively based on self-reported measurements, which presents limitations that are widely discussed in the literature. Our study sought to investigate an alternative measurement for TBC content validity using a latency-based task, namely the Implicit Relational Assessment Procedure (IRAP). The main objective was to assess the strength of the relationship between the concept of “Good Teacher” and “Bad Teacher” and six positive and negative features derived from TBC items using an IRAP preparation. The second objective was to investigate the correlation between IRAP and TBC scores. The participants were 64 undergraduate students (M = 21; F = 43), aged 16 to 36. The IRAP trials included six target stimuli selected from the TBC that could be either “Good Teacher” or “Bad Teacher,” and six labels that could be either positive or negative features. The IRAP revealed that the “Good Teacher” is “Positive,” not “Negative.” On the other hand, the “Bad Teacher” is “Negative,” but participants could not deny that this teacher has “Positive” features. Only one statistically significant correlation was found between the Good Teacher-Negative IRAP trial type and the TBC score. Future studies should expand the use of implicit measurements in psychometric studies using the TBC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.

Similar content being viewed by others

Notes

  1. The effective teacher is one who favors learning without turning educational stimuli into an aversive value for the student. Moreover, what is taught must increase the chances of the learner attenuating or solving problem situations (for her/himself and also for the society of which she/he is a part) outside the classroom. Although we theoretically defend the use of the term “effective teacher” (see Henklain, 2017), studies on the TBC have employed expressions such as “Excellent Teacher,” “Master Teacher,” and sometimes “Good Teacher,” which is easier for participants to understand. The expression “Good Teacher,” for instance, is the smallest and easiest one. For this reason, hereafter we will use the term “Good Teacher” as the construct being evaluated by the TBC.

References

  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Psychological Association.

    Google Scholar 

  • Barnes-Holmes, D. (2012). IRAP (Versão 2012) [Software]. Maynooth: Maynooth University, Ireland.

  • Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., & Stewart, I. (2006). Do you really know what you believe? Developing the Implicit Relational Assessment Procedure (IRAP) as a direct measure of implicit beliefs. Irish Psychologist, 32(7), 169–177 Retrieved May 15, 2018, from http://bit.ly/2Gx9e5r.

  • Barnes-Holmes, D., Hayden, E., Barnes-Holmes, Y., & Stewart, I. (2008). The Implicit Relational Assessment Procedure (IRAP) as a response- time and event-related-potentials methodology for testing natural verbal relations: A preliminary study. The Psychological Record, 58, 497–516. https://doi.org/10.1007/BF03395634.

    Article  Google Scholar 

  • Buskist, W., & Keeley, J. W. (2018). Searching for universal principles of excellence in college and university teaching. New Directions for Teaching and Learning, 156, 95–105. https://doi.org/10.1002/tl.20321.

    Article  Google Scholar 

  • Buskist, W., Sikorski, J., Buckley, T., & Saville, B. K. (2002). Elements of master teaching. In S. F. Davis & W. Buskist (Eds.), The teaching of psychology: Essays in honor of Wilbert J. McKeachie and Charles L. Brewer (pp. 30–39). New York, NY: Psychology Press.

    Google Scholar 

  • Busler, J., Kirk, C., Keeley, J., & Buskist, W. (2017). What constitutes poor teaching? A preliminary inquiry into the misbehaviors of not-so-good instructors. Teaching of Psychology, 44, 1–5. https://doi.org/10.1177/0098628317727907.

    Article  Google Scholar 

  • Cassettari, N. (2014). Avaliação de professors: Uma questão de escolhas [Teacher evaluation: A matter of choices]. Estudos em Avaliação Educacional, 25(57), 166–197. https://doi.org/10.18222/eae255720142829.

    Article  Google Scholar 

  • Chan, G., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2009). Implicit attitudes to work and leisure among North American and Irish individuals: A preliminary study. Internacional Journal of Psychology & Psychological Therapy, 9(3), 317–334 Retrieved May 15, 2018, from https://tinyurl.com/ycuog9t9.

  • Cunha, C. M., de Almeida-Neto, O. P., & Stackfleth, R. (2015). Principais métodos de avaliação psicométrica da validade de instrumentos de medida [Main psychometric evaluation methods of the validity of measuring instruments]. Revista de Atenção de Saúde, 14(47), 75–83. https://doi.org/10.13037/rbcs.vol14n47.3391.

    Article  Google Scholar 

  • Finn, M., Barnes-Holmes, D., Hussey, I., & Graddy, J. (2016). Exploring the behavioral dynamics of the Implicit Relational Assessment Procedure: The impact of three types of introductory rules. The Psychological Record, 66(2), 309–321. https://doi.org/10.1007/2Fs40732-016-0173-4.

    Article  Google Scholar 

  • Finn, M., Barnes-Holmes, D., & McEnteggart, C. (2018). Exploring the single-trial-type-dominance-effect on the IRAP: Developing a differential arbitrarily applicable relational responding effects (DAARRE) model. The Psychological Record, 68, 11–25. https://doi.org/10.1007/s40732-017-0262-z.

    Article  Google Scholar 

  • Ford, C. R. (2016). Identifying effective teaching behaviors of pharmacy faculty master teachers (Doctoral Thesis, Auburn University). Retrieved from https://goo.gl/Y2PbEa

  • Furnham, A. (1986). Response bias, social desirability and dissimulation. Personality and Individual Differences, 7, 385–400. https://doi.org/10.1016/0191-8869(86)90014-0.

    Article  Google Scholar 

  • Henklain, M. H. O. (2017). Aplicações do Teacher Behavior Checklist à formação de avaliação de professores de nível superior: Contribuições analítico-comportamentais e psicométricas [Teacher Behavior Checklist applications to the training and assessment of higher education teachers: Behavior-analytic and psychometrical contributions] (Doctoral dissertation). Retrieved from https://repositorio.ufscar.br/handle/ufscar/9886

  • Hussey, I., Thompson, M., McEnteggart, C., Barnes-Holmes, D., & Barnes-Holmes, Y. (2015). Interpreting and inverting with less cursing: A guide to interpreting IRAP data. Journal of Contextual Behavioral Science, 4(3), 1–6. https://doi.org/10.1016/j.jcbs.2015.05.001.

    Article  Google Scholar 

  • Ismail, E. A., & Groccia, J. E. (2017). Foreign and U.S.-educated faculty members’ views on what constitutes excellent teaching: Effects of gender and discipline. To Improve the Academy, 36(1), 20–38. https://doi.org/10.1002/tia2.20056.

    Article  Google Scholar 

  • Kavanagh, D., Barnes-Holmes, Y., Barnes-Holmes, Y., McEnteggart, C., & Finn, M. (2018). Exploring differential trial-type effects and the impact of a read-aloud procedure on deictic relational responding on the IRAP. The Psychological Record, 68, 163–176. https://doi.org/10.1007/s40732-018-0276-1.

    Article  Google Scholar 

  • Keeley, J., Christopher, A. N., & Buskist, W. (2012). Emerging evidence for excellent teaching across borders. In J. E. Groccia, M. Al-Sudairy, & W. Buskist (Eds.), Handbook of college and university teaching: Global perspectives (pp. 374–390). Thousand Oaks, CA: Sage.

    Chapter  Google Scholar 

  • Keeley, J., Smith, D., & Buskist, W. (2006). The teacher behaviors checklist: Factor analysis of its utility for evaluating teaching. Teaching of Psychology, 33(2), 84–91. https://doi.org/10.1207/s15328023top3302_1.

    Article  Google Scholar 

  • Lammers, W. J., Savina, E., Skotko, D., & Churlyaeva, M. (2010). Faculty and student perceptions of outstanding university teachers in the USA and Russia. Educational Psychology: An International Journal of Experimental Educational Psychology, 30(7), 803–815. https://doi.org/10.1080/01443410.2010.512382.

    Article  Google Scholar 

  • Laros, J. A. (2005). O uso da análise fatorial: Algumas diretrizes para pesquisadores [The use of factorial analysis: Some guidelines to researchers]. In L. Pasquali (Org.), Análise fatorial para pesquisadores (pp. 163–184). Brasília, Brazil: LabPAM.

  • Liu, S., Keeley, J., & Buskist, W. (2015). Chinese college students’ perceptions of characteristics of excellent teachers. Teaching of Psychology, 42(1), 83–86. https://doi.org/10.1177/0098628315620888.

    Article  Google Scholar 

  • McGovern, T. V., & Miller, S. L. (2008). Integrating teacher behaviors with character strengths and virtues for faculty development. Teaching of Psychology, 35, 278–285. https://doi.org/10.1080/00986280802374609.

    Article  Google Scholar 

  • Mizael, T. M., de Almeida, J. H., Silveira, C. C., & De Rose, J. C. (2016). Changing racial bias by transfer of functions in equivalence classes. The Psychological Record, 66, 451–462. https://doi.org/10.1007/s40732-016-0185-0.

    Article  Google Scholar 

  • Pacico, J. C. (2015). Como é feito um teste? Produção de itens [How do we built a test? Item development]. In C. S. Hutz, D. R. Bandeira, & C. M. Trentini (Eds.), Psicometria [Psychometry] (pp. 55–70). Porto Alegre, Brazil: Artmed.

  • Perez, W. F., de Almeida, J. H., De Rose, J., Dorigon, A. H., de Vasconcelos, E. L., da Silva, M. A., . . . Barnes-Holmes, D. (2018). Implicit and explicit measures of transformation of function from facial expressions of fear and of happiness via equivalence relations. The Psychological Record, 69(13), 1–12. https://doi.org/10.1007/s40732-018-0304-1

    Article  Google Scholar 

  • Perez, W. F., Kovac, R., Linares, I. M. P., Barbosa, S. F. U., Gomes, C. T., dos Santos, G. A. R., & de Almeida, J. H. (2017). Utilizando o AAQ-II e o IRAP para prever a ocorrência de esquiva experiencial [Using the AAQ-II and the IRAP to predict the occurrence of experiential avoidance]. Revista Brasileira de Terapia Comportamental e Cognitiva, 19(3), 34–48. https://doi.org/10.31505/rbtcc.v19i3.1052.

    Article  Google Scholar 

  • Perez, W. F., Kovac, R., Nico, Y. C., Fidalgo, A. P., Caro, D., Linares, I. M. P., ... de Almeida, J. H. (2017). Utilizando o Implicit Relational Assessment Procedure (IRAP) para avaliar relações verbais implicadas na esquiva experiencial: Um estudo piloto [Using the Implicit Relational Assessment Procedure (IRAP) to assess the verbal relations entailed on experiential avoidance: A pilot study]. Revista Brasileira de Terapia Comportamental e Cognitiva, 19(3), 77–91. https://doi.org/10.31505/rbtcc.v19i3.1056

  • R Development Core Team. (2017). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/

  • Rabelo, L. Z., Bortoloti, R., & Souza, D. H. (2014). Dolls are for girls and not for boys: Evaluating the appropriateness of the Implicit Relational Assessment Procedure for school-age children. The Psychological Record, 64, 71–77. https://doi.org/10.1007/s40732-014-0006-2.

    Article  Google Scholar 

  • Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22–42. https://doi.org/10.1016/j.stueduc.2016.08.007.

    Article  Google Scholar 

  • Vahey, N. A., Nicholson, E., & Barnes-Holmes, D. (2015). A meta-analysis of criterion effects for the Implicit Relational Assessment Procedure (IRAP) in the clinical domain. Journal of Behavior Therapy and Experimental Psychiatry, 48, 59–65. https://doi.org/10.1016/j.jbtep.2015.01.004.

    Article  PubMed  Google Scholar 

Download references

Funding

This research was supported by the São Paulo Research Foundation (FAPESP 2016/05935-6, 2017/10037-0, 2008/57705-8) and by the Brazilian National Research Council (CNPq, Grants 573972/2008-7, and 465686/2014-1); this study is also part of the research program of the National Institute of Science and Technology on Behavior, Cognition, and Teaching (INCT-ECCE).

Author information

Authors and Affiliations

  1. Psychology Course, Education Center, Federal University of Roraima, Campus of Paricarana, Captain Ene Garcês avenue, 2413, Airport, Boa Vista, Brazil

    Marcelo H. O. Henklain

  2. National Institute of Science and Technology on Behavior, Cognition, and Teaching (INCT-ECCE), São Carlos, Brazil

    Marcelo H. O. Henklain

  3. State University of Londrina, Londrina, Brazil

    Verônica Bender Haydu

  4. National Institute of Science and Technology on Behavior, Cognition, and Teaching (INCT-ECCE), and Federal University of São Carlos, São Carlos, Brazil

    João S. Carmo

  5. Federal University of São Carlos, São Carlos, Brazil

    Monalisa Muniz

  6. Paradigma: Center of Behavioral Sciences and Technology, and National Institute of Science and Technology on Behavior, Cognition, and Teaching (INCT-ECCE), São Carlos, Brazil

    William F. Perez

Authors
  1. Marcelo H. O. Henklain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Verônica Bender Haydu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. João S. Carmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Monalisa Muniz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. William F. Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcelo H. O. Henklain.

Ethics declarations

Conflict of Interest

The authors declare they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This research was approved by the Brazilian platform for ethics committees (Plataforma Brasil, Certificate of Submission for Ethical Appraisal No. 54448416.6.0000.5302).

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Henklain, M.H.O., Haydu, V.B., Carmo, J.S. et al. Expanding the evidence of content validity for the Teacher Behavior Checklist using the IRAP. Psychol Rec 69, 205–214 (2019). https://doi.org/10.1007/s40732-019-00334-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40732-019-00334-9

Keywords

Search

Navigation