Abstract:
A critical aspect of moving psychology into the realm of paradigmatic science, and of ensuring the development and meaningful dissemination of empirically supported interventions, is competency in research design and statistical analysis. We present evidence that psychology graduate programs are not performing as well as they might in this area of training. An expanded version of the Competency Cube Model is used to present the specific skills that are central to seven areas of research design and data analysis by levels of training (bachelors, master’s, doctoral) and career path (clinical, research). The seven areas include: problem conceptualization, measurement and scaling, research methodology, core statistics, specialized statistical procedures, data analysis and management, and communication of findings. In keeping with the Competency Cube Model, we discuss both foundational (broad professional skills) and functional (area-specific skills) competencies in the seven areas, and introduce three overlapping stages of competency, from familiarity to knowledge, knowledge to use, and use to contribution. An extended listing of skills by content area, level of training, and stage of competency is presented to expand the discussion on both the curricula that should be offered for various academic degrees, and the manner in which the knowledge and skills learned should be documented or demonstrated. We recognize the multiple variations and mixtures along a continuum from practicing clinical psychologist to basic research psychologist, and the resulting differences in competencies in research design and statistical analysis. As the field experiences a shortage of quantitative psychologists, researchers need to better document their competence in this area beyond the traditional means (e.g., peer-reviewed publications), as well as seek lifelong learning opportunities to expand their knowledge and skills. Clinical psychologists, on the other hand, must remain sufficiently conversant with this content area so as to evaluate published intervention research and competently evaluate their own practice.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
The following list includes some of our favorite items, both recent and “historical.” We note several items from the series Quantitative Applications in the Social Sciences, by Sage Publications, but recommend the entire series.
Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral training in statistics, measurement, and methodology in psychology: Replication and extension of Aiken, West, Sechrest, and Reno’s (1990) survey of Ph.D. programs in North America. American Psy-chologist, 63, 32–50.
Aiken, L. S., West, S. G., Sechrest, L. B., & Reno, R. R. (1990). Graduate training in statistics, methodology, and measurement in psychology: An international survey. American Psychologist, 45, 721–734.
Aldenderfer, M. S., & Blashfield, R. K. (1984). Cluster analysis. In Sullivan, J. L., & Niemi, R. G., (Series Eds.), Quantitative Applications in the Social Sci-ences, 44. Beverly Hills, CA: Sage.
American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
American Psychological Association. (2002). Ethical principles of psychologists and code of conduct. American Psychologist, 57, 1060–1073.
American Psychological Association. (2005). National standards for high school psychology curricula, August 2005. Washington, DC: Author. Retrieved January 15, 2009, from http://www.apa.org/ed/topss/apa_natlstandards.pdf
American Psychological Association. (2007). APA guidelines for the undergraduate psychology major. Wash-ington, DC: Author. Retrieved January 15, 2009, from www.apa.org/ed/resources.html
American Psychological Association. (2008a). Report of the task force for increasing the number of quantitative psychologists. Washington, DC: Author. Retrieved January 15, 2009, from http://www.apa.org/science/quant-task-force-report.pdf
American Psychological Association. (2008b). Teaching, learning, and assessing in a developmentally coherent curriculum. Washington, DC: American Psycholo-gical Association, Board of Educational Affairs. Retrieved January 15, 2009, from www.apa.org/ed/resources.html
American Psychological Association. (2009). Job outlook for quantitative psychologists. Retrieved January 7, 2009, from http://www.apa.org/science/quant-jobs.html.
American Psychological Association, Center for Work-force Studies. (n.d.). Table 4: Employment characteristics of APA members by membership status, 2000. Retrieved April 10, 2009, from http://research.apa.org/2000membershipt4.pdf
Barlow, D. H., Hayes, S. C., & Nelson, R. O. (1984). The scientist practitioner: Research and accountability in clinical and educational settings. New York: Pergamon Press.
Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). Elmsford, NY: Pergamon Press.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psy-chology, 51, 1173–1182.
Bechtoldt, H., Norcross, J. C., Wyckoff, L. A., Pokrywa, M. L., & Campbell, L. F. (2001). Theoretical orientations and employment settings of clinical and counseling psychologists: A comparative study. The Clinical Psychologist, 54(1), 3–6.
Bernal, G., & Scharró-del-Río, M. (2001). Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultural Diversity and Ethnic Minority Psychology, 7, 328–342.
Bloom, M., Fischer, J., & Orme, J. G. (2009). Evaluating practice: Guidelines for the accountable professional (6th ed.). Boston: Pearson Higher Education.
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally College Publishing Co.
Campion, M. A. (1993). Article review checklist: A criterion checklist for reviewing research articlesin applied psychology. Personnel Psychology, 46, 705–718.
Cleary, T. A. (1968). Test bias: Prediction of grades of negro and white students in integrated colleges. Journal of Educational Measurement, 5, 115–124.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
Cohen, J. (1994). The Earth is round (p < .05). American Psychologist, 49, 997–1013.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
Cohen, L. H. (1979). The research readership and information source reliance of clinical psychologists. Professional Psychology, 10, 780–785.
Council of Science Editors. (2009). Policy on journal access to scientific data. Retrieved January 31, 2009, from http://www.councilscienceeditors.org/services/draft_approved.cfm
Council on Governmental Relations. (2006). Access to and retention of research data: Rights and responsibilities. Retrieved January 31, 2009, from http://206.151.87.67/docs/CompleteDRBooklet.htm
Dana, R. H. (1993). Multicultural assessment perspectives for professional psychology. Boston: Allyn and Bacon.
Dictionary.com. (n.d.). Competence. Dictionary.com unabridged (v 1.1). Retrieved January 08, 2009, from http://dictionary.reference.com/browse/competence
Dillman, D. A. (2000). Mail and internet surveys: The tailored design method (2nd ed.). New York: Wiley.
Elman, N. S., Illfelder-Kaye, J., & Robiner, W. N. (2005). Professional development: Training for professionalism as a foundation for competent practice in psychology. Professional Psychology: Research and Practice, 36, 367–375.
Field, A. (2005). Discovering statistics using SPSS (2nd ed.). London: Sage.
Friedrich, J., Buday, E., & Kerr, D. (2000). Statistical training in psychology: A national survey and com-mentary on undergraduate programs. Teaching of Psychology, 27, 248–257.
Goldfried, M. R., & Wolfe, B. E. (1998). Toward a more clinically valid approach to therapy research. Journal of Consulting and Clinical Psychology, 66, 143–150.
Grimm, L. G., & Yarnold, P. R. (Eds.). (1995). Reading and understanding multivariate statistics. Washington, DC: American Psychological Association.
Grimm, L. G., & Yarnold, P. R. (Eds.). (2000). Reading and understanding more multivariate statistics. Washing-ton, DC: American Psychological Association.
Grissom, R. J., & Kim, J. J. (2005). Effect sizes for research: A broad practical approach. Mahwah, NJ: Lawrence Erlbaum.
Heady, H. R. (2007). Rural veterans: A special concern for rural health advocates. Kansas City, MO: National Rural Health Association. Retrieved February 12, 2009, from http://www.ruralhealthweb.org/go/rural-health-topics/veterans-health
Holmbeck, G. N. (1997). Toward terminological, conceptual, and statistical clarity in the study of mediators and moderators: Examples from the child-clinical and pediatric psychopathology literatures. Journal of Consulting and Clinical Psychology, 65, 599–610.
Huberty, C. J., & Morris, J. D. (1989). Multivariate analysis versus multiple univariate analyses. Psychological Bulletin, 105, 302–308.
Johnson, R. A., & Wichern, D. W. (1982). Applied multivariate statistical analysis. Englewood Cliffs, NJ: Prentice-Hall.
Johnston, J. M., & Pennypacker, H. S. (1980). Strategies and tactics of human behavioral research. Hillsdale, NJ: Lawrence Erlbaum.
Kaschak, M. P., & Moore, C. F. (2000). On the documentation of statistical analyses in the “clicky-box” era. American Psychologist, 55, 1511–1512.
Kaslow, N. J., Borden, K. A., Collins, F. L., Forrest, L., Illfelder-Kaye, J., Nelson, P. D. et al. (2004). Compe-tencies Conference: Future directions in education and credentialing in professional psychology. Journal of Clinical Psychology, 60, 699–712.
Kawachi, I., & Subramanian, S. V. (2006). Measuring and modeling the social and geographic context of trau-ma: A multilevel modeling approach. Journal of Tra-umatic Stress, 19, 195–203.
Kazdin, A. (1995). Preparing and evaluating research reports. Psychological Assessment, 7, 228–237.
Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Boston: Allyn and Bacon.
Keane, T. M., Fairbank, J. A., Caddell, J. M., Zimering, R.T., & Bender, M. E. (1985). A behavioral approach to assessing and treating post-traumatic stress disorder in Vietnam veterans. In Figley, C. R. (Ed.), Trauma and its wake: The study and treatment of post-traumatic stress disorder (pp. 257-294). New York: Brunner/Mazel.
Kim, J., & Mueller, C. W. (1978). Introduction to factor analysis: What it is and how to do it. In Sullivan, J. L., & Niemi, R. G. (Series Eds.), Quantitative Applications in the Social Sciences, 13. Beverly Hills, CA: Sage.
Kuhn, T. S. (1996). The structure of scientific revolutions (3rd ed.). Chicago: University of Chicago Press.
Landau, S. (2002). Using survival analysis in psychology. Understanding Statistics, 1, 233–270.
Leigh, I W., Smith, I. L., Bebeau, M. J., Lichtenberg, J. W., Nelson, P. D., Portnoy, S. et al. (2007). Competency assessment models. Professional Psychology: Research and Practice, 38, 463–473.
Lichtenberg, J. W., Portnoy, S., Bebeau, M. J., Leigh, I.W., Nelson, P. D., Rubin, N. J. et al. (2007). Challenges to the assessment of competence and competencies. Professional Psychology: Research and Practice, 38, 474–478.
Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. In Sullivan, J. L, & Niemi, R. G. (Series Eds.), Quantitative Applications in the Social Sciences, 25. Beverly Hills, CA: Sage.
Marshall, C., & Rossman, C. B. (1989). Designing qualitative research. Newbury Park, CA: Sage.
MacDonald, R. P., & Ho, M-H. R. (2002). Principles and practices in reporting structural equation analyses. Psychological Methods, 7, 64–82.
McBurney, D. H. (2001). Research methods (5th ed.). Belmont, CA: Wadsworth Thomas Learning.
McNemar, Q. (1962). Psychological statistics (3rd ed.). New York: Wiley.
Mooney, C. Z., & Duval, R. D. (1993). Bootstrapping: A nonparametric approach to statistical inference. In Lewis-Beck, M. S. (Series Ed.), Quantitative Applications in the Social Sciences, 95. Newbury Park, CA: Sage.
Murphy, K. R., & Myors, B. (2004). Statistical power analysis: A simple and general model for traditional and modern hypothesis tests (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
Nathan, P. E. (2000). The Boulder model: A dream deferred – or lost? American Psychologist, 55, 250–252.
National Association of School Psychologists. (2003). NCSP renewal guidelines. Retrieved May 1, 2009, from http://www.nasponline.org/certification/New%20NCSP%20booklet.pdf
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
Office of Management and Budget. (2003). Department of Veterans Affairs. Retrieved April 10, 2009, from http://www.whitehouse.gov/omb/budget/fy2004/va.html
Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction (2nd ed.). New York: CBS College Publishing.
Persons, J. B., & Silberschatz, G. (1998). Are results of randomized controlled trials useful to psychotherapists? Journal of Consulting and Clinical Psychology, 66, 126–135.
Popper, K. (2002a). Conjectures and refutations: The growth of scientific knowledge. New York: Routledge. (Original work published 1963)
Popper, K. (2002b). The logic of scientific discovery NewYork: Routledge. (Original work published 1935)
Radloff, L. S. (1977). The CES-D scale: A self report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401.
Reilly, D. H., Barclay, J., & Culbertson, F. (1977). The current status of competency-based training, Pt. 1: Validity, reliability, logistical, and ethical issues. Journal of School Psychology, 34, 474–475.
Roberts, M. C., Borden, K. A., Christiansen, M. D., & Lopez, S. J. (2005). Fostering a culture shift: Assessment of competence in the education and careers of professional psychologists. Professional Psychology: Research and Practice, 36, 355–361.
Rodolfa, E., Bent, R., Eisman, E., Nelson, P., Rehm, L., & Ritchie, P. (2005). A cube model for competency development: Implications for psychology educators and regulators. Professional Psychology: Research and Practice, 36, 347–354.
Rosenthal, R. (1984). Meta-analytic procedures for social research. In Bickman, L., & Rog, D. (Series Eds.), Applied Social Research Methods, 6. Beverly Hills, CA: Sage.
Rosselló, J., Bernal, G., & Rivera-Medina, C. (2008). Individual and group CBT and IPT for Puerto Rican adolescents with depressive symptoms. Cultural Diversity and Ethnic Minority Psychology, 14, 234–245.
Rubin, D. B. (1987). Multiple imputations for nonresponse in surveys. New York: Wiley.
Ruggiero, K. J., Del Ben, K. S., Scotti, J. R., & Rabalais, A. E. (2003). Psychometric properties of the PTSD checklist-civilian version. Journal of Traumatic Stress, 16, 495–502.
Sattler, J. M. (1988). Assessment of children (3rd ed.). San Diego: Sattler.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 142–177.
Scotti, J. R., Majewski, V., Heady, H., & Tunick, R. (2008, November). West Virginia veterans returning from Iraq and Afghanistan: Impact on personal and family functioning. In Scotti, J. R. (Chair; J. Lyons, Discussant), Addressing barriers to service utilization for returning Iraq and Afghanistan veterans and families. Symposium and paper presented at the annual meeting of the International Society for Traumatic Stress Studies, Chicago.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.
Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York: Basic Books.
Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis: Modeling change and event occurrence. Oxford: Oxford University Press.
Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage.
Sonis, J., Palmieri, P. A., Lauterbach, D., King, L. A., & King, D. W. (2008). Introduction to the special section: Innovations in trauma research methods, 2007. Journal of Traumatic Stress, 21, 431–432.
Staats, A. W. (1990). Paradigmatic behavior therapy: A unified framework for theory, research, and practice. In Eifert, G. H., & Evans, I. M. (Eds.), Unifying behavioral therapy: Contributions of paradigmatic behaviorism (pp. 14-54). New York: Springer.
Staats, A. W. (1995). Paradigmatic behaviorism and paradigmatic behavior therapy. In O’Donohue, W., & Krasner, L. (Eds.), Theories of behavior therapy: Exploring behavior change (pp. 659-693). Washing-ton, DC: American Psychological Association.
Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd ed.). New York: Harper Collins College.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston: Pearson Education.
Tanielian, T., & Jaycox, L. H. (Eds.). (2008). Invisible wounds of war: Psychological and cognitive injuries, their consequences, and services to assist recovery. Santa Monica, CA: RAND Corporation.
Thomas, J. C., & Hersen, M. (Eds.). (2003). Understand-ing research in clinical and counseling psychology. Mahwah, NJ: Lawrence Erlbaum Associates.
Tomcho, T. J., Rice, D., Foels, R., Folmsbee, L., Vladescu, J., Lissman, R. et al. (2009). APA’s learning objectives for research methods and statistics in practice: A multimethod analysis. Teaching of Psychology, 36, 84–89.
Viswanathan, M. (2005). Measurement error and research design. Thousand Oaks, CA: Sage.
Watkins, Jr., C. E., Schneider, L. J., Manus, M., & Hunton-Shoup, J. (1990). Terminal master’s-level training in counseling psychology: Skills, competencies, and student interests. Professional Psychology: Research and Practice, 21, 216–218.
Weathers, F., Litz, B., Herman, D., Huska, J., & Keane, T. (1993, October). The PTSD Checklist (PCL): Reliability, validity, and diagnostic utility. Paper presented at the annual convention of the International Society for Traumatic Stress Studies, San Antonio, TX.
Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychological journals: Guidelines and explanations. American Psychologist, 54, 594–604.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer Science+Business Media LLC
About this entry
Cite this entry
Scotti, J.R., Jacoby, V.M., Cohen, S., Patrick, J.H. (2010). Design and Analysis. In: Thomas, J.C., Hersen, M. (eds) Handbook of Clinical Psychology Competencies. Springer, New York, NY. https://doi.org/10.1007/978-0-387-09757-2_14
Download citation
DOI: https://doi.org/10.1007/978-0-387-09757-2_14
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-09756-5
Online ISBN: 978-0-387-09757-2
eBook Packages: Behavioral Science