Abstract
Relationships among preferred minus non-preferred hand difference scores from four neuropsychological sensorimotor tests were evaluated in Community (N= 121), Prison Inmate (N = 350), and Psychiatric Inpatient (N = 398) samples. Across the three samples, the average absolute off-diagonal correlations among the four difference scores ranged from .07 to .13, and the squared multiple correlations (used to predict any difference score from the other three) did not exceed .07. Based on the determinant of the correlation matrix (i.e., |R|), the null hypothesis of complete independence was rejected in only the Prison Inmate sample (p < .001), however, departure from the null hypothesis was negligible (i.e., |R| = .94, expected value under the null = .98). Preferred/non-preferred hand ratio scores were very highly correlated with the difference scores and consequently analyses of the ratio scores yielded results that were comparable to those obtained for the difference scores. Results indicate that the four difference scores or four ratio scores measure little in common and therefore cannot corroborate each other. In general, it would be prudent to exercise considerable caution when using difference scores or ratio scores obtained from non-independent measures, either singly or in combination.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arthur, G. (1947). A point scale of performance tests. Revised Form II. New York: Psychological Corporation.
Box, G.E.P. (1949). A general distribution theory for a class of likelihood criteria. Biometrika, 36, 317–346.
Costa, L.D., Vaughan, H.G., Levita, E., & Farber, N. (1963). Purdue Pegboard as a predictor of the presence and laterally of cerebral lesions. Journal of Consulting Psychology, 27, 133–137.
Ctonbach, L.J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.
Cronbach, L.J., & Furby, L. (1970). How we should measure “change”—or should we? Psychological Bulletin, 74, 66–80.
Doward, J., & Posthuma, A. (1993). Validity limits of forensic neuropsychological testing. Journal of Forensic Psychology, 11, 17–26.
Fisher, R.A. (1921). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10, 507–521.
Golden, C. J., Osmon, D.C., Moses, J.A., & Berg, R.A. (1981). Interpretation of the Halstead-Reitan neuropsychological test battery: A casebook approach. New York: Grune & Stratton.
Halstead, W.C. (1947). Brain and intelligence: A quantitative study of the frontal lobes. Chicago: University of Chicago Press.
Lezak, M.D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press.
Lord, F.M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68, 304–305.
Nies, K.J., & Sweet, J.J. (1994). Neuropsychological assessment and malingering: A critical review of past and present strategies. Archives of Clinical Neuropsychology, 9, 501–552.
Nunnally, J.C. (1978). Psychometric theory (2nd ed.). New York: McGraw Hill.
Peter, J.P., Churchill, G.A., & Brown, T.J. (1993). Caution in the use of difference scores. Journal of Consumer Research, 19, 655–662.
Purdue Research Foundation. (1948). Examiner’s manual for the Purdue Pegboard. Chicago: Science Research Associates.
Reddon, J.R. (1987). Fisher’s tanrr−1 transformation of the correlation coefficient and a test for complete independence in a multivariate normal population. Journal of Educational Statistics, 12, 294–300.
Reddon, J.R., Gill, D.G., Gauk, S.E., & Maerz, M.D. (1988). Purdue Pegboard: Test-retest estimates. Perceptual and Motor Skills, 66, 503–506.
Reddon, J.R., Jackson, D.N., & Schopflocher, D. (1985). Distribution of the determinant of the sample correlation matrix: Monte Carlo type one error rates. Journal of Educational Statistics, 10, 384–388.
Royce, J.R., Yeudall, L.T., & Bock, C. (1976). Factor analytic studies of human brain damage: I. First and second-order factors and their brain correlates. Multivariate Behavioral Research, 11, 381–418.
Scheffé, H.A. (1953). A method for judging all contrasts in the analysis of variance. Biometrika, 40, 87–104.
Selby, M.J., Airy-Eggertsen, A.S., & Laver, G.D. (1998). Comparison of neuropsychological test performance in forensic and nonforensic populations. Journal of Forensic Psychology, 16, 45–58.
Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15, 72–101.
Stanley, J. C. (1971). Reliability. In R.L. Thomdike (Ed.), Educational measurement (2nd ed., pp. 356–442). Washington, D.C.: American Council on Education.
Swiercinsky, D. (1978). Manual for the adult neuropsychological evaluation. Springfield: Charles C. Thomas.
Wilks, S.S. (1932). Certain generalizations in the analysis of variance. Biometrika, 24, 471–494.
Williams, R. H., & Zimmerman, D. W. (1996). Are simple gain scores obsolete? Applied Psychological Measurement, 20, 59–69.
Yeudall, L.T., Fromm, D., Reddon, J.R., & Stefanyk, W.O. (1986). Normative data stratified by age and sex for 12 neuropsychological tests. Journal of Clinical Psychology, 42, 918–946.
Yeudall, L. T., Reddon, J. R., Gill, D. M., & Stefanyk, W. O. (1987). Normative data for the Halstead-Reitan Neuropsychological Tests stratified by age and sex. Journal of Clinical Psychology, 43, 346–367.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Reddon, J.R., Vander Veen, S. Difference scores: A caveat illustrated with neuropsychological measures. Curr Psychol 24, 60–67 (2005). https://doi.org/10.1007/s12144-005-1004-y
Issue Date:
DOI: https://doi.org/10.1007/s12144-005-1004-y