One measure of the impact of genetic counseling and testing (GCT) is the extent to which it fosters behavioral change that is consistent with mutation status. We describe and illustrate how two different signal detection methods, receiver operating characteristic (ROC) analysis and recursive partitioning, can be used in this context to evaluate the impact of GCT. We analyzed real screening behavior data obtained in the 12 months following GCT for Hereditary Nonpolyposis Colon Cancer (HNPCC) using these two different signal detection approaches. Each approach demonstrated that GCT had an impact on behavioral outcomes, and was effective in fostering behavioral outcomes appropriate to mutation status. The ROC approach demonstrated that GCT was effective because mutation positive and mutation negative individuals could be distinguished on the basis of the number of recommended screening behaviors. The recursive partitioning approach demonstrated that GCT was effective because there were generally high rates of adherence to screening guidelines among subjects. The recursive partitioning technique also identified four subgroups of subjects, each with distinct characteristics, for which tailored interventions could be developed to increase rates of adherence to screening guidelines. Signal detection methods are easily implemented and are useful techniques for evaluating the impact of GCT.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrykowski, M. A., Munn, R. K., & Studts, J. L. (1996). Interest in learning of personal genetic risk for cancer: A general population survey. Prev Med, 25, 527–536.
Berchuck, A., Cirisano, F., Lancaster, J. M., Schildkraut, J. M., Wiseman, R. W., Futreal, A., et al. (1996). Role of BRCA1 mutation screening in the management of familial ovarian cancer. Am J Obstet Gynecol, 175, 738–746.
Bronner, C. E., Baker, S. M., Morrison, P. T., Warren, G., Smith, L. G., Lescoe, M. K., et al. (1994). Mutation in the DNA mismatch repair gene homologue hMLH 1 is associated with hereditary non-polyposis colon cancer. Nature, 368, 258–261.
Centor, R. M. (1991). Signal detectability: The use of ROC curves and their analyses. Med Decis Making, 11, 102–106.
Chaliki, H., Loader, S., Levenkron, J. C., Logan-Young, W., Hall, W. J., & Rowley, P. T. (1995). Women’s receptivity to testing for a genetic susceptibility to breast cancer. Am J Public Health, 85, 1133–1135.
Esplen, M. J., Madlensky, L., Butler, K., McKinnon, W., Bapat, B., Wong, J., et al. (2001). Motivations and psychosocial impact of genetic testing for HNPCC. Am J Med Genet, 103, 9–15.
Faraggi, D., & Reiser, B. (2002). Estimation of the area under the ROC curve. Stat Med, 21, 3093–3106.
Fishel, R., Lescoe, M. K., Rao, M. R., Copeland, M. G., Jenkins, N. A., Garber, J., et al. (1993). The human mutator gene homolog MSH2 and its association with hereditary nonpolyposis colon cancer. Cell, 75, 1027–1038.
Hadley, D. H., Jenkins, J. F., Dimond, E., de Carvalho, M., Kirsch, I., & Palmer, C. G. S. (2004). Colon cancer screening practices after genetic counseling and testing for hereditary non-polyposis colorectal cancer. J Clin Oncol, 22, 39–44.
Hadley, D. H., Jenkins, J. F., Dimond, E., Nakahara, K., Grogan, L., Liewehr, D. J., et al. (2003). Genetic counseling and testing in families with hereditary nonpolyposis colorectal cancer. Arch Intern Med, 163(5), 573–582.
Hanley, J. A. (1989). Receiver operating characteristic (ROC) methodology: The state of the art. Crit Rev Diagn Imaging, 29, 307–335.
Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143, 29–36.
Julian-Reynier, C., Eisinger, F., Vennin, P., Chabal, F., Aurran, Y., Nogues, C., et al. (1996). Attitudes towards cancer predictive testing and transmission of information to the family. J Med Genet, 33, 731–736.
Kiernan, M., King, A. C., Kraemer, H. C., Stefanick, M. L., & Killen, J. D. (1998). Characteristics of successful and unsuccessful dieters: An application of signal detection methodology. Ann Behav Med, 20, 1–6.
Kiernan, M., Kraemer, H. C., Winkleby, M. A., King, A. C., & Barr Taylor, C. (2001). Do logistic regression and signal detection identify different subgroups at risk? Implications for the design of tailored interventions. Psychol Methods, 6, 35–48.
King, A. C., Kiernan, M., Oman, R. F., Kraemer, H. C., Hull, M., & Ahn, D. (1997). Can we identify who will adhere to long-term physical activity? Signal detection methodology as a potential aid to clinical decision making. Health Psychol, 16, 380-389.
Kraemer, H. C. (1988). Assessment of 2×2 associations: Generalization of signal-detection methodology. Am Stat, 42, 37–49.
Kraemer, H. C. (1992). Evaluating Medical Tests: Objective and Quantitative Guidelines. Newbury Park, CA: Sage Publications, Inc.
Lerman, C., Biesecker, B., Benkendorf, J. L., Kerner, J., Gomez-Cominero, A., Hughes, C., et al. (1997). Controlled trial of pretest education approaches to enhance informed decision-making for BRCA1 gene testing. J Nat Cancer Inst, 89, 148–157.
McNeil, B. J., Keeler, E., & Adelstein, S. J. (1975). Primer on certain elements of medical decision making. NEJM, 293, 211–215.
Miki, Y., Swensen, J., Shattuck-Eidens, D., Futreal, P. A., Harshman, K., Tavtigian, S., et al. (1994). A strong candidate for the breast and ovarian cancer susceptibility gene BRCA1. Science, 266, 66–71.
Peters, J. A., & Stopfer, J. E. (1996). Role of the genetic counselor in familial cancer. Oncology, 10, 159–166.
Radloff, L. S. (1977). The CES-D Scale: A self-report depression scale for research in the general population. Appl Psychol Meas, 1, 385–401.
Richards, M. P. M., Hallowell, N., Green, J. M., Murton, F., & Statham, H. (1995). Counseling families with hereditary breast and ovarian cancer: A psychosocial perspective. J Genet it Counsel, 4, 219–233.
Solomon, C. H., Pho, L. N., & Burt, R. W. (2002). Current status of genetic testing for colorectal cancer susceptibility. Oncology, 16(2), 161–171.
Sox, H. C., Blatt, M., Higgins, M. C., & Marton, K. I. (1988). Medical decision making. Boston: Butterworth-Heinemann.
Struewing, J. P., Lerman, C., Kase, R. G., Giambarresi, T. R., & Tucker, M. A. (1995). Anticipated uptake and impact of genetic testing in hereditary breast and ovarian cancer families. Cancer Epidemiol Biomarkers Prev, 4, 169–173.
Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy. Their ROCs and implied models. Psychol Bull, 99, 100–117.
Swets, J. A., & Pickett, R. M. (1982). Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press.
Tavtigian, S. V., Simard, J., Rommens, J., Couch, F., Shattuck-Eidens, D., Neuhausen, S., et al. (1996). The complete BRCA2 gene and mutations in chromosome 13q-linked kindreds. Nature Genet, 12, 333–337.
Taylor, K. M., & Kelner, M. J. (1996). The emerging role of the physician in genetic counseling and testing for heritable breast, ovarian, and colon cancer. Can Med Assoc Jl, 154, 1155–1158.
Wong, N., Lasko, D., Rabelo, R., Pinsky, L., Gordon, P. H., & Foulkes, W. (2001). Genetic counseling and interpretation of genetic tests in familial adenomatous polyposis and hereditary nonpolyposis colorectal cancer. Dis Colon Rectum, 44, 271–279.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Palmer, C.G., Hadley, D.W. Evaluating the Impact of Genetic Counseling and Testing with Signal Detection Methods. J Genet Counsel 14, 17–27 (2005). https://doi.org/10.1007/s10897-005-1497-4
Issue Date:
DOI: https://doi.org/10.1007/s10897-005-1497-4