Abstract
Even when relatively infrequent, deviant response sets, such as defensive and careless responding, can have remarkably robust effects on individual and group data and thereby distort clinical evaluations and research outcomes. Given such potential adverse impacts and the widespread use of self-report measures when appraising addictions and addictive behavior, the detection of deviant response sets is an important clinical and research objective. Using a video game questionnaire as an exemplar, we examined the capacity of individuals to manipulate questionnaire scores and the effectiveness of various items to detect defensive, careless, and random responding. Individuals who obtained elevated questionnaire results when instructed to respond honesty often reduced their scores to unremarkable levels under “fake good” instructions. Most types of items for detecting defensive responding were ineffective with a possible exception, although items for detecting random and careless responding seemed promising. Potential guides for item development and use are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We did change some of the anchor points on the King scale to mirror adaptations made in other video questionnaires. These changes would be problematic if one attempted exact comparisons to results on the original scale, but that was not our intent and any such differences do not relate to the research aims or outcomes reported here. Additionally, although the formal name of the scale is the Problem Video Game Playing Test, or PVGT, for the sake of convenience we will simply refer to it as the “King” scale.
Categories (a) and (b) were used as exclusionary criteria in the previously described examination of defensiveness items given the potential of such response sets to distort outcome under comparable study conditions. For the present analyses involving the detection of random and careless responding, these very participants who seemingly demonstrated deviant response sets were the focus of interest.
For purposes of clarity, the example assumes that none of the careless responders will also be a problem user (i.e., a true case), which is unlikely to be literally true. However, unless the rate of true cases is exceedingly high among careless responders who obtain elevated scores, a marked distortion in outcome will still result, although perhaps not quite as extreme as we have projected. Our basic intent here is to illustrate potential problems caused by non-detection of random responders, rather than to arrive at precise figures.
References
Baer, R. A., Ballenger, J., Berry, D. T. R., & Wetter, M. W. (1997). Detection of random-responding on the MMPI-A. Journal of Personality Assessment, 68, 139–151.
Beach, D. A. (1989). Identifying the random responder. Journal of Psychology, 123, 101–103.
Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., & Pidruzny, J. N. (2009). The development of the Game Engagement Questionnaire: a measure of engagement in video game-playing. Journal of Experimental Social Psychology, 45, 624–634.
Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, W. G., & Kaemmer, B. (2001). Minnesota Multiphasic Personality Inventory-2 (MMPI-2): Manual for administration, scoring, and interpretation (Rev. ed.). Minneapolis: University of Minnesota Press.
Crede, M. (2010). Random responding as a threat to the validity of effect size estimates in correlational research. Educational and Psychological Measurement, 70, 596–612.
Gallen, R. T., & Berry, D. T. R. (1997). Partially random MMPI-2 protocols: when are they interpretable? Assessment, 4, 61–68.
Gentile, D. (2009). Pathological video-game use among youth ages 8 to 18: a national study. Psychological Science, 20, 594–602.
Greene, R. L. (2011). The MMPI-2/MMPI-2-RF: An interpretive manual (3rd ed.). Boston: Allyn & Bacon.
Griffiths, M. D. (2010). The use of online methodologies in data collection for gambling and gaming addictions. International Journal of Mental Health and Addiction, 8, 8–20.
Hathaway, S. R., & McKinley, J. C. (1943). Manual for administering and scoring the MMPI. Minneapolis: University of Minnesota Press.
King, D. L., Delfabbro, P. H., & Zajac, I. T. (2011). Preliminary validation of a new clinical tool for identifying problem video game playing. International Journal of Mental Health and Addiction, 9, 72–87.
Kuss, D. J., & Griffiths, M. D. (2011). Internet gaming addiction: a systematic review of empirical research. International Journal of Mental Health and Addiction. doi:10.1007/s11469-011-9318-5. Advanced publication online.
Lemmens, J. S., Valkenburg, P. M., & Peter, J. (2009). Development and validation of a game addiction scale for adolescents. Media Psychology, 12, 77–95.
Rogers, R. (2008). Current status of clinical methods. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp. 391–410). New York: Guilford Press.
Stein, L. A. R., & Rogers, R. (2008). Denial and misreporting of substance abuse. In R. Rogers (Ed.), Clinical assessment of malingering and deception (3rd ed., pp. 87–108). New York: Guilford Press.
Salguero, R. A. T., & Moran, R. M. B. (2002). Measuring problem video game playing in adolescents. Addiction, 97, 1601–1606.
Sublette, V. A., & Mullan, B. (2012). Consequences of play: a systematic review of the effects of online gaming. International Journal of Mental Health and Addiction, 10, 3–23.
Weinstein, A. M. (2010). Computer and video game addiction—a comparison between game users and non-game users. The American Journal of Drug and Alcohol Abuse, 36, 268–276.
Young, K. S. (1998). Caught in the net: How to recognize the signs of internet addiction and a winning strategy for recovery. New York: Wiley.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Faust, K.A., Faust, D., Baker, A.M. et al. Refining Video Game Use Questionnaires for Research and Clinical Application: Detection of Problematic Response Sets. Int J Ment Health Addiction 10, 936–947 (2012). https://doi.org/10.1007/s11469-012-9390-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11469-012-9390-5