Behavior Research Methods

, Volume 39, Issue 4, pp 940–949 | Cite as

Development and validation of a World-Wide-Web-based neurocognitive assessment battery: WebNeuro

  • Steven M. Silverstein
  • Sarah Berten
  • Patricia Olson
  • Robert Paul
  • Leanne M. Williams
  • Nicholas Cooper
  • Evian Gordon


Assessment of neurocognitive functioning is a critical task in many clinical, educational, service, and industrial settings. We report on descriptive and validation data of a new, World-Wide-Web-based, comprehensive battery of neurocognitive functioning, WebNeuro, that can be used in both applied and research contexts. Fifty healthy control participants completed both WebNeuro, and an established non-Internet-based computerized cognitive assessment battery, IntegNeuro, that uses a touchscreen platform. Results indicated comparability across the two batteries, in terms of critical single test scores, factor analysis derived indices, overall performance scores, and sex differences. These results support the validity of WebNeuro as a neurocognitive assessment measure. Advantages of its use in applied and research settings are discussed.


Schizophrenia Work Memory Capacity Colored Word Continuous Performance Test Healthy Control Participant 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. American Psychological Association (1986).Guidelines for computer-based tests and interpretations. Washington, D.C.: American Psychological Association.Google Scholar
  2. Baddeley, A. D., Emslie, H., &Nimmo-Smith, I. (1993). The Spotthe-Word test: A robust estimate of verbal intelligence based on lexical decision.British Journal of Clinical Psychology,32, 55–65.PubMedGoogle Scholar
  3. Baker, E. L., Letz, R. E., Fidler, A. T., Shalat, S., Plantamura, D., &Lyndon, M. A. (1985). Computer-based neurobehavioral evaluation system for occupational and environmental epidemiology: Methodology and validation studies.Neurobehavioral Toxicology & Teratology,7, 369–377.Google Scholar
  4. Bobholz, J. A., &Rao, S. M. (2003). Cognitive dysfunction in multiple sclerosis: A review of recent developments.Current Opinion in Neurology,16, 283–288.CrossRefPubMedGoogle Scholar
  5. Braff, D. L., &Saccuzzo, D. P. (1982). Effect of antipsychotic medication on speed of information processing in schizophrenic patients.American Journal of Psychiatry,139, 1127–1130.PubMedGoogle Scholar
  6. Brandt, J., Shpritz, B., Codori, A. M., Margolis, R., &Rosenblatt, A. (2002). Neuropsychological manifestations of the genetic mutation for Huntington’s disease in presymptomatic individuals.Journal of the International Neuropsychological Society,8, 918–924.CrossRefPubMedGoogle Scholar
  7. Calev, A., Korin, Y., Kugelmass, S., &Lerer, B. (1987). Performance of chronic schizophrenics on matched word and design recall tasks.Biological Psychiatry,22, 699–709.CrossRefPubMedGoogle Scholar
  8. Chan, R. C., Chen, E. Y., Cheung, E. F., Chen, R. Y., &Cheung, H. K. (2004). Problem-solving ability in chronic schizophrenia. A comparison study of patients with traumatic brain injury.European Archives for Psychiatry & Clinical Neuroscience,254, 236–241.Google Scholar
  9. Cohen, J. D., Barch, D. M., Carter, C., &Servan-Schreiber, D. (1999). Context-processing deficits in schizophrenia: Converging evidence from three theoretically motivated cognitive tasks.Journal of Abnormal Psychology,108, 120–133.CrossRefPubMedGoogle Scholar
  10. Cohen, J. D., &Servan-Schreiber, D. (1992). Context, cortex, and dopamine: A connectionist approach to behavior and biology in schizophrenia.Psychological Review,99, 45–777.CrossRefPubMedGoogle Scholar
  11. Cysique, L. A., Maruff, P., Darby, D., &Brew, B. J. (2006). The assessment of cognitive function in advanced HIV-1 infection and AIDS dementia complex using a new computerised cognitive test battery.Archives of Clinical Neuropsychology,21, 185–194.CrossRefPubMedGoogle Scholar
  12. Devivo, K., Rothland, J., Price, L., &Fein, G. (1999). Computerized assessment of arithmetic computation skills with MicroCog.Journal of the International Neuropsychological Society,3, 199–200.Google Scholar
  13. Docherty, N. M., Hawkins, K. A., Hoffman, R. E., Quinlan, D., Rakfeldt, J., &Sledge, W. H. (1996). Working memory, attention, and communication disturbances in schizophrenia.Journal of Abnormal Psychology,105, 212–219.CrossRefPubMedGoogle Scholar
  14. Elwood, R. W. (2001). MicroCog: Assessment of cognitive functioning.Neuropsychology Review,11, 89–100.CrossRefPubMedGoogle Scholar
  15. Gold, J. M., Queern, C., Iannone, V. N., &Buchanan, R. W. (1999). Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia. I: Sensitivity, reliability, and validity.American Journal of Psychiatry,156, 1944–1950.PubMedGoogle Scholar
  16. Goldberg, T. E., Weinberger, D. R., Berman, K. F., Pliskin, N. H., &Podd, M. H. (1987). Further evidence for dementia of the prefrontal type in schizophrenia? A controlled study of teaching the Wisconsin Card Sorting Test.Archives of General Psychiatry,44, 1008–1014.PubMedGoogle Scholar
  17. Golden, C. J. (1978).Stroop color and word test: A manual for clinical and experimental uses. Wood Dale, IL: Stoelting.Google Scholar
  18. Gordon, E. (2003). Integrative neuroscience.Neuropsychopharmacology,28(Suppl. 1), 2–8.CrossRefGoogle Scholar
  19. Gordon, E., Cooper, N., Rennie, C., Hermens, D., &Williams, L. M. (2005). Integrative neuroscience: The role of a standardized database.Clinical EEG & Neuroscience,36, 64–75.Google Scholar
  20. Green, M. F. (1998).Schizophrenia from a neurocognitive perspective. Boston: Allyn & Bacon.Google Scholar
  21. Green, M. F. (2006). Cognitive impairment and functional outcome in schizophrenia and bipolar disorder.Journal of Clinical Psychiatry,67 (Suppl. 9), 3–8.PubMedGoogle Scholar
  22. Green M. F., &Nuechterlein, K. H. (2004). The MATRICS initiative: Developing a consensus cognitive battery for clinical trials.Schizophrenia Research,72, 1–3.CrossRefPubMedGoogle Scholar
  23. Green, M. F., Olivier, B., Crawley, J., Penn, D., &Silverstein, S. (2005). Social cognition in schizophrenia: Recommendations from the MATRICS New Approaches Conference.Schizophrenia Bulletin,31, 882–887.CrossRefPubMedGoogle Scholar
  24. Hobart, M. P., Goldberg, R., Bartko, J. J., &Gold, J. M. (1999). Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia. II: Convergent/discriminant validity and diagnostic group comparisons.American Journal of Psychiatry,156, 1951–1957.PubMedGoogle Scholar
  25. Keefe, R. S., Goldberg, T. E., Harvey, P. D., Gold, J. M., Poe, M. P., &Coughenour, L. (2004). The Brief Assessment of Cognition in Schizophrenia: Reliability, sensitivity, and comparison with a standard neurocognitive battery.Schizophrenia Research,68, 283–297.CrossRefPubMedGoogle Scholar
  26. Kane, R. L., &Kay, G. G. (1992). Computerized assessment in neuropsychology: A review of tests and test batteries.Neuropsychology Review,3, 1–117.CrossRefPubMedGoogle Scholar
  27. Knight, R. A., &Silverstein, S. M. (1998). The role of cognitive psychology in guiding research on cognitive deficits in schizophrenia. In M. F. Lenzenweger & R. H. Dworkin (Eds.),Origins and development of schizophrenia: Advances in experimental psychopathology (pp. 247–295). Washington, DC: American Psychological Association.CrossRefGoogle Scholar
  28. Kozma-Wiebe, P., Silverstein, S. M., Fehér, Á., Kovács, I., Ulhaas, P., &Wilkniss, S. (2006). Development of a World-Wide Web based contour integration test: Reliability and validity.Computers in Human Behavior,22, 971–980.CrossRefGoogle Scholar
  29. Krantz, J. H., &Dalal, R. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.),Psychological experiments on the Internet (pp. 35–60). San Diego: Academic Press.CrossRefGoogle Scholar
  30. Lovibond, P. F., &Lovibond, S. H. (1995). The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories.Behaviour Research & Therapy,33, 335–343.CrossRefGoogle Scholar
  31. Luce, K. H., Winzelberg, A. J., Das, S., Osborne, M. I., Bryson, S. W., &Taylor, C. B. (2007). Reliability of self-report: Paper versus online administration.Computers in Human Behavior,23, 1384–1389.CrossRefGoogle Scholar
  32. Mindt, M. R., Cherner, M., Marcotte, T. D., Moore, D. J., Bentley, H., Esquivel, M. M., et al. (2003). The functional impact of HIV-associated neuropsychological impairment in Spanish-speaking adults: A pilot study.,Journal of Clinical & Experimental Neuropsychology,25, 122–132.CrossRefGoogle Scholar
  33. Morice, R., &Delehunty, A. (1996). Frontal/executive impairments in schizophrenia.Schizophrenia Bulletin,22, 125–137.PubMedGoogle Scholar
  34. Morris, R. G., Evenden, J. L., Sahakian, B. J., &Robbins, T. W. (1986). Computer-aided assessment of dementia: Comparative studies of neuropsychological deficits in Alzheimer-type dementia and Parkinson’s disease. In. S. M. Stahl, S. D. Iversen, & E. C. Goodman (Eds.),Cognitive neurochemistry (pp. 21–36). Oxford: Oxford University Press.Google Scholar
  35. Nuechterlein, K. H. (1991). Vigilance in schizophrenia and related disorders. In S. R. Steinhauer, J. H. Gruzelier, & J. Zubin (Eds.),Handbook of schizophrenia: Neuropsychology, psychophysiology, and information processing (Vol 5, pp. 393–433). Amsterdam: Elsevier.Google Scholar
  36. Palmer, B. W., Heaton, R. H., Paulsen, J. S., Kuck, J., Braff, D., &Harris, M. J. (1997). Is it possible to be schizophrenic yet europsychologically ormal?Neuropsychology,11, 437–446.CrossRefPubMedGoogle Scholar
  37. Park, S., &Holzman, P. S. (1992). Schizophrenics show spatial working memory deficits.Archives of General Psychiatry,49, 975–981.PubMedGoogle Scholar
  38. Paul, R. H., Lawrence, J., Williams, L. M., Richard, C. C., Cooper, N., &Gordon, E. (2005). Preliminary validity of “Integ-Neuro”: a new computerized battery of neurocognitive tests.International Journal of Neuroscience,115, 1549–1567.CrossRefPubMedGoogle Scholar
  39. Reitan, R. (1958). Validity of the Trail Making Test as an indicator of organic brain damage.Perceptual & Motor Skills,8, 271–276.CrossRefGoogle Scholar
  40. Silverstein, S. M. (1997). Information processing, social cognition, and psychiatric rehabilitation of schizophrenia.Psychiatry,60, 327–340.PubMedGoogle Scholar
  41. Silverstein, S. M., Light, G. A., &Palumbo, D. R. (1998). The Sustained Attention Test: A measure of cognitive dysfunction.Computers in Human Behavior,14, 463–475.CrossRefGoogle Scholar
  42. Silverstein, S. M., Matteson, S., &Knight, R. A. (1996). Reduced top-down influence in auditory perceptual organization in schizophrenia.Journal of Abnormal Psychology,105, 663–667.CrossRefPubMedGoogle Scholar
  43. Silverstein, S. M., Osborn, L. M., &Palumbo, D. R. (1998). Rey-Osterrieth Complex Figure Test performance in acute, chronic, and remitted schizophrenia patients.Journal of Clinical Psychology,54, 985–994.CrossRefPubMedGoogle Scholar
  44. Silverstein, S. M., &Wilkniss, S. M. (2004). The future of cognitive rehabilitation of schizophrenia.Schizophrenia Bulletin,30, 679–692.PubMedGoogle Scholar
  45. Walsh, K. W. (1985).Understanding brain damage: A primer of neuropsychological evaluation. Edinburgh, Scotland: Churchill Livingstone.Google Scholar
  46. Wechsler, D. (1997).Wechsler Memory Scale (3rd ed.). San Antonio, TX: Harcourt Assessment.Google Scholar
  47. Williams, L. M., Simms E., Clark, C. R., Paul, R. H., Rowe, D., &Gordon, E. (2005). The test-retest reliability of a standardized neurocognitive and neurophysiological test battery: “Neuromarker”.International Journal of Neuroscience,115, 1605–1630.CrossRefPubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2007

Authors and Affiliations

  • Steven M. Silverstein
    • 1
  • Sarah Berten
    • 1
  • Patricia Olson
    • 2
  • Robert Paul
    • 3
  • Leanne M. Williams
    • 4
    • 5
  • Nicholas Cooper
    • 6
  • Evian Gordon
    • 4
    • 5
    • 6
  1. 1.Division of Schizophrenia ResearchUniversity of Medicine and Dentistry of New JerseyPiscataway
  2. 2.Northwestern UniversityEvanston
  3. 3.University of MissouriSt. Louis
  4. 4.Westmead HospitalSydneyAustralia
  5. 5.University of SydneySydneyAustralia
  6. 6.Brain Resource CompanySydneyAustralia

Personalised recommendations