Abstract
In academic courses in which one task for the students is to understand empirical methodology and the nature of scientific inquiry, the ability of students to create and implement their own experiments allows them to take intellectual ownership of, and greatly facilitates, the learning process. The Psychology Experiment Authoring Kit (PEAK) is a novel spreadsheet-based interface allowing students and researchers with rudimentary spreadsheet skills to create cognitive and cognitive neuroscience experiments in minutes. Students fill in a spreadsheet listing of independent variables and stimuli, insert columns that represent experimental objects such as slides (presenting text, pictures, and sounds) and feedback displays to create complete experiments, all within a single spreadsheet. The application then executes experiments with centisecond precision. Formal usability testing was done in two stages: (1) detailed coding of 10 individual subjects in one-on-one experimenter/subject videotaped sessions and (2) classroom testing of 64 undergraduates. In both individual and classroom testing, the students learned to effectively use PEAK within 2 h, and were able to create a lexical decision experiment in under 10 min. Findings from the individual testing in Stage 1 resulted in significant changes to documentation and training materials and identification of bugs to be corrected. Stage 2 testing identified additional bugs to be corrected and new features to be considered to facilitate student understanding of the experiment model. Such testing will improve the approach with each semester. The students were typically able to create their own projects in 2 h.
Article PDF
Similar content being viewed by others
References
Bates, T. C., &D’Oliveiro, L. (2003). PsyScript: A Macintosh application for scripting experiments.Behavior Research Methods, Instruments, & Computers,35, 565–576.
Chute, D. L. (1993). MacLaboratory for Psychology: Successes, failures, economics, and outcomes over its decade of development.Behavior Research Methods, Instruments, & Computers,25, 180–188.
Cohen, J., MacWhinney, B., Flatt, M., &Provost, J. (1993). PsyScope: An interactive graphic system for designing and controlling experiments in the psychology laboratory using Macintosh computers.Behavior Research Methods, Instruments, & Computers,25, 257–271.
Cooper, A. (1995).About face: The essentials of user interface design. New York: Wiley.
Cox, R., Hulme, C., &Brown, G. D. A. (1992). STM Experimenter: Using HyperCard and MacRecorder in short-term memory experiments.Behavior Research Methods, Instruments, & Computers,24, 575–579.
Dutta, A. (1995). Experimental RunTime System: Software for developing and running reaction time experiments on IBM-compatible PCs.Behavior Research Methods, Instruments, & Computers,27, 516–519.
Eberhardt, S. P., Neverov, M., &Haneef, O. (1997). RunScript: An extendable object-oriented program for computer-controlled psychology experiments.Behavior Research Methods, Instruments, & Computers,29, 313–321.
Eschman, A., St. James, J., Schneider, W., &Zuccolotto, A. (2005). PsychMate: Providing psychology majors the tools to do real experiments and learn empirical methods.Behavior Research Methods, Instruments, & Computers,37, 301–311.
Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing.Behavior Research Methods, Instruments, & Computers,35, 379–383.
Forster, K. I., &Forster, J. C. (2003). DMDX: A Windows display program with millisecond accuracy.Behavior Research Methods, Instruments, & Computers,35, 116–124.
Fox, L. A., Shor, R. E., &Steinman, R. J. (1971). Semantic gradients and interference in naming color, spatial direction, and numerosity.Journal of Experimental Psychology,91, 59–65.
Haussmann, R. E. (1992). Tachistoscopic presentation and millisecond timing on the IBM PC/XT/AT and PS/2: A Turbo Pascal unit to provide general-purpose routines for CGA, Hercules, EGA, and VGA monitors.Behavior Research Methods, Instruments, & Computers,24, 303–310.
Hawley, K. J. (1991). PsyExper: Another experimental generation sys tem for the IBM PC.Behavior Research Methods, Instruments, & Computers,23, 155–159.
Haxby, J. V., Parasuraman, R., Lalonde, F., &Abboud, H. (1993). SuperLab: General-purpose Macintosh software for human experimental psychology and psychological testing.Behavior Research Methods, Instruments, & Computers,25, 400–405.
Hunt, S. M. J. (1994). MacProbe: A Macintosh-based experimenter’s workstation for the cognitive sciences.Behavior Research Methods, Instruments, & Computers,26, 345–351.
Kessels, R. P. C., Postma, A., &deHaan, E. H. F. (1999). Object Relocation: A program for setting up, running, and analyzing experiments on memory for object locations.Behavior Research Methods, Instruments, & Computers,31, 423–428.
Norman, D. A. (2002).The design of everyday things. New York: Basic Books.
Pallier, C., Dupoux, E., &Jeannin, X. (1997). EXPE: An expandable programming language for on-line psychological experiments.Behavior Research Methods, Instruments, & Computers,29, 322–327.
Palya, W. L., &Walter, D. E. (1993). A powerful, inexpensive experiment controller or IBM PC interface and experiment control language.Behavior Research Methods, Instruments, & Computers,25, 127–136.
Palya, W. L., Walter, D. E., &Chu, J. Y. M. (1995). An inexpensive 1-millisecond experiment control interface for IBM PCs and its userfriendly control language.Behavior Research Methods, Instruments, & Computers,27, 129–130.
Posner, M. I., Snyder, C. R. R., &Davidson, B. J. (1980). Attention and the detection of signals.Journal of Experimental Psychology: General,109, 160–174.
Pulkin, B. V. (1996). Programming without programming: The system Butterfly for professional psychologists.Behavior Research Methods, Instruments, & Computers,28, 577–583.
Rosinski, R. R. (1977). Picture-word interference is semantically based.Child Development,48, 643–647.
Schneider, W. (1989). Enhancing a standard experimental delivery system (MEL) for advanced psychological experimentation.Behavior Research Methods, Instruments, & Computers,21, 240–244.
Schneider, W., Eschman, A., & Zuccolotto, A. (2002).E-Prime: A user’s guide. Pittsburgh: Psychology Software Tools.
Schneider, W., &Scholz, K. W. (1973). Requirements for minicomputer operating systems for human experimentation and an implementation on a 4K PDP-8 computer.Behavior Research Methods, Instruments, & Instrumentation,5, 173–177.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions.Journal of Experimental Psychology,18, 643–662.
Toma, R. J., &Tsao, Y.-C. (1985). Interference effects in the picture-word Stroop task.Perceptual & Motor Skills,61, 223–228.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by NIH Grant 1R43MH072020-01, Managing Complexity in Psychology Experiment Generation. We thank Erik Reichle, Julie Fiez, and their students at the University of Pittsburgh for participating in the usability study. PEAK and PsychMate are products of Psychology Software Tools, Inc., 2050 Ardmore Boulevard, Suite 200, Pittsburgh, PA 15221 (www.pstnet.com). The materials used to assess PEAK are available to researchers upon request for testing ease of use and to illustrate the staged testing procedure described. Components of the PEAK interface are under patent application. One or more of the authors has a direct financial interest in some of the software or hardware described in this article.
Rights and permissions
About this article
Cite this article
Schneider, W., Bolger, D.J., Eschman, A. et al. Psychology Experiment Authoring Kit (PEAK): Formal usability testing of an easy-to-use method for creating computerized experiments. Behavior Research Methods 37, 312–323 (2005). https://doi.org/10.3758/BF03192699
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.3758/BF03192699