Measuring Procedural Knowledge More Simply with a Single-Response Situational Judgment Test

Abstract

Purpose

This paper describes the development of a situational judgment test (SJT) based on single-response options developed directly from critical incidents and reports a study that tested the SJT’s concurrent validity against ratings of job performance.

Design/Methodology/Approach

Situational judgment test items were developed from critical incidents provided by administrators of volunteer agencies. Volunteers who worked at another agency completed the SJT and a self-report personality test. Supervisors rated their job performance on three dimensions.

Findings

Situational judgment test scores representing procedural knowledge about work effort were significantly correlated with ratings of work effort performance (r = .28). Conscientiousness was correlated with work effort knowledge (r = .26), but not with work effort performance (r = −.02).

Implications

These results provide some preliminary evidence that a single-response SJT format based upon critical incidents can produce valid measures of procedural knowledge and might be a useful alternative to the traditional multiple-response format.

Originality/Value

This paper presents a novel way to construct SJTs using single-response options that are less laborious to develop than the multiple-response options in traditional SJT formats. Results of the validity study suggest that this new single-response format can predict job performance and encourage further research on the viability of this approach.