Skip to main content

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

Abstract

The computerized adaptive form of cognitive diagnostic testing, CD-CAT, has gained increasing attention in the domain of personalized measurements for its ability to categorize individual mastery status of fine-grained attributes more accurately and efficiently through administering items tailored to one’s ability progressively. How to select the next item based on previous response(s) is crucial for the success of CD-CAT. Previous item selection strategies for CD-CAT have often followed a greedy or semi-greedy approach, which makes it difficult to strike a balance between diagnostic performance and item bank utilization. To address this issue, this study takes a graph perspective and transforms the item selection problem in CD-CAT into a path-searching problem, in which paths refer to possible test construction and nodes refer to individual items. A heuristic function is defined to predict the prospect of a path, indicating how well the corresponding test can diagnose the current examinee. Two search mechanisms with different biases towards item exposure control are proposed to approximate the optimal path with the best prospect. The first unused item on the resulting path is selected as the next item. The above components compose a novel CD-CAT item selection framework based on heuristic search. Simulation studies are conducted under a variety of conditions regarding bank designs, bank-quality conditions, and testing scenarios. The results are compared with different types of classic item selection strategies in CD-CAT, showing that the proposed framework can enhance bank utilization at a smaller cost of diagnostic performance.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

Download references

Acknowledgements

This work was supported by the Guangdong Basic and Applied Basic Research Foundation (Nos. 2021A1515010844, 2023A1515011349).

Author information

Authors and Affiliations

Authors

Contributions

X. Cao and Y. Lin conceived the present idea, designed the algorithms, and organized the experiments. X. Cao implemented the algorithms, performed the experiments, and analyzed the results. All authors participated in result discussions and contributed to the writing and/or revision of the manuscript.

Corresponding author

Correspondence to Ying Lin.

Ethics declarations

Consent to publication

All the authors gave their consent to the publication of this work.

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open practices statement

The data and code of this study are available in the Open Science Framework (OSF) (https://osf.io/kczfs/?view_only=d3b07f35cc144bb1940836f3937b6baa). None of the experiments were preregistered.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 12401 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, X., Lin, Y., Liu, D. et al. Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework. Behav Res (2023). https://doi.org/10.3758/s13428-023-02228-9

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.3758/s13428-023-02228-9

Keywords