Skip to main content

Multiple Equating of Separate IRT Calibrations

Abstract

When test forms are calibrated separately, item response theory parameters are not comparable because they are expressed on different measurement scales. The equating process includes the conversion of item parameter estimates on a common scale and the determination of comparable test scores. Various statistical methods have been proposed to perform equating between two test forms. This paper provides a generalization to multiple test forms of the mean-geometric mean, the mean-mean, the Haebara, and the Stocking–Lord methods. The proposed methods estimate simultaneously the equating coefficients that permit the scale transformation of the parameters of all forms to the scale of the base form. Asymptotic standard errors of the equating coefficients are derived. A simulation study is presented to illustrate the performance of the methods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

References

  • Baldwin, P. (2013). On mean-sigma estimators and bias. British Journal of Mathematical and Statistical Psychology, 66, 277–289. doi:10.1111/j.2044-8317.2012.02048.x.

    Article  PubMed  Google Scholar 

  • Battauz, M. (2013). IRT test equating in complex linkage plans. Psychometrika, 78, 464–480. doi:10.1007/s11336-012-9316-y.

    Article  PubMed  Google Scholar 

  • Battauz, M. (2015). equateIRT: An R package for IRT test equating. Journal of Statistical Software, 68, 1–22. doi:10.18637/jss.v068.i07.

    Article  Google Scholar 

  • Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459. doi:10.1007/BF02293801.

    Article  Google Scholar 

  • Deming, W. E., & Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. The Annals of Mathematical Statistics, 11, 427–444. doi:10.1214/aoms/1177731829.

    Article  Google Scholar 

  • Goodman, L. A. (1968). The analysis of cross-classified data: independence, quasi-independence and interactions in contingency tables with or without missing entries. Journal of the American Statistical Association, 63, 1091–1131. doi:10.1080/01621459.1968.10480916.

    Google Scholar 

  • Haberman, S. J. (2009). Linking parameter estimates derived from an item response model through separate calibrations. ETS Research Report Series, 2009, i-9. doi:10.1002/j.2333-8504.2009.tb02197.x.

  • Haberman, S. J., Lee, Y. H. & Qian, J. (2009). Jackknifing techniques for evaluation of equating accuracy . ETS Research Report Series, 2009, i-37. doi:10.1002/j.2333-8504.2009.tb02196.x.

  • Haebara, T. (1980). Equating logistic ability scales by a weighted least squares method. Japanese Psychological Research, 22, 144–149.

    Article  Google Scholar 

  • Kim, S., & Kolen, M. J. (2007). Effects on scale linking of different definitions of criterion functions for the IRT characteristic curve methods. Journal of Educational and Behavioral Statistics, 32, 371–397. doi:10.3102/1076998607302632.

    Article  Google Scholar 

  • Kolen, M. J., & Brennan, R. L. (2014). Test equating, scaling, and linking: methods and practices (3rd ed.). New York: Springer.

    Book  Google Scholar 

  • Lee, Y.-H., & Haberman, S. J. (2013). Harmonic regression and scale stability. Psychometrika, 78, 815–829. doi:10.1007/S11336-013-9337-1.

    Article  PubMed  Google Scholar 

  • Loyd, B. H., & Hoover, H. D. (1980). Vertical equating using the Rasch model. Journal of Educational Measurement, 17, 179–193. doi:10.1111/j.1745-3984.1980.tb00825.x.

    Article  Google Scholar 

  • Marco, G. L. (1977). Item characteristic curve solutions to three intractable testing problems. Journal of Educational Measurement, 14, 139–160. doi:10.1111/j.1745-3984.1977.tb00033.x.

    Article  Google Scholar 

  • Michaelides, M. P., & Haertel, E. H. (2014). Selection of common items as an unrecognized source of variability in test equating: A bootstrap approximation assuming random sampling of common items. Applied Measurement in Education, 27, 46–57. doi:10.1080/08957347.2013.853069.

    Article  Google Scholar 

  • Mislevy R. J. & Bock R. D. (1990). BILOG 3. Item analysis and test scoring with binary logistic models. Mooresville, IN: Scientific Software.

  • Ogasawara, H. (2000). Asymptotic standard errors of IRT equating coefficients using moments. Economic Review (Otaru University of Commerce), 51, 1–23.

    Google Scholar 

  • Ogasawara, H. (2001a). Item response theory true score equatings and their standard errors. Journal of Educational and Behavioral Statistics, 26, 31–50. doi:10.3102/10769986026001031.

    Article  Google Scholar 

  • Ogasawara, H. (2001b). Standard errors of item response theory equating/linking by response function methods. Applied Psychological Measurement, 25, 53–67. doi:10.1177/01466216010251004.

    Article  Google Scholar 

  • Ogasawara, H. (2003). Asymptotic standard errors of IRT observed-score equating methods. Psychometrika, 68, 193–211. doi:10.1007/BF02294797.

    Article  Google Scholar 

  • R Development Core Team. (2016). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.

  • Rizopoulos, D. (2006). ltm: An R package for latent variable modelling and item response theory analyses. Journal of Statistical Software, 17, 1–25. doi:10.18637/jss.v017.i05.

    Article  Google Scholar 

  • Stocking, M., & Lord, M. L. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201–210. doi:10.1177/014662168300700208.

    Article  Google Scholar 

  • van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. New York: Springer.

    Book  Google Scholar 

Download references

Acknowledgments

The author wishes to thank the Editor, the Associate Editor, and two anonymous reviewers for their helpful comments and suggestions that greatly improved this work. The author is grateful to Professor R. Bellio for his suggestions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Michela Battauz.

Appendices

Appendix 1: Partial Derivatives Necessary to Obtain the Asymptotic Standard Errors

(a) MM-GM Method

The derivatives in Equation (10), necessary to compute the covariance matrix of the equating coefficients and the synthetic item parameters with the MM-GM method, are given by

$$\begin{aligned}&\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{a}}^\top }=\mathrm {diag}(\hat{\varvec{\beta }}_1) ({\varvec{X}}_1^\top {\varvec{X}}_1)^{-1} {\varvec{X}}_1^\top \left( \mathrm {diag}(\hat{\varvec{a}}) \right) ^{-1}, \end{aligned}$$
(37)
$$\begin{aligned}&\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{b}}^\top }={{\varvec{0}}},\end{aligned}$$
(38)
$$\begin{aligned}&\frac{\partial \hat{\varvec{\beta }}_2}{\partial \hat{\varvec{a}}^\top }= \frac{\partial \hat{\varvec{\beta }}_2}{\partial \hat{\varvec{A}}^\top } \frac{\partial \hat{\varvec{A}}}{\partial \hat{\varvec{a}}^\top }= ({\varvec{X}}_2^\top {\varvec{X}}_2)^{-1} {\varvec{X}}_2^\top \mathrm {diag}(\hat{\varvec{b}}) {\varvec{T}}^\top \frac{\partial \hat{\varvec{A}}}{\partial \hat{\varvec{a}}^\top }, \end{aligned}$$
(39)
$$\begin{aligned}&\frac{\partial \hat{\varvec{\beta }}_2}{\partial \hat{\varvec{b}}^\top }=({\varvec{X}}_2^\top {\varvec{X}}_2)^{-1} {\varvec{X}}_2^\top \mathrm {diag}(\hat{\varvec{A}}_n), \end{aligned}$$
(40)

where \(\frac{\partial \hat{\varvec{A}}}{\partial \hat{\varvec{a}}^\top }\) is given in the first \(T-1\) rows of \(\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{a}}^\top }\).

(b) MM-M Method

The derivatives of the equating coefficients \(A_t\), \(t=2,\dots ,T\), with respect to the estimated item discrimination parameters \(\hat{a}_{js}\), \(j=1,\dots ,v\), \(s=1,\dots ,T\), namely \(\frac{\partial \hat{A}_t}{\partial \hat{a}_{js}}\), cannot be found in closed form but can instead be determined numerically. These derivatives are included in the matrix \(\frac{\partial \hat{\varvec{A}}}{\partial \hat{\varvec{a}}^\top }\) that corresponds to the first \(T-1\) rows of \(\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{a}}^\top }\). The derivatives of the synthetic discrimination parameters \(\hat{a}_j^*\) with respect to the discrimination parameter estimates obtained from each calibration can be then found as follows:

$$\begin{aligned} \frac{\partial \hat{a}^*_j}{\partial \hat{a}_{jt}}= & {} \frac{1}{ \sum _{s\in U_j} \hat{A}_s } -\frac{\sum _{s\in U_j} \hat{a}_{js}}{\left( \sum _{s\in U_j} \hat{A}_s \right) ^2} \sum _{s\in U_j}\frac{\partial \hat{A}_s}{\partial \hat{a}_{jt}} , \end{aligned}$$
(41)
$$\begin{aligned} \frac{\partial \hat{a}^*_j}{\partial \hat{a}_{it}}= & {} -\frac{\sum _{s\in U_j} \hat{a}_{js}}{\left( \sum _{s\in U_j} \hat{A}_s \right) ^2} \sum _{s\in U_j}\frac{\partial \hat{A}_s}{\partial \hat{a}_{it}} , \quad \forall i\ne j. \end{aligned}$$
(42)

These derivatives form the matrix \(\frac{\partial \hat{\varvec{a}}^*}{\partial \hat{\varvec{a}}^\top }\) that corresponds to the last v rows of \(\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{a}}^\top }\). The derivatives \(\frac{\partial \hat{\varvec{\beta }}_1}{\partial \hat{\varvec{b}}^\top }\), \(\frac{\partial \hat{\varvec{\beta }}_2}{\partial \hat{\varvec{a}}^\top }\) and \(\frac{\partial \hat{\varvec{\beta }}_2}{\partial \hat{\varvec{b}}^\top }\) can then be determined as explained in Appendix 1(a) for the MM-GM method, using the appropriate matrices \(\frac{\partial \hat{\varvec{A}}}{\partial \hat{\varvec{a}}^\top }\) and \(\hat{\varvec{A}}_n\).

(c) MIRF and MTRF Methods

In order to obtain the partial derivatives necessary to compute the asymptotic standard errors of the equating coefficients, \(P^*_{mjt}\) will be written as follows:

$$\begin{aligned} P^*_{mjt}=\hat{c}_{jt}+(1-\hat{c}_{jt})\frac{\exp ( LP_{mjt} ) }{1+\exp ( LP_{mjt} )}, \end{aligned}$$
(43)

where

$$\begin{aligned} LP_{mjt}=D\frac{1}{u_j} \left( \sum _{\begin{array}{c} s\in U_j \\ s\ne t \end{array}}\frac{\hat{a}_{js}}{\hat{A}_s} \hat{A}_t + \hat{a}_{jt}\right) y_m -D\frac{1}{u_j} \sum _{s\in U_j}\frac{\hat{a}_{js}}{\hat{A}_s} \left( \frac{1}{u_j}\sum _{s\in U_j}(\hat{b}_{js} \hat{A}_s + \hat{B}_s)-\hat{B}_t \right) .\nonumber \\ \end{aligned}$$
(44)

In the following, all the derivatives entering in Equations (26), (27) (28), (32), (33), and (34) will be given as

$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{A}_t} = \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \cdot \frac{\partial LP_{mjt}}{\partial \hat{A}_t}, \end{aligned}$$
(45)

where

$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial LP_{mjt}}= & {} (P^*_{mjt}-\hat{c}_{jt})\left( 1-\frac{P^*_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}}\right) , \end{aligned}$$
(46)
$$\begin{aligned} \frac{\partial LP_{mjt}}{\partial \hat{A}_t}= & {} \frac{D}{u_j} \left[ \sum _{\begin{array}{c} s\in U_j \\ s\ne t \end{array}}\frac{\hat{a}_{js}}{\hat{A}_s} y_m + \frac{\hat{a}_{jt}}{\hat{A}_t^2}(\hat{b}_j^*-\hat{B}_t) I_{U_j}(t) -\hat{a}_j^*\hat{b}_{jt} I_{U_j}(t) \right] , \end{aligned}$$
(47)

and \(I_{U_j}(t)\) is an indicator function, which is 1 if \(t\in U_j\).

$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{A}_k} = \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \cdot \frac{\partial LP_{mjt}}{\partial \hat{A}_k}, \quad \forall k\ne t, \end{aligned}$$
(48)

where

$$\begin{aligned} \frac{\partial LP_{mjt}}{\partial \hat{A}_k}= & {} \frac{D}{u_j} \left[ -\frac{\hat{a}_{jk}}{\hat{A}_k^2} \hat{A}_t y_m + \frac{\hat{a}_{jk}}{\hat{A}_k^2}(\hat{b}_j^*-\hat{B}_t)-\hat{a}_j^*\hat{b}_{jk} \right] I_{U_j}(k), \end{aligned}$$
(49)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{B}_t}= & {} \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \cdot \frac{\partial LP_{mjt}}{\partial \hat{B}_t}, \end{aligned}$$
(50)

where

$$\begin{aligned} \frac{\partial LP_{mjt}}{\partial \hat{B}_t}= & {} D \hat{a}_j^* \left( 1-\frac{1}{u_j} I_{U_j}(t) \right) , \end{aligned}$$
(51)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{B}_k}= & {} \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \cdot \frac{\partial LP_{mjt}}{\partial \hat{B}_k}, \quad \forall k\ne t, \end{aligned}$$
(52)

where

$$\begin{aligned} \frac{\partial LP_{mjt}}{\partial \hat{B}_k}= & {} - D \hat{a}_j^* \frac{1}{u_j} I_{U_j}(k), \end{aligned}$$
(53)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_t \partial \hat{B}_k}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_k} \frac{\partial LP_{mjt}}{\partial \hat{B}_t} , \quad \forall k, \end{aligned}$$
(54)

where

$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_k} = \frac{\partial P^*_{mjt}}{\partial \hat{B}_k} -2 \frac{P^*_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}} \frac{\partial P^*_{mjt}}{\partial \hat{B}_k}. \end{aligned}$$
(55)

All other second derivatives of \(P^*_{mjt}\) with respect to \(LP_{mjt}\) and one of these variables \(\hat{B}_h, \hat{B}_t, \hat{A}_k, \hat{A}_h, \hat{A}_t, \hat{a}_{jk}, \hat{a}_{jh}, \hat{a}_{jt}, \hat{b}_{jk}, \hat{b}_{jh}, \hat{b}_{jt}\) are analogous and can be obtained by substituting \(\hat{B}_k\) with the appropriate variable in (55). The other derivatives entering in Equations (27), (28), (33) and (34) are

$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_t \partial \hat{A}_k}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_k} \frac{\partial LP_{mjt}}{\partial \hat{B}_t} - \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \frac{\hat{a}_{jk}}{\hat{A}_k^2} \left( 1-\frac{1}{u_j} I_{U_j}(t) \right) I_{U_j}(k), \quad \forall k, \end{aligned}$$
(56)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_k \partial \hat{B}_h}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_h} \frac{\partial LP_{mjt}}{\partial \hat{B}_k} , \quad \forall k \ne t, \; \forall h, \end{aligned}$$
(57)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_k \partial \hat{A}_h}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_h} \frac{\partial LP_{mjt}}{\partial \hat{B}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j^2} \frac{\hat{a}_{jh}}{\hat{A}_h^2} I_{U_j}(k) I_{U_j}(h) , \quad \forall k\ne t, \; \forall h, \end{aligned}$$
(58)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t^2}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_t} \frac{\partial LP_{mjt}}{\partial \hat{A}_t}+ \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ -2 \frac{\hat{a}_{jt}}{\hat{A}_t^3} (\hat{b}_j^*-\hat{B}_t) +2\frac{\hat{a}_{jt}\hat{b}_{jt}}{u_j \hat{A}_t^2} \right] I_{U_j}(t), \end{aligned}$$
(59)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{A}_k}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_k} \frac{\partial LP_{mjt}}{\partial \hat{A}_t}+ \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \nonumber \\&\quad \left[ - \frac{\hat{a}_{jk}}{\hat{A}_k^2} y_m + \frac{\hat{a}_{jt}\hat{b}_{jk}}{u_j \hat{A}_t^2} I_{U_j}(t) + \frac{\hat{a}_{jk}\hat{b}_{jt}}{u_j \hat{A}_k^2} I_{U_j}(t) \right] I_{U_j}(k) , \quad \forall k\ne t, \end{aligned}$$
(60)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{B}_t}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_t} \frac{\partial LP_{mjt}}{\partial \hat{A}_t}+ \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \frac{\hat{a}_{jt}}{\hat{A}_t^2} \left( \frac{1}{u_j}-1 \right) I_{U_j}(t), \end{aligned}$$
(61)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{B}_k}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_k} \frac{\partial LP_{mjt}}{\partial \hat{A}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j^2} \frac{\hat{a}_{jt}}{\hat{A}_t^2} I_{U_j}(t) I_{U_j}(k) , \quad \forall k\ne t, \end{aligned}$$
(62)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k^2}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_k} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \nonumber \\&\quad \left[ 2 \frac{\hat{a}_{jk}}{\hat{A}_k^3}\hat{A}_t y_m -2 \frac{\hat{a}_{jk}}{\hat{A}_k^3} (\hat{b}_j^*-\hat{B}_t) + 2 \frac{\hat{a}_{jk}\hat{b}_{jk}}{u_j \hat{A}_k^2} \right] I_{U_j}(k) , \quad \forall k\ne t, \end{aligned}$$
(63)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{A}_t}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{A}_t} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \nonumber \\&\quad \left[ - \frac{\hat{a}_{jk}}{\hat{A}_k^2} y_m + \frac{\hat{a}_{jk}\hat{b}_{jt}}{u_j \hat{A}_k^2} I_{U_j}(t) + \frac{\hat{a}_{jt}\hat{b}_{jk}}{u_j \hat{A}_t^2} I_{U_j}(t) \right] I_{U_j}(k) , \quad \forall k\ne t, \end{aligned}$$
(64)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{B}_h}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_h} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j^2} \frac{\hat{a}_{jk}}{\hat{A}_k^2} I_{U_j}(k) I_{U_j}(h) , \quad \forall k, \; h\ne t, \end{aligned}$$
(65)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{B}_t}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{B}_t} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \frac{\hat{a}_{jk}}{\hat{A}_k^2} \left( \frac{1}{u_j} I_{U_j}(t) - 1 \right) I_{U_j}(k), \quad \forall k\ne t, \qquad \end{aligned}$$
(66)
(67)
$$\begin{aligned} \frac{\partial P_{mjt}}{\partial \hat{c}_{jt}}= & {} 1- \frac{P_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}}, \end{aligned}$$
(68)
$$\begin{aligned} \frac{\partial P_{mjt}}{\partial \hat{a}_{jt}}= & {} (P_{mjt} - \hat{c}_{jt}) \left( 1- \frac{P_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}} \right) D (y_m -\hat{b}_{jt}), \end{aligned}$$
(69)
$$\begin{aligned} \frac{\partial P_{mjt}}{\partial \hat{b}_{jt}}= & {} -(P_{mjt} - \hat{c}_{jt}) \left( 1- \frac{P_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}} \right) D \hat{a}_{jt}, \end{aligned}$$
(70)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{c}_{jt}}= & {} 1- \frac{P^*_{mjt}-\hat{c}_{jt}}{1-\hat{c}_{jt}}, \end{aligned}$$
(71)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{a}_{jt}}= & {} \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ y_m - \frac{1}{\hat{A}_t} (\hat{b}^*_j - \hat{B}_t) \right] I_{U_j}(t), \end{aligned}$$
(72)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{a}_{jk}}= & {} \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ \frac{\hat{A}_t}{\hat{A}_k} y_m - \frac{1}{\hat{A}_k} (\hat{b}^*_j - \hat{B}_t) \right] I_{U_j}(k), \quad \forall k\ne t, \end{aligned}$$
(73)
$$\begin{aligned} \frac{\partial P^*_{mjt}}{\partial \hat{b}_{jk}}= & {} - \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \hat{a}^*_j \hat{A}_k I_{U_j}(k), \quad \forall k, \end{aligned}$$
(74)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_t \partial \hat{c}_{jt}}= & {} -\frac{1}{1-\hat{c}_{jt}} \frac{\partial P^*_{mjt}}{\partial \hat{B}_t}, \end{aligned}$$
(75)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_t \partial \hat{a}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{B}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \frac{1}{\hat{A}_k} \left( 1-\frac{1}{u_j} I_{U_j}(t) \right) I_{U_j}(k) , \quad \forall k, \end{aligned}$$
(76)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_t \partial \hat{b}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{B}_t}, \quad \forall k, \end{aligned}$$
(77)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_k \partial \hat{c}_{jt}}= & {} -\frac{1}{1-\hat{c}_{jt}} \frac{\partial P^*_{mjt}}{\partial \hat{B}_k}, \end{aligned}$$
(78)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_k \partial \hat{a}_{jh}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jh}} \frac{\partial LP_{mjt}}{\partial \hat{B}_k} - \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} D \frac{1}{u_j^2} \frac{1}{\hat{A}_h} I_{U_j}(k) I_{U_j}(h), \quad \forall k\ne t , \; \forall h, \end{aligned}$$
(79)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{B}_k \partial \hat{b}_{jh}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jh}} \frac{\partial LP_{mjt}}{\partial \hat{B}_k} , \quad \forall k\ne t , \; \forall h, \end{aligned}$$
(80)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{c}_{jt}}= & {} -\frac{1}{1-\hat{c}_{jt}} \frac{\partial P^*_{mjt}}{\partial \hat{A}_t}, \end{aligned}$$
(81)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{a}_{jt}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jt}} \frac{\partial LP_{mjt}}{\partial \hat{A}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ \frac{1}{\hat{A}_t^2} (\hat{b}_j^*-\hat{B}_t) -\frac{\hat{b}_{jt}}{u_j \hat{A}_t} \right] I_{U_j}(t), \end{aligned}$$
(82)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{a}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{A}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ \frac{1}{\hat{A}_k}y_m -\frac{\hat{b}_{jt}}{u_j \hat{A}_k} I_{U_j}(t) \right] I_{U_j}(k) , \quad \forall k\ne t , \nonumber \\\end{aligned}$$
(83)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{b}_{jt}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jt}} \frac{\partial LP_{mjt}}{\partial \hat{A}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ \frac{\hat{a}_{jt}}{u_j \hat{A}_t} - \hat{a}_j^* \right] I_{U_j}(t), \end{aligned}$$
(84)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_t \partial \hat{b}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{A}_t} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ \frac{\hat{a}_{jt} \hat{A}_k}{u_j \hat{A}_t^2} \right] I_{U_j}(t) I_{U_j}(k) , \quad \forall k\ne t , \end{aligned}$$
(85)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{c}_{jt}}= & {} -\frac{1}{1-\hat{c}_{jt}} \frac{\partial P^*_{mjt}}{\partial \hat{A}_k}, \quad \forall k\ne t, \end{aligned}$$
(86)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{a}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left[ -\frac{\hat{A}_t}{\hat{A}_k^2}y_m + \frac{1}{\hat{A}_k^2} (\hat{b}_j^*-\hat{B}_t) - \frac{\hat{b}_{jk}}{u_j \hat{A}_k} \right] I_{U_j}(k) , \nonumber \\&~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\quad \forall k\ne t , \nonumber \\\end{aligned}$$
(87)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{a}_{jh}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{a}_{jh}} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} - \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j^2} \frac{\hat{b}_{jk}}{\hat{A}_h} I_{U_j}(k) I_{U_j}(h) , \quad \forall k \ne t, \; \forall h \ne k , \end{aligned}$$
(88)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{b}_{jk}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jk}} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j} \left( \frac{\hat{a}_{jk}}{u_j \hat{A}_k} - \hat{a}^*_j \right) I_{U_j}(k) , \quad \forall k\ne t , \end{aligned}$$
(89)
$$\begin{aligned} \frac{\partial ^2 P^*_{mjt}}{\partial \hat{A}_k \partial \hat{b}_{jh}}= & {} \frac{\partial ^2 P^*_{mjt}}{\partial LP_{mjt} \partial \hat{b}_{jh}} \frac{\partial LP_{mjt}}{\partial \hat{A}_k} + \frac{\partial P^*_{mjt}}{\partial LP_{mjt}} \frac{D}{u_j^2} \frac{\hat{a}_{jk} \hat{A}_h}{\hat{A}_k^2} I_{U_j}(k) I_{U_j}(h), \quad \forall k\ne t, \; \forall h\ne k , \end{aligned}$$
(90)

The derivatives of the synthetic discrimination parameters with respect to the item parameter obtained from separate calibration can be found as follows:

$$\begin{aligned} \frac{\partial \hat{a}^*_j}{\partial \hat{a}_{jt}}= & {} \frac{1}{u_j} \left( \frac{1}{\hat{A}_t} I_{U_j}(t) - \sum _{s\in U_j} \frac{\hat{a}_{js}}{\hat{A}_s^2} \frac{\partial \hat{A}_s}{\partial \hat{a}_{jt}} \right) , \end{aligned}$$
(91)
$$\begin{aligned} \frac{\partial \hat{a}^*_j}{\partial \hat{a}_{it}}= & {} \frac{1}{u_j} \left( - \sum _{s\in U_j} \frac{\hat{a}_{js}}{\hat{A}_s^2} \frac{\partial \hat{A}_s}{\partial \hat{a}_{it}} \right) , \quad \forall i\ne j, \end{aligned}$$
(92)
$$\begin{aligned} \frac{\partial \hat{a}^*_j}{\partial \hat{b}_{it}}= & {} \frac{1}{u_j} \left( - \sum _{s\in U_j} \frac{\hat{a}_{js}}{\hat{A}_s^2} \frac{\partial \hat{A}_s}{\partial \hat{b}_{it}} \right) , \quad \forall i, \end{aligned}$$
(93)
$$\begin{aligned} \frac{\partial \hat{b}^*_j}{\partial \hat{a}_{it}}= & {} \frac{1}{u_j} \sum _{s\in U_j} \left( \hat{b}_{jt}\frac{\partial \hat{A}_s}{\partial \hat{a}_{it}} + \frac{\partial \hat{B}_s}{\partial \hat{a}_{it}} \right) , \quad \forall i, \end{aligned}$$
(94)
$$\begin{aligned} \frac{\partial \hat{b}^*_j}{\partial \hat{b}_{jt}}= & {} \frac{1}{u_j} \left[ \hat{A}_t I_{U_j}(t) + \sum _{s\in U_j} \left( \hat{b}_{js}\frac{\partial \hat{A}_s}{\partial \hat{b}_{jt}} + \frac{\partial \hat{B}_s}{\partial \hat{b}_{jt}} \right) \right] , \end{aligned}$$
(95)
$$\begin{aligned} \frac{\partial \hat{b}^*_j}{\partial \hat{b}_{it}}= & {} \frac{1}{u_j} \sum _{s\in U_j} \left( \hat{b}_{js}\frac{\partial \hat{A}_s}{\partial \hat{b}_{it}} + \frac{\partial \hat{B}_s}{\partial \hat{b}_{it}} \right) , \quad \forall i\ne j. \end{aligned}$$
(96)

Appendix 2: Proof of the Correspondence Between the MM-M Method for Two Forms and the Mean-Mean Method

When \(T=2\) the estimator of the equating coefficient \(A_2\) with the MM-M method is given by

$$\begin{aligned} \hat{A}_2=\frac{\sum _{j\in J_2}\hat{a}_{j2}}{\sum _{j\in J_2} \hat{a}^*_j}= \frac{\sum _{j\in J_2}\hat{a}_{j2}}{\sum _{j\in J_1\cap J_2} \frac{\hat{a}_{j1}+\hat{a}_{j2}}{1+\hat{A}_2}+ \sum _{j\in J_2\setminus J_1} \frac{\hat{a}_{j2}}{\hat{A}_2}}, \end{aligned}$$
(97)

from which we obtain

$$\begin{aligned} \hat{A}_2 \sum _{j\in J_1\cap J_2} \frac{\hat{a}_{j1}+\hat{a}_{j2}}{1+\hat{A}_2}+ \hat{A}_2 \sum _{j\in J_2\setminus J_1} \frac{\hat{a}_{j2}}{\hat{A}_2} = \sum _{j\in J_1\cap J_2}\hat{a}_{j2}+\sum _{j\in J_2\setminus J_1}\hat{a}_{j2}, \end{aligned}$$
(98)

and

$$\begin{aligned} \frac{\hat{A}_2 }{1+\hat{A}_2} \sum _{j\in J_1\cap J_2} \hat{a}_{j1}+\hat{a}_{j2}= \sum _{j\in J_1\cap J_2}\hat{a}_{j2}. \end{aligned}$$
(99)

We then obtain

$$\begin{aligned} \hat{A}_2 \sum _{j\in J_1\cap J_2} \hat{a}_{j1}= \sum _{j\in J_1\cap J_2}\hat{a}_{j2}. \end{aligned}$$
(100)

The estimator of the equating coefficient \(A_2\) is then equal to

$$\begin{aligned} \hat{A}_2 = \frac{\sum _{j\in J_1\cap J_2}\hat{a}_{j2}}{\sum _{j\in J_1\cap J_2} \hat{a}_{j1}}, \end{aligned}$$
(101)

which corresponds to the mean-mean estimator of the equating coefficient A for two forms.

Appendix 3: Proof of the Symmetry Property of MIRF and MTRF Methods

In order to convert item parameters on the scale of Form r, the equating coefficients are transformed as follows:

$$\begin{aligned} \hat{A}_t' = \frac{\hat{A}_t}{\hat{A}_r} \quad \text {and} \quad \hat{B}_t' = \frac{\hat{B}_t - \hat{B}_r}{\hat{A}_r},\quad \text {for } t=1,\dots ,T, \end{aligned}$$

so that \(\hat{A}_r'=1\) and \(\hat{B}_r'=0\). If \(\hat{A}_t\) is replaced with \(\hat{A}_t' \) and \(\hat{B}_t\) is replaced with \(\hat{B}_t'\) in Equation (22), it is simple to verify that \(\hat{a}_{jt}^*\) and \(\hat{b}_{jt}^*\) do not vary after this substitution. Consequently, Equations (17) and (29) are invariant with respect to changes of the base form, thus proving the symmetry property.

Appendix 4: Variability of Estimated Abilities

The following equation gives the conversion of estimated abilities from the scale of Form t to the scale of the base form

$$\begin{aligned} \theta ^*=\theta _t A_t + B_t. \end{aligned}$$

The estimated ability \(\hat{\theta }_t\) can be transformed using the estimated equating coefficients

$$\begin{aligned} \hat{\theta }^*=\hat{\theta }_t \hat{A}_t + \hat{B}_t. \end{aligned}$$

The variance of \(\hat{\theta }^*\) given \(\hat{\theta }_t\) is

$$\begin{aligned} \mathrm {var}(\hat{\theta }^*|\hat{\theta }_t)=\hat{\theta }_t^2 \mathrm {var}(\hat{A}_t)+\mathrm {var}(\hat{B}_t)+ 2\hat{\theta }_t \mathrm {cov}(\hat{A}_t,\hat{B}_t), \end{aligned}$$

while the conditional expected value is

$$\begin{aligned} \mathrm {E}(\hat{\theta }^*|\hat{\theta }_t)=\hat{\theta }_t \mathrm {E}(\hat{A}_t)+\mathrm {E}(\hat{B}_t)= \hat{\theta }_t (A_t+o(1))+ B_t+o(1), \end{aligned}$$

provided that the estimators \(\hat{A}_t\) and \(\hat{B}_t\) are consistent. So, the variance of \(\hat{\theta }^*\) is

$$\begin{aligned} \mathrm {var}(\hat{\theta }^*)&=\mathrm {E}\{\mathrm {var}(\hat{\theta }^*|\hat{\theta }_t)\}+ \mathrm {var}\{\mathrm {E}(\hat{\theta }^*|\hat{\theta }_t)\} \\&= \mathrm {var}(\hat{A}_t)+\mathrm {var}(\hat{B}_t)+\mathrm {var}(\hat{\theta }_t) A_t^2+ o(1), \end{aligned}$$

where \(\mathrm {E}(\theta _t)\) and \(\mathrm {var}(\theta _t)\) are assumed to be 0 and 1, respectively, as usual with the marginal maximum likelihood estimation method. Hence, if the reliability of \(\hat{\theta }_t\) is

$$\begin{aligned} \rho (\hat{\theta }_t)=\frac{\mathrm {var}(\theta _t)}{\mathrm {var}(\hat{\theta }_t)}= \frac{1}{\mathrm {var}(\hat{\theta }_t)}, \end{aligned}$$
(102)

the reliability of \(\hat{\theta }^*\) is

$$\begin{aligned} \rho (\hat{\theta }^*)=\frac{\mathrm {var}(\theta ^*)}{\mathrm {var}(\hat{\theta }^*)}\simeq \frac{\mathrm {var}(\theta _t) A_t^2}{\mathrm {var}(\hat{A}_t)+\mathrm {var}(\hat{B}_t)+\mathrm {var}(\hat{\theta }_t) A_t^2}= \frac{ A_t^2}{\mathrm {var}(\hat{A}_t)+\mathrm {var}(\hat{B}_t)+\mathrm {var}(\hat{\theta }_t) A_t^2}.\qquad \quad \end{aligned}$$
(103)

The reliability of \(\hat{\theta }^*\) is then always greater than the reliability of \(\hat{\theta }_t\), due to variability of the estimated equating coefficients. These reliabilities can be estimated by substituting the true values with their estimates in (102) and (103). An estimate of \(\mathrm {var}(\hat{\theta }_t)\) is \(1+\hat{se}^2(\hat{\theta }_t)\), where \(\hat{se}(\hat{\theta }_t)\) is the estimated standard error of \(\hat{\theta }_t\).

Another quantity of interest is the standard error of \(\hat{\theta }^*\), which can be obtained as follows:

$$\begin{aligned} se(\hat{\theta }^*)=\{ \mathrm {var}(\hat{\theta }^*)-\mathrm {var}(\theta ^*) \} ^{1/2} \simeq \{ \mathrm {var}(\hat{A}_t)+\mathrm {var}(\hat{B}_t)+se^2(\hat{\theta }_t) A_t^2 \} ^{1/2}. \end{aligned}$$

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Battauz, M. Multiple Equating of Separate IRT Calibrations. Psychometrika 82, 610–636 (2017). https://doi.org/10.1007/s11336-016-9517-x

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-016-9517-x

Keywords

  • equating coefficients
  • Haebara
  • item response theory
  • linking
  • mean-geometric mean
  • mean-mean
  • standard errors
  • Stocking–Lord