Similarity, Equivalence Testing, and Discrimination Theory

Abstract

This chapter discusses equivalence testing and how difference tests are modified in their analyses to guard against Type II error (missing a true difference). Concepts of test power and required sample sizes are discussed and illustrated. An alternative approach to equivalence, namely interval testing is introduced along with the concept of paired one-sided tests. Two theoretical approaches to the measurement of the size of a difference are introduced: discriminator theory (also called guessing models) and the signal detection or Thurstonian models.

Difference testing method constitute a major foundation for sensory evaluation and consumer testing. These methods attempt to answer fundamental questions about stimulus and product similarity before descriptive or hedonic evaluations are even relevant. In many applications involving product or process changes, difference testing is the most appropriate mechanism for answering questions concerning product substitutability.

—D. M. Ennis (1993)

An erratum to this chapter can be found online at http://dx.doi.org/10.1007/978-1-4419-6488-5_20.

5.1 Appendix: Non-Central t-Test for Equivalence of Scaled Data

Bi (2007) described a similarity test for two means, as might come from some scaled data such as acceptability ratings, descriptive panel data, or quality control panel data. The critical test statistic is T _AH after the original authors of the test, Anderson and Hauck. If we have two means, M ₁ and M ₂, from two groups of panelists with N panelists per group and a variance estimate, S, the test proceeds as follows:

$$T_{{\textrm{AH}}} = \frac{{M_1 - M_2 }}{{s\sqrt {2/N} }}$$

((5.13))

The variance estimate, S, can be based on the two samples, where

$$S^2 = \frac{{S_1^2 + S_2^2 }}{N}$$

((5.14))

and we must also estimate a non-centrality parameter, δ,

$$\delta = \frac{{\Delta _{\textrm{o}} }}{{s\sqrt {2/N} }}$$

((5.15))

where ∆_ο is the allowable difference interval.

The calculated p-value is then

$$p = t_\nu (\left| {T_{{\textrm{AH}}} } \right| - \delta ) - t_\nu ( - \left| {T_{{\textrm{AH}}} } \right| - \delta )$$

((5.16))

and t _ν is the p-value from the common central t-distribution value for ν = 2(N–1) degrees of freedom. If p is less than our cutoff, usually 0.05, then we can conclude that our difference is within the acceptable interval and we have equivalence.

For paired data, the situation is even simpler, but in order to calculate your critical value, you need a calculator for critical points of the non-central F-distribution, as found in various statistical packages.

To apply this, perform a simple dependent samples (paired data) t-test. Determine the maximum allowable difference in terms of the scale difference and normalize this by stating it in standard deviation units. The obtained value of t is then compared to the critical value as follows:

$$C = \sqrt F$$

((5.17))

where the F value corresponds to a value for the non-central F-distribution for 1, N–1 degrees of freedom, and a non-centrality parameter, given by N(É›), and (É›) is the size of the critical difference in standard deviation units. If you do not have easy access to a calculator for the critical values of a non-central F, a very useful table is given in Gacula et al. (2009) where the value of T may be directly compared to the critical value based on an alpha level of 0.05 and various levels of (É›) (Appendix Table A.30, pp. 812–813 in Gacula et al., 2009). The absolute value of the obtained t-value must be less than the critical C value to fall in the range of significant similarity or equivalence.

Worked examples can be found in Bi (2005) and Gacula et al. (2009).