The SIMCLAS Model: Simultaneous Analysis of Coupled Binary Data Matrices with Noise Heterogeneity Between and Within Data Blocks

Abstract

In many research domains different pieces of information are collected regarding the same set of objects. Each piece of information constitutes a data block, and all these (coupled) blocks have the object mode in common. When analyzing such data, an important aim is to obtain an overall picture of the structure underlying the whole set of coupled data blocks. A further challenge consists of accounting for the differences in information value that exist between and within (i.e., between the objects of a single block) data blocks. To tackle these issues, analysis techniques may be useful in which all available pieces of information are integrated and in which at the same time noise heterogeneity is taken into account. For the case of binary coupled data, however, only methods exist that go for a simultaneous analysis of all data blocks but that do not account for noise heterogeneity. Therefore, in this paper, the SIMCLAS model, being a Hierarchical Classes model for the simultaneous analysis of coupled binary two-way matrices, is presented. In this model, noise heterogeneity between and within the data blocks is accounted for by downweighting entries from noisy blocks/objects within a block. In a simulation study it is shown that (1) the SIMCLAS technique recovers the underlying structure of coupled data to a very large extent, and (2) the SIMCLAS technique outperforms a Hierarchical Classes technique in which all entries contribute equally to the analysis (i.e., noise homogeneity within and between blocks). The latter is also demonstrated in an application of both techniques to empirical data on categorization of semantic concepts.

Appendix: Simulated Annealing to Estimate the Bundle Matrices, Conditional on the Noise Parameters

To estimate, in Step 2 of the SIMCLAS algorithm (see section The SIMCLAS algorithm), the binary bundle matrices A and B ⁿ that maximize the loss function, conditionally upon the noise parameters, a simulated annealing procedure is adopted. Simulated annealing is a local search technique that implies a walk through the solution space. In particular, a chain of solutions, consisting of several subchains, is generated by each time creating a candidate solution based on the current solution. Next, the loss function values of the current and the candidate solution are compared. If the candidate solution has a better loss function value f, it is accepted, which implies that the current solution is replaced by the candidate solution. If the candidate solution, however, has a worse loss function value, it is accepted with a probability that depends on its relative quality (i.e., the difference in loss function value f between the current solution and the candidate one) and the current temperature, a quantity that controls the acceptance probability. At the end of each subchain the temperature is decreased. Subchains are generated until a prespecified subchain stop criterion is met. Finally, the best encountered solution in the chain is retained.

Based on the results of a pilot study and on the SA implementations that have been used for other Hierarchical Classes models (see, Ceulemans et al. 2007), we implemented the procedure for generating a single SA chain (see Algorithm A1 for pseudo-code) in the SIMCLAS algorithm as follows:

• 1. An initial solution S _current and associated initial loss value L _current is obtained by replacing the P columns of each bundle matrix by P data vectors sampled at random (i.e., for A, column vectors are drawn from the different D ⁿ, whereas for each B ⁿ, row vectors are chosen from the corresponding D ⁿ).

• 2. The initial temperature T _initial is obtained by running a subchain of solutions and accepting all solutions; subsequently, the average increase in the likelihood function across those links in which worse solutions are accepted, is divided by ln(0.8); as such, during the first subchains in which the algorithm is still far from the optimal solution, worse solutions are accepted with a high probability (see Kirkpatrick et al. 1983; Aarts et al. 1997; Ceulemans et al. 2007).

• 3. A candidate solution S _trial, and associated loss value L _trial, is obtained from the current solution S _current by altering the value of a randomly chosen cell of a randomly chosen bundle matrix, with each cell of each bundle matrix having the same probability of being changed.

• 4. A worse candidate solution is accepted if: p<exp((L _trial−L _current)/T _current), with p being a number generated from a uniform (0,1) distribution.

• 5. A subchain stops (1) if the number of generated solutions i _gen equals the maximum number of solutions $\mathit{CL} = ((I+\sum_{n=1}^{N} J_{n})\times2^{P})\times5$, or (2) if the number of accepted solutions i _acc equals CL×0.10.

• 6. At the end of each subchain, the temperature is decreased by a factor α=0.90, implying a smaller acceptance probability for worse solutions: T _current=0.9×T _current.

• 7. An SA chain stops when (1) the current temperature becomes smaller than T _stop=0.000001, or (2) the number of subsequent subchains i _id with an identical loss value L _current for the last accepted solution in each subchain (i.e., L _current=L _previous) equals $\mathit {max}_{i_{\mathrm{id}}}$, which is set to five.

• 8. The retained solution is the best encountered solution S _best across all subchains.

Algorithm A1

Pseudo-code for generating a single SA chain in Step 2 of the SIMCLAS algorithm.

To lower the risk of ending in a suboptimal solution (i.e., local optimum), a multi-start procedure may be advised, which consists of running 100 SA chains, each time with a different initial solution and initial temperature (see Steps 1 and 2), and retaining the best encountered solution across all chains (see Ceulemans et al. 2007).