Data Fusion

Abstract

To paraphrase Churchill, certainty in face recognition [rather than oil] lies in variety and variety alone. Variety comes in several forms and drives the way fusion takes place. Fusion can include raw or transformed data and their cues, on one side, and the face classifiers and the voting schemes that combine them, on the other side. A voting scheme implements the equivalent of a gating network that arbitrates among the scores made available by the Cartesian product of classifiers and face representations used (Poh and Bengio, 2006). The data originates from one or several modalities using different sensors, e.g., audio and video or 2D and 3D, and/or one or several image domains, e.g., face and ear. The modality itself can be further defined along specific channels, e.g., the RGB space for color. Each modality can be sampled over space and/or time. This leads to multi sample fusion, using parts and/or (temporal) multi-image sets. The cues used are drawn from one modality, e.g., shape and texture from 2D, or from several domains, e.g., face and fingerprints or face, voice, and lip movement. What makes the face attractive or not could be used for face recognition too. Particulars like moles, acne and birthmarks are great cues for identification and are used by soft biometrics. Augmented cognition, which expands on soft biometrics (see Ch. 11), accesses mental models and reasons about how biometric events unfold and the reasons behind. Data fusion ultimately operates across a multi-dimensional Cartesian product whose bounds and dimensions are only set by experts' imagination and technical innovation. The basic motivation for data fusion is the justified belief that the biometric dimensions, relatively independent of each other, should accrue and integrate circumstantial evidence leading to better recognition. Divide et Impera and the Gestalt that the whole is more than the sum of its parts are a succinct description of what data fusion thrives to achieve. Last but not least, data fusion implements a convoluted architecture, similar to that found in the neocortex, where the activity at the lower levels is pruned by projections or cortical feedback from the higher levels. The level of data fusion at the lower levels is reduced when the higher levels can explain or predict the input (Mumford, 1992; Rao and Ballard, I999). This is similar to the analysis by synthesis framework used in image understanding.