High-dimensional scalar function visualization using principal parameterizations

Abstract

Insightful visualization of multidimensional scalar fields, in particular parameter spaces, is key to many computational science and engineering disciplines. We propose a principal component-based approach to visualize such fields that accurately reflects their sensitivity to their input parameters. The method performs dimensionality reduction on the space formed by all possible partial functions (i.e., those defined by fixing one or more input parameters to specific values), which are projected to low-dimensional parameterized manifolds such as 3D curves, surfaces, and ensembles thereof. Our mapping provides a direct geometrical and visual interpretation in terms of Sobol’s celebrated method for variance-based sensitivity analysis. We furthermore contribute a practical realization of the proposed method by means of tensor decomposition, which enables accurate yet interactive integration and multilinear principal component analysis of high-dimensional models.

Appendices

Appendix

Details on cross-approximation

Cross-approximation is a surrogate modeling technique that incrementally builds a compressed TT tensor approximating a target black-box function by evaluating the function at a batch of samples per iteration. These samples are chosen in a clever adaptive manner so as to minimize the model’s relative error using the smallest possible number of evaluations. The error is defined as $\Vert \tilde{\textbf{X}} - \textbf{X}\Vert / \Vert \textbf{X}\Vert$, where $\textbf{X}$ are the groundtruth evaluations and $\tilde{\textbf{X}}$ is the model’s prediction. These samples are also used as a validation set, and the process is stopped as soon as the error is below a user-defined threshold $\epsilon$. This is a standard way to approximate the generalization error when building compressed tensors, and the estimation is known [31] to be reliable in practice.

We used $\epsilon := 10^{-4}$ in all cases, which required between $10^5$ and $10^7$ samples and resulted in a few seconds’ time to build each TT metamodel. Although not needed for our work, note that there are alternatives to cross-approximation for the case when function evaluation is expensive [23].

In this paper, we have used the implementation of cross-approximation that is released in the ttpy Python toolbox [1].

Step-by-step illustration of the algorithm

In Fig. 16, we show a flowchart of the proposed algorithm using an intermediate visualization along all its steps. To this end, we take a simplistic 3D use case function that is simple to understand and can directly be displayed both as a 3D rendering as well as a sequence of 2D slices.

Fig. 16

Flowchart of the proposed method. For illustrative purposes, we consider a 3D function f representing a grayscale video: f(x, y, t) is the intensity of pixel (x, y) at time $0 \le t \le 1$. The variable of interest is t. The video captures a static ball in a fixed scene and camera position where the light source moves in a circular fashion. In addition, the light is suddenly dimmed in the middle of the video, which is manifested as a jump in t’s curve along the $\pi _x$ axis. The three red boxes correspond to the coordinates $(\pi _x(t), \pi _y(t), \pi _z(t))$ at time $t=0.5$, right before the light source is dimmed. The original video is a tensor of shape $256 \times 256 \times 256$; for the sake of presentation, only 10 timesteps are shown in this diagram

Operations in the TT compressed domain

The TT format allows for efficient computation of several operations without having to explicitly decompress the tensor it represents. This appendix covers the three main operations that are leveraged in the paper.

1.1 Multidimensional integrals

Suppose our original function is expressed as a TT with cores ${\mathcal {T}}^{(1)}, \dots , {\mathcal {T}}^{(N)}$. To marginalize the n-th variable away (i.e. compute the expected value along $x_n$), one simply needs to replace the core ${\mathcal {T}}^{(n)}$ by $\widehat{{\mathcal {T}}}^{(n)}$ where

$$\begin{aligned} \widehat{{\mathcal {T}}}^{(n)}:= \frac{1}{I_n} \sum _{i=1}^{i=I_n} {\mathcal {T}}^{(n)}[i], \end{aligned}$$

where $I_n$ is the shape of the grid along dimension n. In other words, we simply need to average the n-th core along its second dimension.

1.2 Element-wise arithmetics

If two tensors are in the TT format with cores ${\mathcal {T}}_1^{(1)}, \dots , {\mathcal {T}}_1^{(N)}$ and ${\mathcal {T}}_2^{(1)}, \dots , {\mathcal {T}}_2^{(N)}$, then their element-wise sum ${\mathcal {T}}_3 = {\mathcal {T}}_1 + {\mathcal {T}}_2$ is given by the following cores:

$$\begin{aligned} {\mathcal {T}}_3^{(n)}[i_n]:= {\left\{ \begin{array}{ll} \left( \begin{array}{cc} \mathcal {T}_1^{(1)} [i_1]; &{} \mathcal {T}_2^{(1)} [i_1] \\ \end{array} \right) &{} \text{ if } n = 1; \\ \left( \begin{array}{cc} \mathcal {T}_1^{(n)} [i_n]; &{} 0 \\ 0; &{} \mathcal {T}_2^{(n)} [i_n] \end{array} \right) &{} \text{ if } n = 2, \dots , N-1; \\ \left( \begin{array}{c} \mathcal {T}_1^{(N)} [i_N] \\ \mathcal {T}_2^{(N)} [i_N] \end{array} \right) &{} \text{ if } n = N. \\ \end{array}\right. } \end{aligned}$$

To subtract two tensors, it is enough to flip the sign of the second one by flipping the sign of its first core, and then sum them as above.

The operations just described allow us to obtain up to ${\mathcal {M}}$ (line 8 from Algorithm 1) in the TT format.

1.3 PCA projection

In practice, once we have ${\mathcal {M}}$, we found it more efficient to compute its PCA projection in the compressed domain via the SVD decomposition as follows. Let ${\mathcal {M}}$, which represents a matrix of size $I^K \times I^{N-K}$, be given by TT cores ${\mathcal {M}}^{(1)}, \dots , {\mathcal {M}}^{(N)}$. We proceed as follows:

• 1. We left-orthogonalize [31] the TT. This is equivalent to finding the RQ decomposition of ${\mathcal {M}}$, with the first K cores representing the ${\textbf{R}}$ part (a matrix of shape $I^K \times R$, where R is the K-th TT rank of ${\mathcal {M}}$) and the remaining cores the ${\textbf{Q}}$ part (of shape $R \times I^{N-K}$).

• 2. We discard ${\textbf{Q}}$ by keeping the first K cores only.

• 3. We perform rank-truncation [31] on the last rank in order to decrease it to 3. This involves a SVD decomposition on the last core and is equivalent to keeping the three leading left singular vectors of ${\textbf{R}}$.

The resulting tensor has shape $I^K \times 3$, as desired, and represents a mapping from the original $I^K$ grid to ${\mathbb {R}}^3$, i.e. a discretized K-dimensional manifold, that is as close as possible to ${\mathcal {M}}$ in the $L^2$ sense.