Zero-Inflated Poisson Tensor Factorization for Sparse Purchase Data in E-Commerce Markets

Abstract

Nonnegative tensor factorization (NTF) plays a crucial role in extracting latent factors and predicting future sales from purchase data consisting of user and item attributes. However, the increase in these attributes leads to tensor data becoming sparse, causing a reduction in decomposition accuracy. For example, when there are numerous combinations of unavailable item genres and prices, the purchase history data becomes sparse and follows a distribution where all its elements are zero. To address this issue, we propose a novel NTF method assuming zero-inflated Poisson (ZIP) distribution based on Expectation-Maximization (EM) algorithm. This enables us to effectively handle sparsity in high-dimensional multiway data and identify combinations of user and item attributes that are potentially not likely to be purchased. We verified the effectiveness of the proposed approach through numerical experiments using real-world e-commerce data. The results showed our proposed ZIP model outperforms existing methods in both in-sample and out-of-sample experiments. Moreover, the proposed method qualitatively demonstrated the effectiveness of handling sparsity.

Appendices

A Computational Time

We compare the seconds per step of the above experiments to evaluate computational time. Figure 5 (a) shows the average computational time of 10 trial runs for the four factorization models. The initial values are uniformly sampled across all models for each trial. Due to the complexity of parameters and update algorithms, our proposed method resulted in the slowest computational time.

Fig. 5.

Computational time per step; (a) Each tensor factorization model. (b) Backends of implementation.

Additionally, we compared the computational time of implementations using our JAX + JIT compiler method with Tensorly in Fig. 5 (b). Both of these methods assume a normal distribution on the tensor and use the same algorithm. Our proposed algorithm significantly mitigates the increase in computational time with the increase in rank compared to the alternative.

B ZICMP Distribution Model

We demonstrate the Zero-Inflated Conway-Maxwell-Poisson (ZICMP) distribution tensor factorization model, which is a modified version of the Conway-Maxwell-Poisson (CMP) distribution in place of the Poisson distribution [22,23,24]. This enables to represent over and under dispersion which was not able by the ZIP distribution. When random variable y follows this distribution, the probability mass function is defined as follows:

$$\begin{aligned} f(y;\lambda , \nu , p) &= {\left\{ \begin{array}{ll}p+(1-p) \frac{1}{Z(\lambda , \nu )} &{} \text{ if } y=0 , \\ (1-p) \frac{\lambda ^y}{(y !)^\nu } \frac{1}{Z(\lambda , \nu )} &{} \text {otherwise} , \end{array}\right. } \quad \text {where}~Z(\lambda , \nu )&=\sum _{h=0}^{\infty } \frac{\lambda ^h}{(h !)^\nu }. \end{aligned}$$

p is the mixture ratio and $\lambda$ is the expected value of the ZICMP distribution. $\nu$ is a nonnegative real number and controls the variance. For $\nu = 1$, its distribution aligns with the Poisson distribution.

Table 4. Evaluation metrics for each algorithm

In contrast to the ZIP model, here we update $\boldsymbol{A}^{(1)},\dots , \boldsymbol{A}^{(n)}$ and $\nu$ using Newton’s method at each step of factor matrices updating phase in Algorithm 1. A magnitude of Newton step can be derived by taking the first and second derivatives of the log-likelihood function with respect to $a^{(k)}_{i_kr}$ and $\nu$.

$$\begin{aligned} \varDelta a^{(k)}_{i_{k}r} &= \frac{- \displaystyle \sum _{i_1,\dots ,i_{k-1},i_{k+1},\dots ,i_{n}}\left( 1-z_{i_1 \dots i_n} \right) \left( -\frac{Z^{\prime }}{Z}+\frac{y_{i_1 \dots i_n}}{\lambda }\right) \prod _{\begin{array}{c} l=1 \\ l \ne k \end{array}}^n a^{(l)}_{i_l r}}{\displaystyle \sum _{i_1,\dots ,i_{k-1},i_{k+1},\dots ,i_{n}}\left( 1-z_{i_1 \dots i_n} \right) \Biggl (-\frac{Z^{\prime \prime }}{Z}+\left( \frac{Z^{\prime }}{Z}\right) ^{2}-\frac{y_{i_1 \dots i_n}}{\lambda ^2} \Biggl ) \prod _{\begin{array}{c} l=1 \\ l \ne k \end{array}}^n (a^{(l)}_{i_l r})^2}, \\ \varDelta \nu & =- \frac{\displaystyle \sum _{i_1 \dots i_n}\left( 1-z_{i_1 \dots i_n} \right) \Biggl (-\frac{\frac{\partial Z}{\partial \nu }}{Z}-\log \left( y_{i_1 \dots i_n} !\right) \Biggl )}{\displaystyle \sum _{i_1 \dots i_n}\left( 1-z_{i_1 \dots i_n} \right) \Biggl (-\frac{\frac{\partial ^2 Z}{{\partial }{\nu ^2}}}{Z}+\left( \frac{\frac{\partial Z}{\partial \nu }}{Z}\right) ^2\Biggl )}, \\ \text {where} &~ Z^{\prime } = \sum _{h=0}^{\infty } \frac{h \lambda ^{h-1}}{(h !)^\nu }, ~Z^{\prime \prime }= \sum _{h=0}^{\infty } \frac{h(h-1) \lambda ^{h-2}}{(h !)^\nu }, ~\lambda = \sum _r \prod _{l = 1}^n a^{(l)}_{i_l r}. \end{aligned}$$

We present the results of the numerical experiments for tensor factorization assuming the ZICMP distribution in Table 4. Our algorithm encountered overflow with large tensor sizes or element values. To address this, we reduced the tensor size to a 5th-mode tensor by eliminating the region mode.

As a result, RMSPE outperformed the other algorithms, but not in other evaluation metrics. While it can be assumed that the ZICMP distribution improved the expressiveness of the distribution, the fitting to sparsity did not work.