A personalized point-of-interest recommendation system for O2O commerce

Abstract

Online-to-offline (O2O) commerce, e.g., the internet celebrity economy, provides a seamless service experience between online commerce and offline bricks-and-mortar commerce. This type of commerce model is closely related to location-based social networks (LBSNs), which incorporate mobility patterns and human social ties. Personalized point-of-interest (POI) recommendations are crucial for O2O commerce in LBSNs; such recommendations not only help users explore new venues but also enable many location-based services, e.g., the targeting of mobile advertisements to users. However, producing personalized POI recommendations for O2O commerce is highly challenging, since LBSNs involve heterogeneous types of data and the user-POI matrix is very sparse. LBSNs have substantially altered how people interact by sharing a wide range of user information, such as the products and services that users use and the places and events that users visit. To address these challenges in O2O commerce LBSNs, we analyze users’ check-in behaviors in detail and introduce the concept of a heterogeneous information network (HIN). Then, we propose a HIN-based POI recommendation system, which consists of two components: an improved singular value decomposition (SVD++) and factorization machines (FMs). The results of experiments on two real-world O2O commerce websites, namely, Gowalla and Foursquare, demonstrate that our method is more accurate than baseline methods. Additionally, a case study of the bricks-and-mortar brand of internet celebrity indicates that our proposed POI recommendation system can be used to conduct online promotion and purchasing to drive offline marketing and consumption.

Appendix

Weighted meta path-based similarity

As described in related works, the temporal characteristics of user behavior in O2O commerce LBSNs contain two aspects: periodicity and preference variance. To capture the temporal cyclic pattern, a time-indexing scheme that encodes a standard time stamp to a specified time slot was devised. We consider the preference variance in a two-hierarchy category: time of day and day of the week. Therefore, as shown in Table 6, a week is divided into weekdays and the weekend, while a day is divided into four sessions. Hence, in total, there are 8 time slots, which can represent both weekly and daily preference variances.

Table 6 Time-slot scheme of a twenty-four-hour clock

Many real-world networks, especially O2O commerce LBSNs, contain attribute values on links. For instance, to represent covisitation events, the time slot is used as the weight for the user-POI link. However, few conventional HINs deal with attribute values on links. In this paper, we employ the concept of WHIN to handle this issue (Shi et al. 2019).

Definition 1.

Weighted heterogeneous information network. A weighted heterogeneous information network is defined as a directed graph G = (V, E, W) with schema $S=\left(\mathcal{A},\mathcal{R},\mathcal{W}\right)$, where V is the node set, E is the link set, W is the attribute value (weight) set, $\mathcal{A}=\left\{A\right\}$ is the node type set, $\mathcal{R}=\left\{R\right\}$ is the link type set, and $\mathcal{W}=\left\{W\right\}$ is the attribute value type set. Each object v ∈ V maps to an object type $\varphi (v)\in \mathcal{A}$ by function $\varphi :V\longrightarrow \mathcal{A}$, each link e ∈ E maps to a relation $\psi (e)\in \mathcal{R}$ by function $\varphi :E\longrightarrow \mathcal{R}$, and each attribute value w ∈ W maps to an attribute value type $\theta (w)\in \mathcal{W}$ via a function $\uptheta :W\longrightarrow \mathcal{W}$.

With the concept of WHIN, an intuitive strategy is to extend the conventional meta path to deal with attribute values on relations, namely, to a weighted meta path.

Definition 2.

Weighted meta path. A weighted meta path is a meta path that is based on an attribute value constraint on relations; this constraint is denoted as ${A}_1\overset{\delta_1\left({R}_1\right)}{\to }{A}_2\overset{\delta_2\left({R}_2\right)}{\to}\cdots \overset{\delta_l\left({R}_l\right)}{\to }{A}_{l+1}\mid \mathcal{C}$. The attribute value function δ(R) is a set of values from the attribute value range of relation R. ${A}_i\overset{\delta_i\left({R}_i\right)}{\to }{A}_{i+1}$ represents the relation R_i between A_i and A_i + 1 that is based on the attribute values δ_i(R_i). The constraint $\mathcal{C}$ is a set of correlation constraints among attribute value functions.

Most similarity measures on meta paths are based on matrix multiplication, e.g., counting the probability of a random walk or the number of meta path instances. However, these similarity measures fail to handle the information of attribute value constraints on multiple relations of O2O commerce. To address this problem, we introduce the concept of an atomic meta path and use an ingenious solution to ensure that the existing path-based similarity measure remains available in the weighted meta path.

Definition 3

Atomic meta path. Given a weighted meta path $\mathcal{P}$, the atomic meta path is a subset of the weighted meta path $\mathcal{P}$ in which all attribute value functions δ(R) take a specified value. Namely, a weighted meta path is a complete set of atomic meta paths that satisfy the constraint $\mathcal{C}$.

Given the above definitions, an intuitive idea is to estimate the similarities between the source and the target nodes. In this paper, we employ PathSim (Sun et al. 2011) as the similarity measure, which can identify peer objects in the network. PathSim can be expressed as:

$$ S\left(x,y|{\mathcal{P}}_{\mathcal{C}}\right)=\frac{2\times {\sum}_{{\mathcal{P}}_{\alpha}\in {\mathcal{P}}_{\mathcal{C}}}\left|\left\{{p}_{x\rightsquigarrow y}:{p}_{x\rightsquigarrow y}\in {\mathcal{P}}_{\alpha}\right\}\right|}{\sum_{{\mathcal{P}}_{\alpha}\in {\mathcal{P}}_{\mathcal{C}}}\left|\left\{{p}_{x\rightsquigarrow x}:{p}_{x\rightsquigarrow x}\in {\mathcal{P}}_{\alpha}\right\}\right|+{\sum}_{{\mathcal{P}}_{\alpha}\in {\mathcal{P}}_{\mathcal{C}}}\left|\left\{{p}_{y\rightsquigarrow y}:{p}_{y\rightsquigarrow y}\in {\mathcal{P}}_{\alpha}\right\}\right|} $$

(3)

where ${\mathcal{P}}_{\mathcal{C}}$ is a weighted meta path with attribute value constraint $\mathcal{C}$, ${\mathcal{P}}_{\alpha }$ is an atomic meta path of ${\mathcal{P}}_{\mathcal{C}}$, and ${p}_{x\rightsquigarrow y}\in {\mathcal{P}}_{\alpha }$indicates a path instance connecting node x and y along atomic meta path ${\mathcal{P}}_{\alpha }$. As PathSim counts the number of path instances along the meta path with a normalized term, all users are treated identically. In particular, we should consider the effect of the normalized term in PathSim.

Since a weighted meta path is a combination of corresponding atomic meta paths, we regard the similarity measure that is based on a weighted meta path as the sum of the similarity measures that are based on the corresponding atomic meta paths. Moreover, the PathSim strategy along a weighted meta path can be described as the following steps: Initially, we count the number of path instances along each atomic meta path. Then, we sum the corresponding numbers along every atomic meta path before normalization. Therefore, our solution identifies similar users more accurately because they have the same preferences.

The longer the meta paths are, the less likely they are to generate satisfactory similarity measures because they fail to convey a distinct meaning (Sun et al. 2011). Thus, we employ 7 meaningful meta paths whose lengths are no longer than 4. All the weighted and unweighted meta paths that are used in this paper are listed in Table 7. These meta paths all begin from the source node U (user) and end at the target node P (POI), while intermediate nodes can be interpreted by various latent features in the LBSNs of O2O commerce, e.g., user preferences, temporal effects, and geographical and social influences.

Table 7 Meta paths that are used for the Gowalla and Foursquare datasets

By calculating the similarities between all users and all POIs along the meta paths $\mathcal{P}$, a user-POI similarity matrix $\hat{R}\in {\mathbb{R}}^{m\times n}$ is obtained, where $\hat{R_{ij}}$represents the similarity between user u_iand POIp_j, and mand n denote the numbers of users and POIs, respectively. With L meta paths in an O2O commerce LBSN,we can obtain L user-POI similarity matrices that differ in terms of semantics and are denoted by ${\hat{R}}^1,\cdots, {\hat{R}}^L$.

Latent features in O2O commerce LBSNs

After user-POI similarity matrices have been obtained, we employ matrix factorization (MF) to identify the latent features of users and POIs in O2O commerce. Our MF approach is based on a state-of-the-art model, namely, SVD++, which was proposed by Koren (2008). To alleviate the noise problem and overcome the data sparsity in the user-POI similarity matrices, SVD++ considers user and POI biases and the influence of rated POIs, in addition to the user- and POI-specific vectors, for rating prediction. An SVD++ model associates each user u with a user-factor vector p_u ∈ ℝ^F and each POI j with a POI-factor vectorq_j ∈ ℝ^F. Formally, the rating for user uon POI j is predicted by:

$$ {\hat{r}}_{u,j}=\mu +{b}_u+{b}_j+{q}_j^{\mathrm{T}}\left({p}_u+{\left|{N}_u\right|}^{-\frac{1}{2}}{\sum}_{i\in {N}_u}{\mathcal{Y}}_i\right) $$

(4)

where μ is the global average rating; b_u and b_j represent the user and POI biases, respectively; N_u is a set of POIs for which user u exhibited an implicit preference; and ${\mathcal{Y}}_i$denotes the implicit influence of POIs that have been rated by user u in the past on the ratings of unknown POIs in the future. Thus, the feature vector of user u can also be represented by the set of rated POIs andmodeled as $\left({p}_u+{\left|{N}_u\right|}^{-\frac{1}{2}}{\sum}_{i\in {N}_u}{\mathcal{Y}}_i\right)$ rather than simply being represented as p_u.

Assuming that user preferences in O2O commerce are controlled by few factors, we factor the similarity matrix $\hat{R}$ into two low-rank matricesby solving the optimizing problem in (5):

$$ \underset{b_u,{b}_j,{q}_j,{p}_u,{y}_i}{\min}\sum \limits_{\left(u,j\right)\in \kappa}\left({r}_{uj}-{\hat{r}}_{uj}\right)+{\lambda}_{up}\left({b}_u^2+{b}_j^2+{\left\Vert {p}_u\right\Vert}_F^2+{\left\Vert {q}_j\right\Vert}_F^2+\frac{\sum \limits_{j\in {N}_u}{\left\Vert {y}_j\right\Vert}_F^2}{{\left\Vert {q}_j\right\Vert}_F^2}\right) $$

(5)

where κ is a set of (u, j)pairs for which r_uj is known and λ_updenotes the hyperparameter that controls the influence of the regularization to avoid overfitting. For L user-POI similarity matrices, we can obtain L groups of latent features of users and POIs; these features are denoted as U⁽¹⁾, ⋯, U^(L), P⁽¹⁾, ⋯, P^(L).

POI recommendation with factorization machine

With L groups of user and POI latent features in O2O commerce LBSNs, an intuitive strategy is to combine these features linearly to generate ratings. In previous approaches, the rating is predicted by a weighted ensemble of inner products of user-specific and POI-specific vectors from every meta path. However, this conventional approach fails to capture the interactions between and among inter-meta path features, thereby possibly decreasing prediction accuracy.

Therefore, an FM-based method is proposed for generating POI recommendations in O2O commerce LBSNs. First, we concatenate all the user and POI features from L meta paths. The result is denoted by Xⁿ, which represents the feature vector of the n-th sample after concatenation.

$$ {X}^n={u}_i^1,\cdots, {u}_i^l,\cdots, {u}_i^L,{p}_j^1\cdots, {p}_j^l,\cdots, {p}_j^L $$

(6)

where ${u}_i^l$ and ${p}_j^l$represent user and POI features, respectively, that were generated from the l-th meta path. Given all of the latent features in Eq. (6), the formula for the FM is as follows:

$$ {\hat{y}}^n\left(W,V\right)={w}_0+\sum \limits_{i=1}^d{w}_i{x}_i^n+\sum \limits_{i=1}^d\sum \limits_{j=i+1}^d\left\langle {v}_i,{v}_j\right\rangle {x}_i^n{x}_j^n $$

(7)

where w₀is the global bias, Wis the first-order weights for modeling the strength of the latent features, V is the second-order weights for modeling the interactions among latent features, v_i represents the i-th variable with K factors, and ${x}_i^n$is the i-th feature in Xⁿ.Particularly, d = 2LF denotes the number of latent features that are generated by L meta paths, where F is a constant, namely, the rank that is used to factor every similarity matrix.

Although learning interactions between latent features substantially improves the performance of the recommendation model, the dense feature matrices that are generated by SVD++ increase the computational cost of learning parameters. Simultaneously, several meta paths help improve prediction accuracy only a little because the information that is contained in them can be covered by others. Therefore, it is crucial to select the most discriminating features for high-dimensional data in O2O commerce LBSNs.

To overcome this problem, we employ group lasso regularization for the FM method Meier et al. 2008). The group lasso is an extension of the lasso to variable selection on predefined groups of variables and is especially suitable for high-dimensional problems.

We denote the whole parameter vector by $\beta ={\left({\beta}_{\upgamma_1}^{\mathrm{T}},\cdots, {\beta}_{\upgamma_{\mathrm{g}}}^{\mathrm{T}},\cdots, {\beta}_{\upgamma_{\mathrm{G}}}^{\mathrm{T}}\right)}^{\mathrm{T}}$. Then, the group lasso regularization is defined as follows:

$$ \phi \left(\beta \right)=\sum \limits_{g=1}^G{\left\Vert {\beta}_{\gamma_g}\right\Vert}_2 $$

(8)

where ‖∙‖₂ is the l₂-norm, ${\beta}_{\gamma_g}$ is the parameter vector that correspondsto the g-th group of variables, and γ_g is the corresponding index set for g = 1, 2, ⋯G. In this model, the groups are the meta path-based features.

For the first-order parameters Win Eq. (7), we apply the group lasso to w_l. Therefore, the regularization can be reformulated as follows:

$$ {\phi}_W(W)={\sum}_{l=1}^{2L}{\left\Vert {W}_l\right\Vert}_2 $$

(9)

where w_l ∈ ℝ^F, which models the weights of the latent feature set for the l-th meta path. For the second-order parameters Vin Eq. (7), the corresponding regularizer is

$$ {\phi}_V(V)={\sum}_{l=1}^{2L}{\left\Vert {V}_l\right\Vert}_F $$

(10)

where v_l ∈ ℝ^F × K is the l-th block of V, which corresponds to the features of the l-th meta path; and where ‖∙‖_F is the Frobenius norm.

With group lasso regularizations, our model automatically preserves useful and removes redundant features in the unit of groups. Thus, the objective function to minimize is as follows:

$$ h\left(W,V\right)={\sum}_{n=1}^N{\left({y}^n-{\hat{y}}^n\left(W,V\right)\right)}^2+{\lambda}_w{\phi}_W(W)+{\lambda}_v{\phi}_V(V) $$

(11)

where λ_Wand λ_V are the constants that control the regularization of Wand V, respectively. As the constants increase, the corresponding regularization becomes heavier.

Model optimization

Due to the use of group lasso regularization, the objective function is non-smooth. The objective function is also not convex for V. An improved stochastic variance reduced gradient (SVRG++) can effectively solve non-convex objective functions with a non-smooth ℓ₂ regularizer (Allen-Zhu and Yuan 2016). Let us denote the initial vectors by w^ϕ and v^ϕ. Then, our algorithm can be divided into S epochs, where the s-th epoch consists of m_sstochastic gradient steps. Within each epoch, we compute the full gradients ${\overset{\sim }{\mu}}_{s-1}\longleftarrow \nabla f\left({\overset{\sim }{w}}^{s-1}\right)$ and ${\overset{\sim }{\mu}}_{s-1}\longleftarrow \nabla f\left({\overset{\sim }{v}}^{s-1}\right)$, where ${\overset{\sim }{w}}^{s-1}$and ${\overset{\sim }{v}}^{s-1}$ are the average points of the previous epochs, consecutively. Moreover, m_s doublesevery two consecutive epochs, thereby distinguishing SVRG++ from other variance-reduction-based methods. As shown in line 7, ${\overset{\sim }{\mu}}_{s-1}$is used to define the variance-reduced stochastic gradient ξin every stochastic gradient step. Finally, the starting vectors of the next epoch are set as ${w}_{m_s-1}^{s-1}$and ${v}_{m_s-1}^{s-1}$, which is the ending vector of this epoch. SVRG++is presented as Algorithm 1, where η is the step length.