Construction and analysis of multi-relationship bipartite network model

Bipartite networks that characterize complex relationships among data arise in various domains. The existing bipartite network models are mainly based on a type of relationship between objects, and cannot effectively describe multiple relationships in the real world. In this paper, we propose a multi-relationship bipartite network (MBN) model, which can describe multiple relationships between two types of objects, and realizes simple weighted bipartite network reconstruction. Our model contains three major modules, namely multi-relationship bipartite network modeling (MBNM), multi-relationship aggregation module (MAM) and network reconstruction module (NRM). In MBNM, a multi-relationship bipartite network is proposed to describe multiple relationships between two types of objects. In the MAM, considering that different relationships have different information for the model, we introduce a novel relationship-level attention mechanism, and the aggregation of multiple relationships is carried out through the importance of each relationship. Based on the learning framework, the NRM can learn the potential representations of nodes after multi-relationship aggregation, and design a nonlinear fusion mechanism to reconstruct weighted bipartite network. We conducted extensive experiments on three real-world datasets and the results show that multi-relationship aggregation can effectively improve the performance of the model. In addition, experiments also show that our model can outperform existing competitive baseline method.


Introduction
In the era of information overload, the relationships in various domains can be characterized in the form of networks, including item recommendation [1], disease analysis [2], and social network research [3] to name a few. In light of the diversity of complex relationships in the real world, current research attention has been paid to the modeling of two objects on complex networks, giving rise to a novel and also powerful kind of network model, dubbed as bipartite network [4].
A bipartite network is a special network whose vertices are divided into two independent components, and edges only exist between two independent node components, and there B Bofeng Zhang bfzhang@sspu.edu.cn 1 based on traditional bipartite networks cannot describe multiple complex relationships between two types of objects.
Based on this, we propose a novel multi-relationship bipartite network (MBN) model to describe the multiple relationships between two types of objects. This model aggregates multiple relationships and reconstructs a weighted bipartite network through a multi-layer neural network, which can not only reflect multiple relationships, but also achieve bipartite network modeling. The performance of the MBN model is proved through experiments. In addition, compared with the seven methods of MF, DMF, DAE, VAE, DeepLTSC, DeepT-SQP and DLP, the MBN model has better performance. The main contributions in our paper are as follows: (1) The definition of multi-relationship bipartite network (MBN) is proposed, which effectively describes the complex multiple relationships between two types of objects in reality. (2) A novel relationship-level attention mechanism is introduced to focus on the importance of different relationships. (3) The nonlinear fusion mechanism based on depth features is designed to realize the reconstruction of the weighted bipartite network. (4) Experiments based on three real datasets demonstrate the effectiveness of our method and show that our method can outperform baseline methods.

Related work
Our work is related to the studies of bipartite network, attention mechanism, and collaborative filtering. Therefore, in this section, we briefly review the relevant literature in these areas.

Bipartite network
A bipartite network can abstract a complex system into a network composed of two types of nodes, and there are only edges between different types of nodes. It can describe complex systems with two types of objects and a single relationship, such as the purchase relationship between users and items, and the scientific research relationship between authors and papers. As shown in Fig. 1a, the bipartite network modeling of a complex system composed of user-item purchase relationships. The advantage of modeling complex systems as bipartite networks is that complex systems can be analyzed based on complex network theory, such as analyzing the stability of complex systems based on network analysis, predicting unknown relationships based on link prediction, and predicting network development trends based on network evolution. Based on its special structure, related research has received extensive attention. At present, the Bipartite Graph Neural Networks (BGNN) model based on the special structure of the bipartite network has received the greater attention [18]. To it essential characteristics, BGNN recursively updates each node feature through message passing (or aggregation) of its neighbors, by which the patterns of graph topology and node features are both captured, and then performing the corresponding recommendation. Based on the special structure of the bipartite network, this method proposes IDMP as the encoder and IDA by adversarial learning to address the node feature inconsistency issue in bipartite networks, and realize node representation learning. In addition, the GLICR model [19], the HRDR model [20] and the MARank model [21] are based on bipartite network, combined with item features and user comments for recommendation, which not only solves the problem of network sparseness, but also achieves excellent recommendation performance. Based on the bipartite network, this paper proposes a multi-relationship bipartite network (MBN) model, in which there are multiple types of edges between nodes to describe multiple relationships between objects in the real world.

Attention mechanism
The attention mechanism originates from human vision. Humans scan the global image to obtain the target area that needs to be focused on, and pay more attention to the area, while ignoring other irrelevant information. At present, in the field of deep learning, the attention mechanism mainly focuses on important features and ignores unimportant features, which are generally reflected in the form of weights. As shown in Fig. 1b, the process of paying attention to features. First, the feature is concerned, and the importance of the feature is expressed in the form of weights. Then, the weight and the feature are multiplied to obtain the attention-based feature, which amplifies the main feature and realizes the purpose of the model identifying the main feature information. Based on the attention mechanism, the importance of each latent features or factors can be distinguished to enhance the accuracy of the model. The attention mechanism is widely used in deep learning, among which the Heterogeneous Graph Attention Network (HAN) has received widespread attention [22]. Specifically, HAN is based on hierarchical attention, where the purpose of node-level attention is to learn the significance between a node and its meta-path based neighbors, and semantic-level attention can learn the importance of different meta-paths. In addition, in the recommendation system, HACN models users and items based on review text [23]. The model evaluates the contribution of each review text based on two layers of attention, and realizes the matching degree between the review texts and the target user (item). Based on this model, the feature rep-  resentation of users and items can be adaptively enhanced, and effective information can be fully utilized to reduce the interference of irrelevant information. In addition, the DANet model [24] and the CBAM model [25] are based on the attention mechanism, which can enhance the discriminative ability of feature representations. In this paper, we introduce a novel relationship-level attention mechanism to focus on the importance of different relationships, and the aggregation of multiple relationships is carried out through the importance of each relationship.

Collaborative filtering
Collaborative filtering (CF) is the main technology based on interaction recommendation [26], which aims to represent users and items through latent feature vectors. Matrix Factorization (MF) is one of the most popular techniques. Matrix factorization splits a matrix into a product of smaller matrices. An example of matrix factorization is shown in Fig. 1c.
There are unknown ratings in the rating matrix. Based on the known rating, the matrix is factorized to obtain two implicit matrices, and the unknown rating in the matrix is complemented by the implicit matrix. The MF model tries to learn the potential features of users and items by matching the user-item interaction matrix with the dot product (DP) operation [27]. Then, the rating prediction is made through the DP operation of the potential features for a given user-item pair.
With the development of artificial intelligence, it has been naturally applied to the research of recommender systems [28]. The NeuMF model realizes the combination of neural network and matrix factorization [29]. The model inputs user and item feature vectors, and replaces the DP operation with a neural architecture to achieve deep collaborative filtering with implicit feedback. Based on the deep neural network architecture, NeuMF can model the potential feature interaction between users and items, and shows superior performance than existing latent factor learning techniques.
In this paper, a multi-layer neural network is used to learn the latent features in the interaction matrix, and a nonlinear fusion mechanism is designed to realize the prediction of new interactions.

Definition of MBN
The network and the corresponding adjacency matrix .., t l } is the interaction type of different nodes. The Multi-relationship bipartite network can be represented by the adjacency matrix A M B N , where l represents the type of relationship.
As shown in Fig. 2, the schematic diagram of the network and the corresponding adjacency matrix, where U = {u 1 , u 2 , u 3 }, V = {v 1 , v 2 , v 3 }, w represents weight, and different edges represent different relationships.

The MBN model
The overview of MBN model is shown in Fig. 3. There are three modules in the MBN model: multi-relationship bipartite network modeling (MBNM), multi-relationship aggregation module (MAM) and network reconstruction module (NRM). The MBNM model multiple relationships between two types of objects, and describes complex multiple relationships in reality. Bipartite networks can only model one relationship between two types of nodes, and cannot effectively model complex systems with multiple relationships between users and items. Therefore, this paper proposes a multi-relationship bipartite network, which contains two types of nodes (users and items), and can describe various relationships between users and items. The MAM introduces a relationship-level attention mechanism to focus on the importance of different relationships, and realizes the aggregation of multiple relationships through the importance of each relationship. The attention mechanism is designed to analyze the influence of various relationships on purchase relationship, and predict the user purchasing relationship based on the importance of different relationships. The NRM designs a nonlinear fusion mechanism based on features to realize the reconstruction of the weighted bipartite network. This module learns representations of users and items through neural networks, and designs a fusion mechanism to predict user purchases of items.

Algorithm process
The workflow of MBN is depicted in Algorithm 1. First, the importance of each relationship to the target relationship (purchase) is calculated in a multi-relationship bipartite network. Next, the adjacency matrix of each relationship and its importance are weighted to obtain an adjacency matrix A, which contains all types of relationships. Then, the representation X u of the users and the representation X v of the items are learned based on neural networks. Finally, based on the product of X u and X v , the fusion scoreR of the user to purchase the item is obtained. The importance w i of each relation is calculated. 5:

Multi-relationship bipartite network modeling (MBNM)
A type of relationship is often used by the existing works for bipartite network modeling. For example, nodes represent users and items, and edges represent purchase relationships. However, there are multiple relationships between objects in the real world, and modeling based on a type of relationship cannot describe the complex relationships between objects [30,31]. Therefore, this paper proposes a multi-relationship bipartite network (MBN), in which vertices are divided into two independent components, multiple types of edges only exist between two independent node components, and there are no edges between nodes of the same type. The structure of MBN is complex, and it is more difficult to analyze than a type of relationship network. Therefore, we consider transforming the MBN into bipartite networks, where each bipartite network represents a type of relationship. Assuming that there are l types of relationships, MBN is represented as l bipartite networks, each bipartite network represents a type of relationship, and these bipartite networks are independent of each other. These l bipartite networks can be represented by adjacency matrices A 1 BN , A 2 BN ,...,A l BN , respectively.

Multi-relationship aggregation module (MAM)
In Multi-relationship Aggregation Module (MAM), we will explore how to aggregate multiple relationships between two types of objects. One popular aggregation function is the mean operation, where we can simply average the contribution of each relationship. However, different relationships are of different usefulness, and contribute differently for modeling. Hence, we propose to design a novel relationship-level attention mechanism to focus on the importance of relationships, and realize the aggregation of multiple relationships through the attention mechanism. The aggregation of multiple relationships is computed by the following formula:

Network Reconstruction module (NRM)
In Network Reconstruction Module (NRM), we will learn the representations of nodes via multi-layer neural network from adjacency matrix A. There are two multi-layer neural networks in NRM, User Network and Item Network. In the case of users and items in the recommendation system, the User Network learns the representation of users, and the Item Network learns the representation of items. Since the User Network and the Item Network have similar structures, we will focus on illustrating the User Network in detail. The same process is applied for Item Network. We utilize the multi-layer neural network, which can learn the feature of the user from the adjacency matrix A as follows: Where the input x of the User Network is a row in the adjacency matrix A (i.e., the feature of the user), h is the number of hidden layers in the neural network, σ is a sigmoid function, w represents the parameter weight, b represents the bias. Based on the learning of the neural network, the representation x h of user is obtained. The features of m users are represented as Similarly, we can also get the representation of item in a similar way.
Based on the multi-layer neural network, the User Network outputs the depth feature X u of the user, and the Item Network outputs the depth feature X v of the item, and designs a nonlinear fusion mechanism to calculate the fusion score between user and item, and then the weighted bipartite network is reconstructed based on the fusion score. The formula of this nonlinear fusion mechanism is as follows: whereR represents the fusion score, X u and X v represent the depth features of users and items, respectively, σ is an activation function. Since the nonlinear fusion mechanism obtains continuous values, this paper employs the square loss function to train MBN model. In addition, the L2 regularization term is introduced into the loss function to improve the generalization ability of the model and avoid over-fitting. Assuming that the model is over-fitting, the value of the parameter w will generally be relatively large. w can be constrained based on the size of the parameter α. Therefore, our final loss function is expressed as: where u denotes the set of users, v denotes the set of items, R i j denotes the real purchase relationship, α is the regularization parameter, w represents the parameter weight in the model, and Q represents the number of w.

Experiment
Experiments were performed on a workstation equipped with Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz, NVIDIA Geforce GTX 1080Ti GPU and 192 GB RAM. The time complexity of this method is

Dataset investigation
To evaluate the performance of MBN, we used the User Behavior dataset, which is a dataset of user behaviors from TaoBao [32][33][34]. At the same time, we also evaluate our MBN on two datasets collected from MovieLens [35] and Ciao [36]. These datasets contain user ratings for items 1-5. In this paper, the ratings are divided into five behaviors for analysis, r = 1: very dislike behavior; r = 2: dislike behavior; r = 3: neutral behavior; r = 4: like behavior; r = 5: very like behavior. This paper regards very like behavior (r = 5) as the main relationship, and other behaviors (r = 1−4) as auxiliary relationships. In this paper, the User Behavior dataset is introduced in detail and used as the main research object to analyze the impact of user relationship on purchases. This dataset randomly selected users who have behaviors including click, purchase, adding item to shopping cart and item favoring during November 25 to December 03, 2017. The dataset contains 4 different types of behaviors, they are Pv: page view of an item's detail page, equivalent to an item click, Fav: favor an item, Cart: add an item to shopping cart, Buy: purchase an item. In this paper, we take the item category as the item, where Buy is the label, and Pv, Fav and Cart are the multiple relationships between users and items. In addition, we select three small datasets as experimental dataset, as shown in Table 1. Based on the same data processing for datasets MovieLens and Ciao. In this paper, the three relationships of Pv, Fav, and Cart are used as auxiliary relationships, and purchase relationships are used as targets. Distinguish which types of auxiliary relation- ships are more important in the forecasting task on the target relationship. In the MB-GMN model, the purchase behavior is regarded as the target behavior, and other behaviors (page view, add-to-cart) are regarded as auxiliary behaviors [40]. In the CML model, the purchase behaviors are set as the target behaviors and other types of interactions are considered as the auxiliary behaviors [41]. In the KHGT model, page view, add-to-cart, and add-to-favorite are used as auxiliary behavioral signals to predict the impact on target behaviors (purchases) [42]. These research show that it is feasible to predict purchase relationship based on auxiliary relationships. Therefore, this paper predicts the target relationship (purchase) based on the auxiliary relationship (Pv, Fav, Cart). In this paper, B R is based on statistical knowledge. The B R idea is the ratio of purchases in Pv relationships, the ratio of purchases in Fav relationships, and the ratio of purchases in Cart relationships. B R counts which relationship has a greater impact on the purchase relationship. The meaning of the formula is the proportion of the target relationships under certain auxiliary relationships, which reflects the importance of the auxiliary relationship. As shown in Fig. 4, an example of B R calculation, the Cart relationship is carried out for three items, and only two of them are purchased. Therefore, B R is calculated to be equal to 2/3, which reflects the importance of the Cart relationship to the purchase. BR = buy behavior (8) Among them, behavior represents the number of interactions between user and the item based on a type of relationship, and buy represents the number of purchases based on this relationship.
As shown in Table 2, based on the Cart relationship between user and item, BR is the highest, which indicates that the Cart relationship is the most important when purchasing item, and this relationship can better guide users to purchase item. Based on the PV relationship between user and item, BR is the lowest, which shows that this type of relationship has a weaker impact on the purchase of item. In addition, the number of PV relationships is the most, and the bipartite network based on this type of relationship is dense; Buy: purchase of items Cart: add an item to shopping cart  the number of Fav relationships is the least, and the bipartite network is the sparsest.

Baseline method
We evaluate our method with the following baseline methods: MF [37]: Based on matrix factorization, sparse matrix can be factored into low-dimensional latent vectors, and potential relationships can be mined based on latent vectors. At the same time, the program is simple and easy to implement.
DMF [38]: Based on the latent vectors obtained by MF and the characteristics of machine learning, the deep latent features between users and items can be mined.
DAE [39]:Based on the characteristics of high dimension and sparsity of the data, it is necessary to reduce the dimension of the data. This paper selects DAE for processing to ensure the robustness of the output data. The method introduces noise into the input data, compresses the data into a lowdimensional feature space based on encoding, and restores noise-free data based on decoding, and represents nodes by low-dimensional features.
VAE [43]: VAE can learn the smooth hidden state of input data, and the encoded data can not only be effectively distinguished, but also have intersection in distribution. This method is based on AE (Auto-Encoder), in which the encoder represents the potential features based on probability distribution, and then reconstructs the input based on the decoder.
DeepLTSC [44]: This method is a novel implicit service feature extraction and service feature augmentation model, which can extract implicit features and enhance them, which can comprehensively improve the prediction accuracy on all of service categories.
DeepTSQP [45]: This method proposes a new feature representation method, and the hidden features are mined in the context content of nodes to realize QoS prediction.
DLP [46]: The method extracts the latent features between target nodes based on the local structure of the bipartite network, and completes the link prediction based on the deep learning framework.

Evaluation metrics
The performance of the model is evaluated based on 5fold cross validation. The 5-fold cross validation divides the dataset into 5 independent equal parts, and then one part (20%) is used as the test dataset, and the remaining 4 parts (80%) are used as the training dataset. This process is repeated 5 times, and the trained model is obtained on average.
The well-known Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) are adopted for performance evaluation [47,48]. MAE reflects the error of the measurement, and RMSE reflects the precision of the measurement. The main difference between MAE and RMSE is that the measurement methods are different, and the performance of the model is evaluated based on the error. In this paper, there is a purchase relationship between user and item, the target value is 1, and no purchase relationship is regarded as 0. Three relationships, Pv, Fav, and Cart, are input into our model to predict purchase relationship values. At the same time, the MovieLens and Ciao datasets are used as experimental subjects. These datasets contain user ratings for items 1-5. In this paper, a score of 5 is considered the target relationships, and 1-4 are considered auxiliary relationships. The model is Where r i j is the true value, if there is a relationship between i and j, r i j = 1, otherwise r i j = 0,r i j is the predicted value of the MBN model, and L is the number of observation samples.

Performance comparison
We compared our method with the above baselines in terms of MAE and RMSE, and the results are shown in Table 3.
We have the following observations: (1) MBN achieves the best performance on all the datasets, which consistently and significantly outperforms all the baselines. It indicates that MBN is beneficial to describe the purchase relationship between users and items. ing that the fluctuation of the prediction results is small, and the performance is stable.

Result on a type of relationship
In this section, modeling is based on a type of relationship, that is, only one relationship is used for modeling in the MBN model. As shown in Table 4, based on the Cart relationship, this model has the smallest MAE and RMSE, which shows that the Cart relationship has the most significant impact on purchases, and the results are consistent with the dataset investigation.

Result on two types of relationship
In this section, modeling is based on two types of relationships. As shown in Fig. 5 Table 5, the best performance based on two types of relationship. When aggregation is based on Pv and Cart, MBN has the smallest MAE and RMSE on the three datasets. This shows that compared to the aggregation of other relationships, the aggregation of Pv and Cart can reflect the relationship characteristics of users when buying items.

Result on multi-relationships
This section will verify the performance of multirelationship aggregation. Modeling is based on three types of relationships to analyze the performance of the MBN model. As shown in Fig. 6, when the Attention of Pv, Fav and Cart are 0.2, 0.1 and 0.7 respectively, MBN has the best performance in terms of MAE on the three datasets.
As shown in Table 6, compared to the above work, the aggregation of the three types of relationships can improve the performance of the model, and the average MAE is reduced by 0.045, 0.043, and 0.040, respectively. This shows that the aggregation of multiple relationships can truly describe real purchase relationships.

Discussion
The MBN adds complexity to represent a group of independent bipartite networks. It is thus simpler to have those independent bipartite networks and perform any necessary cross-network computations when needed. However, cross-network computation requires a large number of edge relationships, such as cross-network computation between drug-target, drug-disease, and disease-RNA networks. These cross-network computations require a large number of edge relationships, not only drug-target relationships, but also drug-drug, disease-disease relationships [49][50][51]. The datasets in the paper are very sparse, with sparseness of 0.101%, 0.091%, and 0.088%, respectively, which shows that cross-network computation is not advantageous. As shown in Fig. 7a, a cross-network example of multi-relationship sparse network, there is disconnection between the networks, and cross-network computation cannot be performed. Based     on cross-network computation, additional edge relationship information is required, such as adding edge relationship information between u 2 and u 3 or between v 1 and v 3 to connect each network. However, user-user and item-item relationships do not exist in the dataset. At the same time, users and items are represented by ids, and the ids are independent of each other. Moreover, the user's information involves privacy, and more additional information cannot be obtained to establish the edge relationship. Therefore, we do not perform cross-network computations.
Our method is represented based on a group of independent bipartite networks, whose purpose is to obtain the importance of each relationship based on the attention mechanism, and to obtain the adjacency matrix containing various relationship information based on the attention weight. Then the latent features X u and X v of users and items are mined based on the adjacency matrix. Finally, the adjacency matrix R is completed based on the idea of matrix factorization. Specifically, as shown in Fig. 7b. The essence of our method is to learn the latent features of users and items based on a group of independent bipartite networks and predict the purchase relationship between users and items based on the features. This method can effectively deal with sparse networks and make up for the shortcomings of cross-network computation.

Conclusion
In this paper, based on the multiple relationships between two types of objects in the real world, a multi-relationship bipartite network (MBN) model is proposed. This model introduces a relationship-level attention mechanism that aggregates various relationships based on the importance of the relationship. At the same time, a nonlinear fusion mechanism is designed to reconstruct the weighted bipartite network based on the depth features. Extensive experiments have shown that MBN can better describe the multiple relationships between users and items in e-commerce. The relationship between two types of objects in the real world is generally multiple, and modeling based on only one type of relationship cannot describe complex relationships. This model belongs to a more general framework, which can model bipartite networks based on multiple relationships, such as multiple relationships between users and items, drugs and diseases, and researchers and papers, etc. There are still some problems to be studied in the future. For example, there are relationships between objects of the same type in the real world, and these relationships are also of great significance for depicting complex systems in reality. In the future, we will establish edges between nodes of the same type in MBN, aiming to describe complex relationships in reality more truly and effectively.