1 Introduction

We are currently living in the age of data, where almost every task we engage in our daily lives relies on data and can be digitally tracked and supported. In our daily routines, such as shopping, working, socializing, communicating, relaxing, and traveling, we generate vast amounts of diverse data, including numeric variables, images, videos, music, and text [1, 2]. Data science, as a multidisciplinary field that combines mathematics, statistics, and computer science, employs scientific methods, processes, algorithms, and systems to extract knowledge and insights from both structured and unstructured data. Its ultimate objective is to facilitate decision-making processes. In this context, recommender systems have emerged as one of the most significant applications of data science [3]. Recommender systems utilize advanced analytics and learning techniques to sift through extensive data and provide users with relevant and valuable information, thereby assisting them in making informed decisions regarding their everyday needs. These needs encompass a wide range of aspects in our daily work and life, including food, clothing, housing, travel, work, study, and entertainment.

Since the introduction of the first recommender system called ‘Tapestry’ in 1992 [4], recommender systems have achieved remarkable success over the past 30 years. Various types of data with distinct characteristics, including rating data, review data, click/view data, user profile data, and item description data, have been extensively utilized to develop increasingly powerful recommender systems. Simultaneously, due to the inherent complexity, heterogeneity, couplings, dynamics, and uncertainty associated with such data, intricate and advanced data science theories, methods, and techniques have been employed to meticulously analyze this data and extract valuable and informative knowledge for constructing recommender systems. Considering their roles in building recommender systems, these aforementioned data science methods and techniques can generally be categorized into three classes: data preprocessing methods (e.g., techniques for handling missing values), data representation learning methods (e.g., embedding methods for item IDs), and prediction methods for recommendation results (e.g., matrix factorization).

While significant progress and success have been achieved in the field of recommender systems, the primary focus and goal have predominantly revolved around continuously enhancing the recommendation accuracy of these systems over the past few decades [2]. It is only in recent years that individuals from academia and industry have begun to shift their attention towards the trustworthiness of recommendations, which encompasses aspects such as robustness, fairness, transparency, explainability, and security within recommender systems. This shift has led to the emergence of a new recommendation paradigm known as trustworthy recommendation [2], which should be a primary focus for next-generation recommender systems. To attain this objective, a comprehensive set of advanced and state-of-the-art data science and machine learning theories and methods are essential.

Despite the significant progress made in both data science and recommender systems, there has been a lack of systematic investigation and exploration into the relationship between these two fields. No work has comprehensively illustrated the specific roles data science plays in the development of recommender systems, nor has it provided a clear understanding of how advancements in data science research can benefit and advance recommender systems. Therefore, this paper aims to fill this gap by offering a systematic and comprehensive overview of the synergistic relationship between data science and recommender systems. In particular, we focus on elucidating the role of data science throughout the different stages of the recommendation process, ranging from data collection and preprocessing to data representation, prediction of recommendation results, and the construction of next-generation recommender systems.

The structure of the following sections of this paper is as follows: Sect. 2 will provide an illustration of the various types of data that can be utilized for recommendations, along with the corresponding techniques for representing each type of data. In Sect. 3, we will provide a brief outline of representative and commonly used data mining and machine learning models employed in building recommender systems. Subsequently, in Sect. 4, we will present our understanding and thoughts on next-generation recommender systems, along with the relevant data science techniques that can be employed to construct them. Section 5 will summarize the accepted papers in this special issue, and finally, in Sect. 6, we will conclude the paper.

2 Data and data representation for recommendation

In this section, we will conduct a systematic review of various data sources and data types commonly employed in the construction of recommender systems to generate recommendations. It is important to note that raw data from different sources cannot be directly input into machine learning-based recommender systems. Instead, they often require specific formats (e.g., low-dimensional vectors) that can be directly utilized for calculations within machine learning models. Therefore, each type of data is typically accompanied by a specific data representation learning method, aimed at effectively capturing and representing the unique characteristics of the data. Accordingly, we will outline the corresponding methods utilized to represent each type of data, ensuring their suitability for building machine learning-based recommender systems.

2.1 Categorical data and its representation

In the field of recommendation, various types of categorical data from different sources are commonly utilized, including components of user profile data (e.g., gender, community), item feature data (e.g., category, brand), and user-item interaction data (e.g., click, view). To facilitate the computation of machine learning models, particularly neural models, these categorical values are often encoded into vector representations, such as one-hot encoding vectors or low-dimensional numerical embedding vectors. In recent years, with the rapid development of embedding techniques, a multitude of methods have emerged for encoding categorical values from their original space into a latent space, thereby representing each categorical value as a low-dimensional numerical vector. For instance, the Word2Vec-like model [5] is commonly employed to embed an item/user ID, item category, etc., into a latent vector [6, 7].

2.2 Numerical data and its representation

In addition to categorical data, numerical data is also commonly used in recommender systems. Representative numerical data includes users’ ratings on items, certain item feature values like product price, and certain user profile data such as user age. Unlike categorical data, numerical data can be directly used in calculations. For instance, in user-item rating prediction tasks, the rating values are typically estimated using methods like matrix/tensor factorization and neural models. Occasionally, to facilitate computation alongside other data types (e.g., categorical data), certain numerical data values may also be encoded into low-dimensional embedding vectors, utilizing similar representation learning models as those employed for categorical data.

2.3 Textual data and its representation

Textual data used for recommendation typically includes users’ reviews, comments on items, item descriptions, news item content, and so on. To effectively represent such textual data and incorporate it into recommendation models, various representation methods can be employed. These methods include conventional techniques like TF-IDF, which is based on word frequency, topic models such as LDA, and newer word embedding models like Skip-gram and continuous bag-of-words [8, 9]. More recently, advanced language models like BERT and Transformer have become popular for learning more informative representations from given text. In these models, each word or sentence is represented as a numerical vector, which can be further utilized as input for downstream recommendation models.

2.4 Image data and its representation

Image data plays a crucial role in recommendations, particularly for appearance-sensitive items like jewelry, fashionable clothes, and decoration items. The visual appearance of these items often significantly influences user preferences. Representative methods for representing image data include conventional feature extraction techniques such as the scale-invariant feature transform method [10] and the histogram of oriented gradients method [11], as well as deep learning methods. In recent years, convolutional neural networks (CNN) [12] have emerged as a powerful approach for learning informative representations from images. Various deep models based on CNN, such as VGG [13] and ResNet [14], have been developed to improve the representation of image data. These representations can be incorporated into recommendation models to better characterize items and capture users’ preferences towards the appearance aspect of items.

2.5 Sequence data and its representation

Sequence data is prevalent in the field of recommendation and encompasses various types and sources [15, 16]. Some examples include a user’s sequential interaction behaviors (e.g., click, view, purchase) with items during an online shopping session [17,18,19], a user’s sequence of visited points of interest (POIs) during a trip [20], a sequence of news articles read by a user [21,22,23], or a sequence of songs/videos listened to/watched by a user. Various methods are employed to represent sequence data in recommendation systems, including embedding techniques that capture first-order sequential dependencies and deep neural networks that consider higher-order dependencies. Recurrent neural networks (RNN) are widely used as representative deep neural networks for sequence representation. More recently, complex neural architectures have been proposed to capture intricate sequential dependencies. For instance, in sequential or session-based recommendation, graph neural networks (GNN) have been utilized to model skip dependencies between distant items in sequences or dependencies within sequences containing duplicate items [24,25,26]. Temporal convolution networks (TCN) have also been employed to capture long-range dependencies across multiple items in sequences/sessions for recommendations [27]. It is important to note that alongside these neural representation learning methods, conventional sequence models such as sequential pattern mining and Markov chain models have also been utilized to handle sequence data for recommendation purposes. For more detailed information on sequential or session-based recommendation, please refer to our systematic surveys [15, 16] and tutorial [28] on this topic.

2.6 Graph data and its representation

Graph data is not uncommon in recommender systems, as most of the data used for recommendation can be represented as graphs [25, 29]. This is because the majority of the data consists of relational information, which naturally lends itself to graph representation. Typical examples of graph data in recommendations include the bipartite graph connecting users and items, where each edge represents a user-item interaction. Another example is the social graph connecting users, where edges represent different types of social relations between users. To effectively represent graph data, there are generally three approaches: (1) random walk-based methods that capture complex higher-order and indirect relations between different types of nodes (e.g., users and items) in the graph; (2) graph embedding techniques that map each node to a low-dimensional embedding vector to encode the graph’s structural information; and (3) Graph Neural Networks that leverage the power of GNN to learn representation vectors for each node in the graph and capture complex inter-node relations [29]. For more detailed information on graph learning-based recommendation, please refer to our systematic survey on this topic [29].

3 Data mining and machine learning models for recommendation

Once the data for recommendation has been collected and properly represented, it can be input into prediction models to generate recommendation results. These prediction models are typically data mining or machine learning models. It is worth noting that for some conventional models like pattern mining or rule learning, the original data (e.g., user-item interaction records) can be used directly without another representation format [30].

Over the past few decades, a wide range of data mining and machine learning models have been employed in building recommender systems, and many of them have achieved significant success. These models span from traditional and straightforward pattern mining-based methods to advanced and complex deep learning-based methods, which have gained popularity in recent years.

In this section, we will provide a brief overview of various data mining and machine learning models used in recommender systems. Due to space limitations, we will focus on commonly utilized models in each category. Specifically, we will categorize these models into four main classes: pattern/rule mining models, latent factor models, embedding models, and neural models. We will provide a brief introduction to each of these categories in the following sections.

3.1 Pattern/rule mining models

Pattern/rule mining models serve as a naive approach to generate recommendations. They are primarily used to uncover association or correlation relationships among different items, contents, or services, and then utilize those relationships to generate recommendations. One well-known example is the “beer and diapers” case [31], where transaction data from Walmart in the US revealed that young fathers tend to purchase beer and diapers together during a shopping visit, leading to effective sales promotion of both items by packaging them together.

Pattern mining-based recommender systems can be categorized into two types based on whether the order of user-item interactions is considered or not: (1) frequent pattern/association rule mining-based methods that mine association rules among user-item interactions without considering order, and (2) sequential pattern mining-based methods that mine sequential patterns from sequences of interactions [16]. Both types of approaches for recommendations involve three main steps: frequent/sequential pattern mining, pattern matching, and recommendation generation [16].

Pattern mining based methods are mainly utilized in the early stage during the development of recommender systems. They are simple yet effective sometimes depending on the data characteristics. Typical work on frequent pattern/association rule mining-based recommendation and sequential pattern mining-based recommendation include [32, 33] and [34, 35]. A more detailed review and summarization of pattern/rule mining based approaches for recommender systems can be found in [16].

3.2 Latent factor models

Latent factor models, including matrix factorization and tensor factorization models, are commonly used in recommender systems to implement collaborative filtering, which is one of the fundamental concepts in recommendation systems [36]. These models aim to factorize the observed user-item interactions, such as ratings, into latent vector representations for users and items. By utilizing these factorized representations, latent factor model-based recommender systems can estimate unobserved interactions and make recommendations.

In latent factor model-based recommender systems, the focus is often on completing the matrix or tensor representing user-item interactions. This completion task involves predicting the missing values in the matrix or tensor using the observed entries. The matrix or tensor typically represents the user-item interactions, such as a user-item rating matrix, user-item click matrix, or item-item transition matrix/tensor [37]. Latent factor models were widely used in the field of recommendation during the early 2010s. For a more comprehensive review and summary of latent factor models in the context of sequential/session-based recommendation, please refer to the studies conducted by Wang et al. [16, 28].

3.3 Embedding models

Embedding models play a crucial role in recommendation systems by employing various embedding methods, typically shallow neural networks, to transform different objects (such as items, users, and interactions) into latent embedding vectors. These embedding vectors serve as input for downstream prediction models, which generate recommendations. The concept of embedding models originated from the influential word2vec model, which includes the Skip-gram and Continuous Bag-of-Words (CBOW) models. Since then, a wide range of embedding models has been developed for different types of data in recommendation. For instance, in sequential/session-based recommendation, word2vec-like models are commonly used to learn embedding vectors for item IDs in a sequence of items interacted by a user. These models capture the contextual information surrounding the target item and have been applied successfully in tasks such as session-based recommendation [6, 7]. Another example is the Node2Vec model, which learns embedding vectors for nodes in a graph by considering the graph’s structural information. Node2Vec has been widely utilized to learn user embedding vectors in social networks, enabling the capture of inter-user relationships in social-enhanced recommendation systems [38]. Embedding models offer two key advantages. Firstly, they are capable of capturing contextual information, allowing for a more nuanced understanding of the target item or user. Secondly, they typically have a simple yet effective model structure with low computational complexity, making them practical for large-scale recommendation tasks. As a result, embedding models have shown significant potential and have been widely adopted in the field of recommendation over the past decade.

3.4 Neural models

In recent years, there has been a rapid development in neural networks, particularly deep neural networks, which have demonstrated significant potential in learning complex relationships among entities such as items, users, and interactions. These networks are capable of extracting highly informative latent representations from raw data, which greatly benefit the downstream prediction and recommendation tasks. As a result, neural models built on neural networks have been widely adopted in the field of recommendation to improve recommendation performance.

Neural model-based recommender systems can be categorized based on the architecture of the neural network they utilize. Some common classes of neural models include: (1) MLP-based approaches: These models employ multi-layer perceptrons (MLP) to learn the representation of users and items. They typically use feedforward neural networks to capture non-linear interactions between user and item features; (2) RNN-based approaches: Recurrent neural network (RNN) models are utilized to capture sequential dependencies in user-item interactions. They are well-suited for sequential recommendation tasks, such as session-based recommendation, where the order of interactions is crucial; (3) CNN-based approaches: Convolutional neural networks (CNN) are primarily designed for image processing, but they have also been applied to recommendation tasks. CNN-based models can extract meaningful features from item images or other visual data, enhancing the representation of items; (4) GNN-based approaches: Graph neural networks (GNN) are used to model complex relationships among users and items represented as a graph. GNN capture the interactions between connected nodes in the graph, enabling personalized recommendations based on the underlying network structure; (5) Attention-based approaches: These models, including transformer and similar architectures, focus on capturing important and relevant information within a sequence or graph. They use self-attention mechanisms to assign weights to different elements, emphasizing the most relevant ones for generating recommendations; and (6) Large model based approaches, which generally utilize the recently developed advanced large language models (e.g., GPT-4) to complete various recommendation tasks.

In addition to these purely neural network-based approaches, there are hybrid models that combine neural networks with other models such as matrix factorization, point process [39], or probabilistic graphical models. These hybrid approaches leverage the strengths of different models to enhance recommendation accuracy and flexibility. Comprehensive surveys on deep learning for recommender systems can be found in Zhang et al. [40] and Wang et al. [16], which provide a detailed overview of various deep learning models for recommendation, including both general recommender systems and sequential/session-based recommender systems.

3.4.1 MLP-based approaches

MLP layers are often used to transform raw data and sparse features into dense latent features for recommendation purposes [41]. They are capable of capturing complex patterns and relationships in the data, enabling more effective downstream prediction and recommendation. Additionally, MLP layers can be employed to combine multiple data sources, features, or modalities into a unified representation, facilitating the integration of diverse information for improved recommendations [42]. Moreover, MLP layers are frequently combined with other machine learning models to enhance their predictive capabilities. A notable example is the combination of MLP with matrix factorization in neural collaborative filtering, which has shown great potential in improving recommendation performance [43, 44].

3.4.2 RNN-based approaches

RNN have been widely adopted in the field of recommender systems due to their ability to effectively model sequential data and capture the dependencies present in historical user-item interactions and other sequence data. They are particularly well-suited for sequential/session-based recommendation tasks, where long-term and high-order dependencies need to be captured in relatively long sequences or sessions [16, 18]. In the context of recommender systems, RNN based approaches have been applied in various studies, including works such as [20, 45, 46]. These approaches leverage RNN to learn representations of sequential data, enabling them to make personalized recommendations based on users’ historical interactions. Furthermore, RNN are often combined with other machine learning models, such as attention mechanisms, to enhance their ability to capture complex dependencies in user-item interactions and improve the accuracy of recommendations [20]. The combination of RNN with attention mechanisms allows for a more fine-grained modeling of the relationships between different elements in the sequences, leading to more accurate and context-aware recommendations.

3.4.3 CNN-based approaches

CNN are widely used in recommender systems for two main purposes: (1) Learning abstract latent representations: CNN are effective in learning high-level and abstract latent representations from raw input features, particularly for image-related features such as product appearance images. In fashion recommendation, for example, CNN are commonly employed to learn informative latent representations for clothing images, enabling more effective recommendation models [47, 48]. By extracting relevant features from images, CNN can capture visual patterns and characteristics that contribute to personalized recommendations; (2) Modeling complex relationships in user-item interactions: In sequential/session-based recommendation scenarios, CNN are utilized to address various challenges. They can relax the rigid-order assumption over user-item interactions, allowing for more flexible modeling. CNN can capture local features within a certain area of a session, enabling the identification of important patterns and dependencies. They can also learn union-level collection dependencies, considering relationships among different areas within sessions. By applying CNN, recommendation models can effectively capture and utilize complex relationships among user-item interactions, improving the accuracy and quality of recommendations [15, 16, 28].

3.4.4 GNN-based approaches

GNN have gained significant attention and development as a deep learning paradigm specifically designed for graph data. In recommender systems, where data points are often interconnected, GNN have emerged as a natural and effective approach to capture complex relationships and improve recommendation performance. Various types of recommendation tasks and application domains have benefited from the application of GNN, including rating prediction, top-N recommendation, sequential recommendation, social-relation enhanced recommendation, and domains like product recommendation [25], POI-recommendation [20], news recommendation [23], etc. GNN have been widely used in recommender systems to comprehensively capture the relationships among different entities, such as items, users, and interactions. The data can be represented as a graph, where users are connected through social relations and items are connected based on similarity (e.g., category or brand). By leveraging GNN, recommendation models can effectively learn and exploit these relationships to enhance recommendation performance. GNN-based recommender systems can be classified into three main categories based on the specific model structure: gated graph neural network (GGNN) based recommender systems [19, 49], graph attentional network (GAT) based recommender systems [50, 51], and graph convolutional neural network (GCN)/GraphSage based recommender systems [52, 53]. Each category of GNN models has its unique architecture and mechanisms for capturing graph information and conducting recommendation tasks.

Actually, Wang et al. [29] proposed the concept of “graph learning based recommender systems” and provided a comprehensive review and summary of various graph learning approaches, including random walk models, graph embedding methods, and GNN, which can be utilized in building recommender systems. Wu et al. [54] conducted a comprehensive survey specifically focusing on GNN-based recommender systems, exploring the latest advancements and applications in this field.

3.4.5 Attention/transformer-based approaches

Attention models have been generally introduced into the recommendation area to capture selective information (e.g., item features, user attributes, historical interactions) that is more useful and relevant to the recommendation task. For example, attention models are widely utilized in sequential/session-based recommendation to select the historical interactions, items or shopping baskets that are relevant to predicting the next item [21, 22, 55] or the next basket of items [56, 57]. Specifically, [6] is a pioneering study that introduced attention models into session-based recommender systems to select useful and relevant interactions within sessions for next-item recommendation. In addition to basic attention mechanism-based recommender systems, there are several variants, including self-attention and soft/hard attention-based recommender systems, among others.

Transformer [58], as a more complex model built on the basis of the self-attention mechanism, has shown great potential in various machine learning tasks, including recommendation, in recent years. In the recommendation area, Transformer has been frequently employed in sequential/session-based recommendation to comprehensively learn complex and long-range/term sequential or non-sequential dependencies for accurate recommendations. Representative studies in this field include [59, 60].

3.4.6 Large model based approaches

In recent years, large models, especially large language models, have flourished. Models like GPT-3.5 and GPT-4 have demonstrated significant potential in acquiring intelligence and knowledge from vast amounts of text data. What’s even more remarkable is their high generalization ability, allowing them to perform a wide range of tasks by framing them as language-based tasks. Naturally, researchers in the field of recommendation have explored the use of large models for generating recommendations. For instance, Geng et al. [61] introduced the “Pretrain, Personalized Prompt, and Predict Paradigm” (P5) for recommendation. P5 is a flexible and unified text-to-text framework that encompasses various recommendation tasks within a shared framework. Experimental results have shown its superiority in performance. Furthermore, other researchers have conducted empirical studies to investigate whether emerging models like ChatGPT can serve as an effective recommender system [62,63,64].

4 Next-generation recommender systems: trustworthy recommender systems

During the past 30 years, since the introduction of the first recommender system “Tapestry” in 1992 [4], the primary focus of research in the recommendation field has been on improving recommendation accuracy. Accuracy has been the dominant evaluation metric for assessing recommendation performance. However, in recent years, there has been a growing recognition among researchers about the importance of factors beyond accuracy, with trustworthiness of recommendations emerging as a central issue. In other words, the research focus is shifting from solely improving recommendation accuracy to generating trustworthy (e.g., fair, explainable) recommendation results that stakeholders can rely on and trust [2].

4.1 An overview of trustworthy recommender systems

A trustworthy recommender system is characterized by its ability to provide reliable and dependable recommendations that stakeholders, such as users and providers, can rely on with confidence. It goes beyond accuracy and encompasses attributes such as trustworthiness, reliability, and faithfulness [2]. Trustworthy recommendation models should not only accurately capture user preferences and item characteristics to generate accurate recommendations, but also demonstrate stability and effectiveness in complex, dynamic, and challenging contexts [2]. Wang et al. [2] have conducted a comprehensive overview of the research area of trustworthy recommender systems, providing a systematic exploration of the motivation, definitions, concepts, challenges, and solutions. Their work serves as a pioneering effort in this field, presenting a novel trustworthy recommendation ecosystem that includes trustworthy users, items, providers, and recommendation models. Additionally, they propose nine important aspects that contribute to the comprehensive characterization of trustworthy recommender systems. These aspects include robustness, fairness, transparency, explainability, privacy and security, responsibility, human perception of trustworthiness, trustworthiness integration, and trustworthy evaluations.

4.2 Data science and machine learning techniques for building trustworthy recommender systems

In order to enhance the trustworthiness of recommender systems, numerous studies have explored various data science and machine learning techniques. These approaches can be applied in different stages of the recommender system pipeline, including data representation, recommendation generation, and evaluation. During the data representation stage, debiasing and denoising models like Denoising Autoencoders (DAE) [65] can be utilized to remove noise and biases from raw data, enabling the learning of more accurate representations for recommendations. Additionally, data visualization techniques such as t-SNE [66] can be employed to visualize the learned latent representations, making them more transparent and explainable. In the recommendation generation stage, disentanglement learning-based approaches [67] can help mitigate biases in the recommendation process. Set-based techniques can also be employed to present recommendation models to end users in a more natural language format, improving the transparency of the recommendations [68].

For the evaluation stage, the development of new evaluation protocols and metrics is essential for comprehensive assessments of the trustworthiness of recommendations. Moreover, the emerging field of causal machine learning offers opportunities to enhance the trustworthiness of recommender systems. Causal machine learning not only addresses biases but also reveals causal relationships among different entities in recommendations, thereby increasing the explainability and transparency of the recommendation process [69, 70].

5 About this special issue

5.1 Related events/activities for promoting the research in recommender systems

This special issue reinforces our continued efforts to promote the research and innovation in the area of recommender systems. It follows the previous events and activities, including (1) a series of special/topical issues we co-organized at premier data science journals; (2) a set of workshops or special sessions on recommender systems we co-hosted at prestigious data science conferences; and (3) a set of research tutorials we co-delivered at top data science and AI conferences.

To be specific, some selected special/topical issues co-organized by us include the Special Issue on “Intelligent Recommendation with Advanced AI and Learning” with IEEE Intelligent Systems (2019–2020) [1], the Special Issue on “Data Science for Next-generation Recommender Systems” with International Journal of Data Science and Analytics (JDSA) (2021–2023), the Special Issue on “Recommender Systems and Their Advanced Application” with Applied Sciences (2022–2023), and the Special Issue on “Deep Reinforcement Learning for Recommender Systems” with ACM Transactions on Recommender Systems (2023). Some selected and relevant workshops co-organized by us include International Workshop on “Model Selection and Parameter Tuning in Recommender Systems” co-located with CIKM 2019 [71], the First to Fourth International Workshop on “Advanced Neural Algorithms and Theories for Recommender Systems (NeuRec)” co-located with ICDM 2020 to 2023 (the fourth workshop is to be held in December, 2023), the International Workshop on “Data Science and Artificial Intelligence on Responsible Recommendation (DSAI4RRS)” with KDD 2022 [3], the Special Session on “Data Science for Trustworthy Recommendation” at DSAA 2022, the Workshop on “Interactive Recommender Systems” with WSDM 2023 [72]. Some of our co-delivered research tutorials include the Tutorial on “Coupling Everything: A Universal Guideline for Building State-of-The-Art Recommender Systems” delivered at IJCAI 2019, the Tutorial on “Next-Generation Recommender Systems and Their Advanced Applications” delivered at IJCAI 2020, the Tutorial on “Towards Ubiquitous Recommender Systems: Data, Approaches, and Applications” delivered at AAAI 2021, the Tutorial entitled “Complement, Composite and Context: The 3C-Law to Build Multidomain Recommender Systems” delivered at ICDM 2021, the Tutorial entitled “Sequential/Session-based Recommendations: Challenges, Approaches, Applications and Opportunities” delivered at SIGIR 2022 [28], and the Tutorial entitled “Lifelong Learning Cross-domain Recommender Systems” delivered at TheWebConf 2023 [73].

5.2 An overview of this special issue

Since the release of the call for papers for this special issue, we have received a significant number of submissions, demonstrating the high quality of research in this field. Following a rigorous review process, we have selected 10 papers to be included in this special issue for publication. We would like to express our gratitude to Prof. Longbing Cao, the former and founding EIC of JDSA, and Prof. João Gama, the EIC of JDSA, for their invaluable support and assistance in planning and organizing this special issue. Additionally, we extend our appreciation to the reviewers for their selfless dedication to reviewing the papers for this special issue, as well as to the authors for their valuable contributions. To facilitate the readers, we have organized the accepted papers into three sections based on their focused research problems. We will now provide a brief summary of the main contributions of each of the accepted papers, introducing them in succession.

5.3 Implicit and sequential recommendation

In the context of this special issue, the research on recommendations based on users’ implicit or sequential interaction behaviors has been a prominent topic. One of the papers, authored by Kelen et al. [74], focuses on explaining the inner workings of nearest neighbor models for implicit recommendations using probabilistic tools. They propose a novel approach that treats similarity as conditional probability, aiming to provide insights into the recommendation process and address the issue of popularity bias. Another paper by Zhou et al. [75] explores the incorporation of attenuated sentiment information into sequential recommender systems. They investigate how users’ sentiments can influence their purchase behaviors and preferences, and propose a method to capture and utilize this sentiment information in the recommendation process. By considering the attenuated sentiment, the authors aim to enhance the personalization and accuracy of sequential recommendations.

5.4 Characterized application areas of recommendation

In this special issue, there are four papers that focus on the application of recommender systems in different domains beyond the traditional areas, including tourism recommendation, POI recommendation, purchase behavior recommendation, and scholar recommendation. By exploring these domains, the authors contribute to the advancement and practical applications of recommender systems beyond traditional areas. Chung et al. [76] present a novel approach called TPEDTR, which leverages customers’ card transaction data to recommend tourism services to tourists. By analyzing the transaction data, the system identifies patterns and preferences of tourists, enabling personalized recommendations for tourism activities and services. Laroussi [77] proposes a deep meta-level spatio-categorical POI (Point-of-Interest) recommender system (DML-SC) that incorporates additional spatial and categorical information to improve the accuracy of POI recommendations. By considering the spatial and categorical characteristics of POIs, the system enhances the relevance and quality of recommendations for users. Lysenko et al. [78] propose a generative model of client spending based on the temporal point processes framework. This model predicts the expected time and category of a client’s next purchase, providing valuable insights for personalized recommendations and marketing strategies in the context of customer behavior prediction. Desai et al. [79] develop ScholarRec, a recommender system that combines scholastic influence and social collaborations to identify influential scholars from Academic Social Networks (ASNs). By considering both the impact of scholarly works and the influence of social interactions, ScholarRec provides a comprehensive approach to identifying influential scholars in the academic community.

5.5 Beyond recommendation accuracy

For a long time, the focus of recommender system studies has primarily been on improving recommendation accuracy. However, there is a growing recognition among researchers and practitioners that other important aspects should also be considered. In this special issue, four papers address these aspects, going beyond accuracy as the sole metric.

Yuan et al. [80] introduce the concept of learning to rank into recommender systems. They embed bi-linear factorization to model algorithm performances, achieving a trade-off between accuracy and inference time in algorithm selection. This work emphasizes the importance of efficiency in recommender systems. Wang et al. [81] investigate the application of Bayesian deep learning to capture uncertainty in model outputs. By incorporating uncertainty estimation, the authors aim to enhance exploration in the recommendation process, promoting diversity and novelty in recommendations. Sun et al. [82] propose a privacy-preserving approach for cold-start recommendations. They develop new representations for cold-start users based on seed items provided by users, enabling personalized recommendations while preserving user privacy. This work addresses the challenge of providing accurate recommendations for users with limited data. Gebremeskel et al. [83] introduce a pull–push metric to quantify the magnitude of pull measures (disapproving over-personalization) or push measures (disapproving under-personalization) in recommender systems. This metric helps to find a balance between personalization and diversity, considering both aspects to improve recommendation quality.

These papers highlight the importance of aspects such as efficiency, exploration, privacy, and diversity in recommender systems. By expanding the focus beyond accuracy, these studies contribute to a more comprehensive understanding and improvement of recommender systems.

6 Concluding remarks

Recommender system has been one of the most practical applications of data science and machine learning. Actually, for a long time, data science has been the foundation of recommender systems. However, to the best of our knowledge, there is no existing work to systematically review recommender systems from the perspective of data science. Therefore, in this paper, we have conducted a systematic overview of recommender systems from data science perspective to comprehensively discuss the relationships between them. We first summarized various types of data from different sources which can be used for recommendations, and then highlighted the commonly utilized data science and machine learning models for building recommender systems. Afterwards, we have shared some of our preliminary thoughts on the next-generation recommender systems. Finally, we have briefly summarized the contents included in this special issue. It is our hope that this paper can provide some new insights and novel perspectives to readers.