1 Introduction

In the current digital era, users are presented with an overwhelming amount of online information and multiple sources of knowledge, which can lead to a phenomenon known as information overload. Recommender systems represent a promising approach to assist users in managing this challenge by suggesting items that match their preferences. Personality is a core human characteristic that remains relatively stable across time and is suitable for modeling user behavior, in contrast to emotions and mood, which tend to be more transient and context-dependent. Incorporating personality into recommender systems can improve the accuracy of recommendations and enhance user satisfaction by tailoring suggestions to their individual characteristics.

Personality traits have gained significant attention in recommender systems due to their potential to mitigate the cold-start problem when we do not have access to a user’s interaction data, enhance recommendation diversity, and capture users’ complex nature [1, 2]. Accordingly, recent studies have demonstrated the effectiveness of personality-aware recommendation [1,2,3], which employs personality traits to make recommendations, in general domains with abundant open data such as films, music, and books.

While personality-aware recommendation systems have demonstrated success in general domains where data is readily available, research on their applicability in specific domains, such as finance, has been limited due to privacy concerns and the requirement for domain expertise to produce precise recommendations [1, 4]. Consequently, it is worthwhile to explore the potential usefulness of personality traits in finance recommendation systems, as they may help address challenges such as information overload in the financial domain.

In addition, it is important to note that domain-specific variables can have a significant impact on decision-making processes, particularly in the domain of finance while previous studies on personality-aware recommendation have mainly focused on general personality traits, such as the Big-Five personality traits [5]. For instance, factors such as risk tolerance play a critical role in investment decision-making, but may not be as relevant in movie or music recommendations. Thus, it is essential to consider domain-specific variables when developing personality-aware recommendation systems for finance to ensure that they accurately capture the unique characteristics of this domain.

Finally, personality-aware recommendation systems have primarily been used to address the cold-start problem in recommendation [1, 3], but their potential to enhance existing recommendation models with transaction data remains underexplored [2]. Therefore, it is also intriguing to investigate whether incorporating general personality traits and domain-specific psychological traits in non-cold start settings can lead to improved performance in recommendation systems.

In summary, we formulated the following research questions.

  1. 1.

    RQ1: Can general personality traits be useful in stock recommendation tasks?

  2. 2.

    RQ2: Do domain-specific psychological traits contribute to the performance of stock recommendations?

  3. 3.

    RQ3: How can we integrate investors’ general personality traits and domain-specific psychological traits with their interaction history to enhance the stock recommendation model?

The rest of this work broadly corresponds to the research questions.

2 Related Work

2.1 Personality-Aware Recommender System

Personality traits have been increasingly utilized in the research of recommendation [1,2,3, 6]. Utilizing personality traits for a recommender system has three advantages. First, using personality traits for the recommender system will mitigate the cold-start problem, especially for new users rather than items. Second, personality traits can be used to increase recommendation diversity [7]. Third, personality traits help model the complex nature of user behaviors. For example, personality traits are known to be significantly correlated with users’ preferences in some areas such as music and movie preference [3, 6].

Various theories in the literature of personality psychology have attempted to describe human personality traits. Among other theories, the Five-factor model, also known as the Big-Five personality traits theory is one of the most commonly used models, where the human personality is characterized by five factors: Extraversion, Openness to experience, Conscientiousness, Agreeableness, and Neuroticism [5].

While five-factor models are widely used to measure the users’ similarity across various domains in personality-aware recommendations, most works only utilize personality traits to represent users’ psychological traits and ignore other psychological effects which might be as important as personality traits [3, 6]. Thus, previous studies have not explored the benefit of incorporating domain-specific psychological traits such as behavioral biases in finance into the personality-aware recommendation model.

2.2 Stock Recommendation

There is a growing demand for stock recommendations as the number of retail investors using online brokers has been rapidly increasing. Accordingly, many studies have tackled stock recommendation tasks. Stock recommendations can be classified into two approaches: non-personalized stock recommendations and personalized stock recommendations. Most works in stock recommendation fall within the scope of non-personalized recommendation, which focuses on identifying optimal strategies for selecting stocks or portfolios that are likely to be more profitable in the future [8]. On the other hand, little research has been done on personalized stock recommendations due to the lack of open data and difficulties in data collection due to privacy concerns [4, 9,10,11,12,13]. Despite the limited literature on the subject, some studies have tackled the problem of personalized stock recommendations. Collaborative filtering has been used for personalized stock recommendations, oftentimes combined with other recommendation approaches such as order book analysis, and multiple criteria decision analysis [4, 9, 10]. For instance, Robin et al. [4] estimate the investor’s risk tolerance from users’ portfolios and recommends stock based on the relevance of the stock’s risk return with the user’s risk tolerance combined with a collaborative filtering method. The method of personalizing stock recommendations based on investors’ risk tolerance has two shortcomings. First, it suffers from the cold-start problem. Second, it is not clear whether one variable, risk tolerance, can capture the complex nature of investors. Therefore, the benefit of personality-aware recommendations which can mitigate the cold-start problem and help model users’ behaviors needs to be investigated for stock recommendations.

2.3 Behavioral Finance

The theory of modern economics is built on the assumption that human beings are rational agents. These agents aim to maximize their wealth and minimize risk, carefully assessing the risk and return of investment choices to obtain a portfolio that matches their risk aversion. However, empirical studies suggest that the real individual investors’ behaviors are different from those of the assumption. The literature in behavioral finance has shown that psychological traits such as behavioral biases, personality, and cognitive ability affect the financial behaviors of individual investors and suggested that these psychological traits and biases are useful in explaining individual investors’ behavior. The relationships among investors’ traits—such as personality, behavioral biases, cognitive ability, and investment goals—have been extensively studied. This examination spans both empirical research in behavioral finance and theoretical studies. While empirical studies highlight the value of domain-specific psychological factors, including behavioral biases, in explaining and predicting investor behavior, their benefits remain unexplored in personality-aware recommendations [14,15,16,17]. Therefore, the effectiveness of domain-specific psychological traits in stock recommendations merits further investigation.

Table 1 Notation and symbols
Full size table

3 Method

Fig. 1
figure 1

Outline of our proposed recommendation model

Full size image

The overview of our proposed model is presented in Fig. 1. The model comprises four steps: (1) grouping individual investors based on specific criteria, which will be discussed later; (2) measuring user similarity; (3) forming user neighborhoods based on the similarity scores; and (4) predicting investors’ preferences and generating stock recommendations. We also provide a notation list in Table 1 for clarity and consistency.

To group individual investors, we employed one of two methods: a clustering analysis based on psychological traits or an equal division based on the number of past transactions. Specifically, we divided all investors I into \(n_\textrm{cluster}\) groups using one of these methods, which will be described in the fourth and fifth experiments.

$$\begin{aligned} \{C_1, C_2, ... C_{n\_\textrm{cluster}}\} = \text {DM}(I) \end{aligned}$$
(1)

where DM represents the method to divide investors such as the clustering algorithm.

Then, we computed the similarity between investors based on their transaction data, general personality traits, and domain-specific psychological traits. First, we measured the similarity based on transaction data (SimT). SimT was computed using the Pearson correlation coefficient as in Eq. (2).

$$\begin{aligned} SimT(u,v) = \dfrac{\sum _{a\in {Y_{u,v}}} (r_{ua}-\overline{r_u})(r_{va}-\overline{r_v})}{\sqrt{\sum _{a\in {Y_{uv}}}(r_{ua}-\overline{r_u})^{2}}\sqrt{\sum _{a\in {Y_{uv}}}(r_{va}-\overline{r_v})^{2}}} \end{aligned}$$
(2)

where u and v are individual investors from set I, \(r_{u,a}\) is the preference of u to a, \(\overline{r_u}\) is the mean of preference of u, and \(Y_{u,v}\) is the set of stocks both u and v purchased.

Likewise, we computed the similarity based on investors’ psychological traits (SimP). SimP was computed using Pearson correlation coefficient as in Eq. (3).

$$\begin{aligned} SimP(u,v) = \dfrac{\sum _{i\in {Psy}} (p^i_{u}-\overline{p_u})(p^i_{v}-\overline{p_v})}{\sqrt{\sum _{i\in {Psy}}(p^i_{u}-\overline{p_u})^{2}}\sqrt{\sum _{i\in {Psy}}(p^i_{v}-\overline{p_v})^{2}}} \end{aligned}$$
(3)

where Psy is the set of psychological traits, \(p^i_u\) is the value of u’s psychological variable i, and \(\overline{p_u}\) is the mean value of the psychological traits vector for investor u. We computed similarity (Sim) based both on SimP and SimT. Then, Sim was computed using a weighted average of SimT and SimP as in Eq. (4). \(\alpha _{u\in {C_i}}\) was dependent on the cluster investor u belongs to, and computed as in Eq. (5).

$$\begin{aligned} \begin{aligned} Sim(u,v) = \alpha _{u\in {C_i}} SimT(u,v)+(1-\alpha _{u\in {C_i}}) SimP(u,v) \\ \end{aligned} \end{aligned}$$
(4)
$$\begin{aligned} \alpha _{u\in {C_i}} ={}_{\alpha \in {[0,1]}} Score_{C_i}(\alpha ) \end{aligned}$$
(5)

where \(\alpha _{u\in {C_i}}\) is the weight of SimT for u, and \(Score_{C_i}\) shows the evaluation metrics such as the F1 score of the performance of recommendation when the weight parameter is \(\alpha \).

Third, the neighbors of target user x were set as in Eq. (6).

$$\begin{aligned} N(x,k) = \{u\in {I}: |\{v\in {I}:Sim(x,v)<Sim(x,u)\}|<k\} \end{aligned}$$
(6)

where x is a target investor, k is the number of neighbors.

Finally, we predicted the preference score of each stock for the target investor by aggregating the preference scores of their neighbors, weighted by the similarity between the target investor and their neighbors. This was done using Eq. (7). Finally, we recommended the top-n stocks with the highest preference scores to the target investor.

$$\begin{aligned} \widehat{r_{xa}} = \overline{r_x} + \frac{\sum _{y\in {N_x}}Sim(x,y)(r_{y,a} - \overline{r_y})}{\sum _{y\in {N_x}}Sim(x,y)} \end{aligned}$$
(7)

where \(\widehat{r_{xa}}\) is the predicted preference score of x to a, \(\overline{r_x}\) is the average preference score of x, and \(N_x\) represents the set of neighbors of x (Fig. 2).

Fig. 2
figure 2

The details of the clustering analysis. Subfigure (a) shows the elbow method on personality traits. Subfigure (b) shows the clustering analysis on personality traits using Kmeans and reduced the dimension into 2d with t-SNE

Full size image

4 Dataset

4.1 Data Acquisition

In our study, we collected data from a Japanese securities company, focusing on individual investors who had made over 50 transactions in a year. We obtained general personality traits and domain-specific psychological traits data along with past transaction history from a total of 969 investors. The data range from July 2020 to September 2022. We collected various domain-specific psychological traits from investors, including behavioral biases, cognitive ability, investment purposes, and general personality traits. Personality traits were assessed using the ten-item personality inventory (TIPI) [18, 19]. To ensure the validity of the questionnaire domain-specific psychological traits, we referred to the Japan Household Panel Survey (JHPS) questionnaire.Footnote 1 We collected behavioral data including risk preference, time discount, overconfidence, hyperbolic discounting, sign effect, and magnitude effect. To measure cognitive ability, we assessed financial literacy through a set of questions regarding financial knowledge and wealth management and administered a cognitive reflection test to evaluate investors’ cognitive ability [20]. Furthermore, we inquired about investors’ investment goals, including retirement, housing, education, medical expenses, vacation, and other objectives.

We processed the transaction data into a user-item matrix given m users and n items. Following the work in [4], we define \(m\times n\) matrix \(U_f\) with components

$$\begin{aligned} (U_f)_{ij} = f(i,j) \end{aligned}$$
(8)

Let \(q_{i,j,t}\) be the portfolio of user i on stock j on the day t which is obtained from transaction data.

$$\begin{aligned} q_{i,j,t} = \left\{ \begin{array}{ll} 1 &{} \text {if user }i\text { holds stock }j\text { in time } t \\ 0 &{} \text {otherwise} \end{array} \right. \end{aligned}$$
(9)

We define implicit feedback collaborative filtering user-item matrix R as \(U_{f_R}\) in Eq. (10).

$$\begin{aligned} f_R(i,j) = \left\{ \begin{array}{ll} 1 &{} \text {if there is }t\in {T} s.t. \, q_{i,j,t} \ne 0 \\ 0 &{} \text {otherwise} \end{array} \right. \end{aligned}$$
(10)

where T is an entire period. Simply speaking, rows of the R matrix represent, for user i, whether they held stock j during any period.

4.2 Investor Behavior Analysis

Figure 3 presents the hierarchical clustering heatmap of investor behavioral traits. The visualization reveals several noteworthy observations. For instance, in Fig. 3, neuroticism exhibits lower correlations with openness and conscientiousness, while cognitive ability demonstrates a higher correlation with financial literacy. Furthermore, it highlights that investors with low risk aversion tend to exhibit high-risk tolerance, and that annual income and investment experiences are strongly correlated. Additionally, Fig. 3 suggests that investors can be grouped into distinct clusters based on their behavioral traits. The visual representations provide valuable insights into the interrelationships among various investor characteristics.

Fig. 3
figure 3

Hierarchical clustering heatmap of investor traits: This heatmap illustrates the relationships among investors based on their key traits. Each column represents a distinct investor, while each row corresponds to a specific trait. The hierarchical clustering algorithm organizes investors with similar characteristics together

Full size image

5 Experiment

5.1 Tasks and Evaluation Metrics

We evaluate its ability to recommend relevant stocks to individual investors. Specifically, we evaluated the model’s performance in two tasks: general stock recommendation and new stock recommendation.

For the general stock recommendation task, we used test data that consisted of each investor’s five most recent transactions to evaluate the model’s top 10 recommended items. In the new stock recommendation task, we removed the transaction data of stocks that each user had in the test data from the train data. The latter task is particularly important in financial stock recommendations, as investors tend to be biased towards familiar stocks, rather than exploring new assets [21]. Therefore, it is crucial for stock recommender systems to recommend new stocks to investors, which can help mitigate familiarity bias and encourage exploration. We used precision@10, recall@10, and F1 scores as evaluation metrics.

5.2 Experiments

We conducted five experiments to address the three research questions.

To address RQ1, we conducted the first experiment to investigate the impact of investors’ personality traits on stock recommendations. We compared the performance of a personality-based model, which uses the Big-Five personality traits, with a transaction-based model and a random model. In the personality-based model, we set \(n_\textrm{cluster}\) to 1 and \(\alpha _u = 0\) for all investors \(u\in {I}\), while in the transaction-based model, we set \(n_\textrm{cluster}\) to 1 and \(\alpha _u = 1\) for all investors \(u\in {I}\).

To address RQ2, we conducted the second experiment to analyze the value of domain-specific psychological traits in the personality-aware recommendation. We conducted an ablation study for the combinations of general personality traits and domain-specific psychological traits.

To address RQ3, we conducted three experiments to compare the performance of existing methods with that of our proposed recommendation models. In the third experiment, we implemented a weighted average model, which is a modification of the approach proposed by Ning et al. [22] that combines the two similarity metrics, SimP and SimT. Specifically, we varied the weight parameter \(\alpha _u\) from 0 to 1 to investigate its impact on performance. In the fourth and fifth experiments, we aimed to validate the effectiveness of the proposed recommendation models. To determine the optimal weight parameter \(\alpha _{u\in {C_i}}\), we split the dataset into train, validation, and test sets. The test and validation sets contained the most recent and next five transaction records for each investor. We performed a grid search on the train and validation sets and used the best parameter to evaluate performance on the test set. In the fourth experiment, we clustered investors based on their psychological traits, hypothesizing that investors with specific psychological traits would be better predicted by SimP. We tuned the weight parameter \(\alpha _{u\in {C_i}}\) for each cluster \(C_i\) and named this model the cluster model. The number of clusters was determined to be \(n_\textrm{clusters}=8\) using the elbow method as shown in Fig. 2. In the fifth experiment, we partitioned investors into equal groups based on their number of past transactions, hypothesizing that investors with more transaction data would be better predicted by SimT, while investors with limited transaction data could be better predicted by SimP. We tuned the weight parameter \(\alpha _{u\in {C_i}}\) for each cluster \(C_i\) and named this model the division model.

5.3 Results

Table 2 presents the results of the first, fourth, and fifth experiments. The evaluation metrics used are Precision@10 and Recall@10, denoted as P@10 and R@10, respectively. GSR and NSR stand for general stock recommendations and new stock recommendations, respectively. The results demonstrate that the general personality-based model significantly outperformed the random model in both settings.

The second experiment’s result is presented in Table 3. The table shows the performance of the ablation study on the combinations of general personality traits and domain-specific psychological traits. The results indicate that adding domain-specific traits such as cognitive ability, behavioral bias, and purposes of investment improved the performance in both general and new stock recommendation tasks. However, adding more variables did not necessarily lead to higher performance, as the model with all variables did not perform better than the models with a subset of variables.

Table 2 Results for the first, fourth, and fifth experiment
Full size table
Table 3 Results for the second experiment
Full size table
Fig. 4
figure 4

The results for the third experiment. a is the results with varying weight of SimT in General Stock Recommendation and b is the result in New Stock Recommendation

Full size image
Fig. 5
figure 5

The comparison of the F1 score among the division model with the transaction-based model and the cluster model. The x-axis shows \(n_\textrm{cluster}\)

Full size image

The results of the third experiment, presented in Fig. 4, indicate that the performance of the weighted average model mostly fell between the performance of the psychology-based model and the transaction-based model. The results of the fourth experiment, as shown in Table 2 suggest that the cluster model outperformed the transaction-based model in the new stock recommendation task with regard to F1 score. Finally, the results of the fifth experiment, presented in Table 2 and Fig. 5, suggest that most of the division models performed better than the transaction-based model in the new stock recommendation task, with the transaction-based model being outperformed only in one setting when \(n_\textrm{cluster}\) equaled 9 in the general stock recommendation task.

6 Discussion

For RQ1, we can conclude that the comparison between the random model and the general personality-based model in Table 2 supports the value of general personality traits in stock recommendation tasks, which is consistent with previous findings in other recommendation domains like music, book, and movie recommendation. The personality-based model outperformed the random model by a significant margin, demonstrating that personality traits can be leveraged for addressing cold start problems in personalized stock recommendations. However, the performance of the personality-based model was inferior to the transaction-based model, indicating that personality traits should be used in conjunction with transaction data for optimal performance in stock recommendation tasks where past transaction data is available for each investor.

For RQ2, we can conclude that incorporating domain-specific psychological traits in addition to general personality traits can improve recommendation performance, as shown in Table 3. However, further investigation is required to identify the most useful combinations of these variables for optimal recommendation performance. This highlights the need for future research to carefully analyze and select the optimal psychological variables for personalized recommendations.

To address RQ3, we carried out three experiments. The third experiment, presented in Fig. 4, revealed that a simple weighted average of SimP and SimT did not yield better performance than the transaction-based model. This outcome suggested that it is necessary to partition investors into groups with distinct characteristics to take full advantage of general personality traits, domain-specific psychological traits, and transaction data. The fourth experiment, detailed in Table 2, demonstrated that our cluster model outperformed the transaction-based model in the new stock recommendation task, but not in the general stock recommendation. The usefulness of personality-aware recommendation for enhancing the diversity of recommendations is well-documented in literature [1, 2, 7]. Consequently, we consider that the diversity in the recommended lists contributed to the improved performance of our new stock recommendation task, which mandated the provision of diverse recommendations to enable investors to explore new stocks. Moreover, we noted that specific clusters with characteristic psychological traits were better predicted using SimP than others, which needs further investigation. Therefore, it is worthwhile to analyze the characteristics of clusters with different performances. The superior performance of the transaction-based model in general stock recommendation can be attributed to the repeat purchase behavior of stocks, which is influenced by familiarity bias. This bias causes investors to repeatedly purchase certain stocks, and as a result, the transaction-based model that learns directly from past transactions performed better in providing general stock recommendations. The results of the fifth experiment show a similar pattern to that observed in the cluster model. Specifically, the division model outperformed the transaction-based model in most cases for the new stock recommendation task, while it only did so in one case for the general stock recommendation is when the number of clusters was 9. In addition to the diversity added to the recommendations, we argue that psychological traits play a significant role in enhancing the performance of the division model for investors with limited transaction data. Figure 5 shows that the cluster model and the division model outperformed the traditional transaction-based model in new stock recommendations. Therefore, this result supports our hypothesis that dividing the investors into groups with characteristics is essential in exploiting general personality traits, domain-specific psychological traits, and transaction data.

To fully leverage the benefits of dividing investors into groups with different characteristics, it is essential to explore how investors in different groups are affected by psychological traits. Future work should investigate the selection of psychological traits and optimal weights at the cluster level to maximize the benefits of personalized recommendations.

7 Conclusion

In this paper, we examine personality-aware recommendations in the financial domain. Specifically, we conduct five experiments in financial stock recommendation tasks with Precision@10, Recall@10, and F1 scores as evaluation metrics. This paper reports three findings. First, we show that general personality traits such as the Big-Five personality traits are useful for domain-specific recommendations such as stock recommendations. Second, we show that utilizing domain-specific psychological traits enhances the performance of the recommendation. Third, we show that our proposed models that divide investors into groups with characteristics outperform the transaction-based model, especially in the new stock recommendation task. While this paper suggests the benefits of incorporating domain-specific psychological traits for recommendations and proposes a model to utilize all the data, careful analysis of optimal selections of weights and psychological variables needs to be studied in future work.