Financial opinion mining is a branch of traditional opinion mining and sentiment analysis which shares the basic notions of traditional approaches and adds its own domain-specific characteristics. In Sect. 1.1, we start with a common definition of general opinion mining after which we briefly overview traditional research directions. In Sect. 1.2, we compare financial opinion mining and general opinion mining, and in Sect. 1.3, we explain the motivation behind capturing financial opinions. We conclude the chapter with an overview of the structure of this book in Sect. 1.4.

1.1 Opinion Mining and Sentiment Analysis

Life is a series of choices, each of which is informed by personal opinions. A person’s opinion may influence the opinions of others, and in turn influence the decisions they make. Thus a better understanding of people’s opinions would make it possible for us to predict behaviors and guess a person’s next steps. For example, every four years, we attempt to predict the outcome of the US presidential election. If we were able to capture every voter’s opinion, we would be able to accurately predict the election results. However, thus ascertaining all opinions before an election is a difficult problem. We hence must use approximate approaches such as surveys to identify trends. After 2000, with the development of the Web and the increase in information shared by users, researchers began to investigate opinion mining methods to collect information that was once unattainable. In a common definition, an opinion is represented as a quintuple [6]:

\((e,a,s,h,t)\,,\)

in which an opinion holder h holds an opinion about entity e at time t with sentiment s under aspect a. Based on this definition, opinion mining is also termed sentiment analysis.

Although these five components, in particular aspect and sentiment, have been discussed for nearly two decades now [5, 8], they remain the focus of much active research [12, 13] due to the wide variety of potential applications. Figure 1.1 shows an example of an opinion, in this case a product review from Amazon. To simply judge the overall sentiment of the review writer, we can treat the five-star rating as a label indicating that the opinion holder possesses a positive sentiment toward the PlayStation 5 Console. Upon further investigation of the review’s contents, we find that the opinion holder possesses a positive sentiment toward the new controller but a negative sentiment toward the bold design and plastic stand. Components e, h, and t, in turn, are relatively easy to extract from the platform metadata, which explains why the focus of most research remains on aspect-based sentiment analysis. The example in Fig. 1.1 shows that the sentiment s can vary depending on which aspect of the product (i.e., entity e) is in question. Potential task settings include the following:

  1. 1.

    Two-class classification (positive/negative)

  2. 2.

    Three-class classification (positive/neutral/negative)

  3. 3.

    Classification with discrete degrees (one-star to five-star)

  4. 4.

    Regression with continuous sentiment scores (0 to 1 or −1 to 1)

Fig. 1.1
figure 1

A product review from Amazon, where the five-star label indicates the opinion holder possesses a positive opinion toward the PlayStation 5 Console

Full size image
Fig. 1.2
figure 2

The “Helpful” button allows readers to praise the review from a helpfulness aspect

Full size image

After extracting the opinion components, the problem becomes how to evaluate the usefulness and helpfulness of the opinion to readers. Figure 1.2 shows a review with little information. As with humans when making decisions, this kind of opinion may not be useful. The figure also shows a common approach for evaluating the opinion for a product: the “Helpful” button allows readers to annotate the review from a helpfulness aspect. These labels are then used for training supervised models [10]. Note however that false information or opinion spam also exists on online platforms. Detecting this kind of opinion is an area of active research in opinion mining [3]. Both content analysis [11] and spam detection [4, 9] are important research topics. However, opinions with little information are not necessarily opinion spam. Although the review in Fig. 1.2 is not useful for readers, the customer did purchase the product (Verified Purchase).

After sorting out the opinions and constructing quintuples from the various sources, we can (1) summarize opinions for a certain entity, (2) submit queries to search for opinions, and (3) compare opinions. The tasks mentioned in this section illustrate the work done over the past two decades on opinion mining and sentiment analysis.

1.2 Financial Opinion Mining

In this book, we define a financial opinion as an opinion related to a financial instrument. A financial opinion also has the five components mentioned in Sect. 1.1. One major difference is that sentiment in a financial opinion is termed market sentiment (bullish/bearish), which is different from sentiment (positive/negative) in general opinion mining research. For example, an investor holding a bullish position may possess negative sentiment because the price is falling. Studies have been done which contrast general sentiment and market sentiment, yielding the following findings:

  • Three-quarters of the negative words in the Harvard Dictionary are not negative words in financial narrative [7].

  • Bullish words in the financial domain are sometimes labeled as neutral words in general sentiment dictionaries [1].

  • Positive sentiment does not always lead to bullish market sentiment [2].

Financial opinion is different from general opinion in that many financial opinions focus on forecasting the future instead of describing an experience. Many general opinions such as product reviews are based on the experience of using certain products. In contrast, financial opinion predicts future phenomena based on whatever facts are available. We define financial opinions in such a way as to yield an overall view from opinion analysis to the interaction between opinions and financial instruments. Table 1.1 shows the components of a financial opinion. In this book, we discuss financial opinion mining using these components.

Table 1.1 Notations used in this book and associated information extracted from Fig. 1.3
Full size table
Fig. 1.3
figure 3

Investor opinion shared on Stocktwits, a social media platform for finance

Full size image

Here, we go through the components using the instance shown in Fig. 1.3, which is a post from Stocktwits, a social media platform for finance. First, e denotes the target financial instrument ($AAPL) that the opinion holder (h, i.e., William) is discussing, and s denotes the market sentiment (bullish) of h on e. Temporal information is crucial for financial documents. A financial opinion can include two kinds of temporal information: the publishing time of the document (\(t^p\), i.e., 1/3/20 11:44 PM) and the validity period of the opinion (\(t^v\)). In this example, the validity period of the price, which ranges from 303 to 307, is “this week”, which means that we should not take this tweet into account after one week. In most opinion mining tasks, opinions have no such validity period. However, due to the dynamic nature of the financial market, financial opinions do have validity periods, even the opinions of professional stock analysts are the same. Most reports from professional analysts have validity periods under one year.

Market information before \(t^p\) (\(M^e_{t^p}\)) may also be mentioned by the investor. Even it is not mentioned, recording market information can help us better understand the financial opinion. For example, if the writer in Fig. 1.3 does not provide the “bullish” tag, we can compare 303–307 with the close price (297.32) to infer that this investor has a bullish market sentiment about e.

In this book, we adopt the notions of argument mining to represent the full picture of financial opinion mining. In Chap. 2, we discuss this in detail. We can consider the market sentiment to be the main claim, which may consist of several claims (C). In each claim (c), there may exist several premises (P) that support the claim from different aspects (a); with each claim has its degree (d) of market sentiment. The quality of the opinion (q) and the influence power of the opinion (\( ip \)) should be evaluated. For example, a professional analyst’s report may be of greater quality than a social media post and thus exert greater influence on the market.

Fig. 1.4
figure 4

Comparison of an order book at two time points. The change in the financial market is caused by changes in investor opinions

Full size image

1.3 Why Study Financial Opinion Mining?

Having described the components of a financial opinion, we now lay out the motivation for capturing financial opinion and thus why we seek to extract these components. We begin with the financial market operation model. Figure 1.4 shows an example of an order book, which lists the interests of buyers and sellers at a given time toward a given financial instrument. The figure lists the prices at which investors are willing to buy or sell, along with the quantity at each price level. Note that the deal price moves from 496.5 to 497.0 in only ten seconds; the quantities at different price levels change as well. Is it that during these ten seconds, the fundamental information of the company has suddenly changed, for instance the earnings per share? If not, what has caused the deal price to move from 496.5 to 497.0 so quickly? Below are some possible scenarios.

  • Because there exists an arbitrage opportunity, the trading algorithm or trader sends the order at $497.

  • A new investor sends a new order at $497.

  • Some investors change their willingness to buy at prices lower than $497 to higher than $497.

Regardless of the rationale, we find that the change in the financial market is caused by changes in investor opinions. In connection to this, note that automatic trading algorithms are constructed based on human beings, and the rationales behind these algorithms can be viewed as opinions. In the example in Fig. 1.4, these ten seconds have resulted in changes not only to the deal price but also to the quantity at each price level. This shows that investor opinions are always changing. Indeed, ideally, given the ability to accurately capture all investor opinions, we would be able to perfectly predict market movements.

Financial opinion mining is one way to analyze the financial market and provide a rationale for market movements. For example, stock prices in energy and travel sectors surged in 2020 because many investors believed that the Pfizer vaccine could resolve the COVID-19 crisis.

Thus, we see that financial opinion mining is more complex than general opinion mining tasks: we seek to understand the decision process of all kinds of investors, regardless of whether they are (1) professional or amateur, (2) rational or irrational, or (3) well-informed or ill-informed. Even if two investors are provided with the same information, they could make different decisions under different rationales. Also, two bullish opinions may have different amounts of confidence or cause different degrees of impact on the market. These phenomena continue to complicate financial opinion mining.

Although we focus on financial opinion mining in this book, similar concepts can be adopted in other domains. We propose application scenarios in other domains in Chap. 7. In sum, solving the issues in financial opinion mining would provide solutions for other opinion-oriented tasks as well.

1.4 Overview of the Book

In Chap. 2, we describe in detail the components of financial opinions and raise several examples for reference. We further use the notions of argument mining to understand the structure of a single financial opinion. We also propose structures between opinions and those between opinions and financial instruments. In Chap. 3, we discuss opinions from various sources, including managers, professionals, social media users, and journalists, and then mention possible research directions for each kind of source. In Chap. 4 we explain how current techniques are used to extract opinion components and link relations between opinions. We also discuss opinion quality and the evaluation of influence. Because numerals contain much useful information in financial narratives, we discuss several numeral-related tasks in Chap. 5. Following this, in Chap. 6 we lay out application scenarios for financial opinion mining in the financial technology (FinTech) industry. We then conclude in Chap. 7, highlighting future directions and unexplored issues and suggesting approaches to adopting the notions proposed in this book to other domains.