The Impact of customizable Market Interfaces on Trading Performance
- First Online:
- Cite this article as:
- Teschner, F., Kranz, T.T. & Weinhardt, C. Electron Markets (2015) 25: 325. doi:10.1007/s12525-014-0172-8
- 921 Downloads
One reason for market failure is the inherent complexity that excludes non-sophisticated users. Market complexity can be reduced by adapting the market rules or by simplifying the user interface. Just recently researchers started to address this topic and identified the need to merge market and interface design. Thus far it remains unclear how to design user-centric market interfaces. In a prediction market for economic variables, traders can customize their trading interface according to their informational needs. Surprisingly, we show that on average an increase in information reduces trading performance. An explanation for this effect might lie in cognitive theory. Displaying more information increases the participants’ cognitive load and hence might reduce trading performance. We are able to distinguish between trading behaviour and performance and thereby provide insight into the interplay between information and decision making. Finally, we also track the influence of individual information elements and identify those that improve or decrease trading performance.
KeywordsMarket Design Trading Interfaces Prediction Markets
The Internet has increased the number of complex markets (e.g. Energy or P2P resource sharing, sports betting platforms) dramatically. One reason why non-sophisticated users might have difficulties interacting with such markets is the amount of information they have to cope with. Addressing this problem, researchers have identified the need to merge interface and market design (Seuken et al. 2010a). The main idea is to hide or reduce market complexities while maintaining economic efficiency. One way to accomplish this is to simplify the market interface. To do so, one needs to understand which information elements support and which elements hinder the individual trading process. Moreover, as individuals have different informational needs and vary in experience, it seems fruitful to develop customizable market interfaces.
As a follow-up to Teschner et al. (2012), we study a prediction market called Economic Indicator Exchange (EIX), which forecasts economic indicators such as GDP, inflation, investments, export and unemployment figures in Germany. The basic idea of prediction markets is to trade contracts whose payoffs depend on the outcome of future events. Market participants form expectations about the outcome of an event (e.g. the economic growth in the next quarter). Comparable to trading on financial markets, they buy if they find that prices underestimate the event in question and they sell if they find that prices overestimate the probability of an event. The advantages of this research setting are twofold. Firstly, from an individual perspective, market participants interact in a repeated decision-making environment closely resembling decision making in financial markets. Secondly, as the outcomes of events in prediction markets are finally known, we can ex-post measure the participants’ trading performance. This study differs from Teschner et al. (2012) in two ways. First, we do not examine information usage in a mobile app setting but in a web setting. Second, this study does not only present conceptual ideas, but instead additionally analyses information usage and trade data. In contrast to financial market research we have full data access to the whole trading universe linked to individual trading portfolios and additionally can measure the individual usage of the trading interface. We conduct a field experiment with more than 800 participants and over 75,000 trading decisions, in which participants can individually customize their trading interface. The trading interface consists of seven information elements, which can be opened and closed separately. We show that participants choose different information elements depending on their self-assessed market knowledge. We are able to distinguish between trading performance and behaviour and thereby provide novel insights into the interplay between interface, information and decision making.
Surprisingly, we find that on average an increase in the number of open information elements reduces trading performance. An explanation for this observation may lie in cognitive theory. Complexity increases the participants’ information load and hence can reduce decision accuracy (Malhotra 1982).
The remainder of this paper is structured as follows: the second section presents a review of related work in the (market) user interface domain. Additionally, a short introduction to prediction markets is given. The third section details the field experiment setting and the framing of the participants’ trading process. The subsequent section first presents some descriptive data and then introduces the evaluation methodology. Specifically, we use market measures to separately analyse trading performance and trading behaviour. In section six we link the interface elements to trading outcome and interpret the results. Finally, section seven concludes this paper.
In the following section we will first present related work in the market interface design domain as well as a short section on effect of information displays on decision process. Both sections motivate our explorative approach on studying market interfaces. We then introduce related work in the prediction market area to provide some background of our experimental setting.
Market Interface Design
A fundamental assumption of many market designers is that participants are sophisticated and bounded rational. Hence, participants are able to express their expectations as bids and understand the underlying implications. As a consequence designers have developed mechanisms that are theoretically efficient if participants are perfectly rational (Maskin 2008). Assuming a perfectly rational participant, designing the market interface is just a means of presenting the mechanism. In this line of reasoning oftentimes the market interface is used to present as much information as possible to support the trading process. However, it is well known that individuals are bounded rational (Simon 1997) and might not be able to cognitively handle all available information. This leads to the question, which information supports trading decision and which information distracts them.
Challenged by the rise of complex markets (e.g. Energy or P2P resource sharing, sports betting platforms) in which non-sophisticated users find it hard to interact, Seuken et al. (2010a) proposed the idea of Hidden Market Design. ‘The primary goal […] is to find the right trade-off between hiding or reducing some of the market complexities while maximizing economic efficiency attained in equilibrium.’ Hence, the goal is to lower the entrance barriers (e.g. market complexities) for non-sophisticated users to participate in markets. The simplification can be achieved by either changing the user interface or adapting the market rules. Following that idea, they design a market-based P2P backup application (Seuken et al. 2010b; Seuken et al. 2012). In these papers they address both aspects: the user interface eliciting participants’ preferences and the market rules, standardizing the market interaction.
In a field experiment, Teschner and Weinhardt (2011) evaluate three different interfaces, ranging from an interface that provides the market participants with the maximum amount of information, to a maximally hidden interface, that hides most of the complexities from the users. Testing the idea of Hidden Market Design they find that those market participants that use the hidden market interface are more likely to make profitable trades. The study strongly suggests redesigning standard market interfaces.
Decision Processes in Trading Environments
There are various studies on individual trading behaviour and decision making in financial markets (e.g. De Bondt 1998). They focus on excess trading (e.g. Barber and Odean 2000), the effect of experience or IQ on trading performance (e.g. Grinblatt et al. 2012) as well as the disposition effect (e.g. Odean 1998).
But to our knowledge there exists no empirical work on decision processes in trading environments with focus on the trading interface. Kauffman and L. Diamond (1990) highlight the importance of research on behavioural decision making and information presentation effects. They examine how behavioural effects may become operative in screen-based securities and foreign exchange trading activities, where users can choose among information presentation formats which should support trader decision making. They present a model to identify where and how information, heuristics and biases might affect decision making in the trading environment.
In the domains of decision support systems and online shopping environments the influence of the interface on decision behaviour has been repeatedly demonstrated. Kleinmuntz and Schkade (1993) find that information displays influence decision processes by facilitating some decision strategies while hindering others. Decision makers balance the desire to maximize accuracy against the desire to minimize effort. Investigating the relationship between problem representation and task type in information acquisition, Vessey (1991) develops the cognitive fit theory. This theory proposes that a correspondence between task and information presentation leads to superior task performance for individual users. In several studies, cognitive fit theory has provided an explanation for performance differences among users across different presentation formats such as tables, graphs, and schematic faces (Vessey 1994; Vessey and Galletta 1991). Additionally, they show that increasing interface flexibility instead of an informed choice of display format may be harmful rather than helpful to the problem solver. Similarly Speier and Morris (2003) compare the use of visual and text-based interfaces for low and high complexity tasks. They find that participants perform better in low-complexity environments by using text-based query tools. However, in the high-complexity environments participants perform better with visual support. Turning to the optimal pool of available information in decision support systems, empirical work has shown that users can handle only a certain amount of data. Malhotra (1982) concludes that individuals cannot optimally handle more than 10 information items or attributes simultaneously. Testing decision accuracy Streufert et al. (1967) show that as information load increases, decision making first increases, reaches an optimum (information load ten) and then decreases. Finally, in an interactive home shopping simulation Ariely (2000) tested how the participants’ control over information influences their utilization of this information. He compared four settings; if information control was high/low and the task complexity was high/low. He finds that increased control over information leads to better performance in tasks with low complexity and lower performance in the high complexity setting. He reasoned, that for participants in the low complexity setting, when demand on processing resources is low, more information is beneficial. In complex situations however the information is detrimental to performance due to the additional burden of selecting the right information (Ariely 2000). He concludes, that when cognitive load is high (when the task is novel or difficult) high information control can be harmful.
To summarize previous work, the amount and control of information, as well as the information representation does influence user behaviour. On the one hand, information control improves performance by improving the fit between actions and outcomes. On the other hand, in terms of cost (disadvantages), information control requires the user to invest processing resources in managing the information amount and flow. As a conclusion, information control has both positive and negative effects on performance. The two tasks of processing and managing information are related and co-dependent. Moreover literature has repeatedly shown that too much information leads to cognitive overload resulting in decreased decision performance. As a consequence we rely on information systems to filter, aggregate and present this information in a manner that supports the decision making process. Thus far there are no guidelines on what information is needed to support trading decisions. In order to create such guidelines, one needs to understand which information elements support and which elements hinder the individual trading process. Moreover, as individuals have different informational needs and vary in experience, it seems fruitful to develop customized market interfaces. We analyse decision performance and behaviour in a market experiment setting, namely a prediction market.
Prediction Markets for Economic Outcomes
Prediction markets have a long track record of successful application in a wide area ranging from political to sport events, sometimes outperforming established forecasting methods (Berg et al. 2008; Goel et al. 2010; Luckner et al. 2008). They facilitate and support decision making through aggregating expectations about events (Hahn and Tetlock 2006). The roots of their predictive power are twofold; the market provides the incentives for traders to truthfully disclose their information and an algorithm to weight opinions (Arrow et al. 2008). They facilitate and support decision making through aggregating expectations about events (Berg and Rietz 2003; Hahn and Tetlock 2006; Hanson 1999). Mostly a simple continuous double auction is used as a market mechanism - similar to most conventional stock markets. Also comparable to financial markets, participants buy a stock if they find that prices underestimate the event in question and they sell a stock if they find that prices overestimate the probability of the said event.
In 2002, Goldman Sachs and Deutsche Bank created a market to predict macro-economic outcomes such as ISM Manufacturing, change in Non-Farm Payrolls, Initial Jobless Claims and consumer price index (Gadanecz et al. 2007). By analysing the forecast efficiency, Gurkaynak and Wolfers (2006) find that market-generated forecasts are very similar but more accurate than survey-based forecasts. Similarly, in an attempt to forecast inflation changes in Germany and Bulgaria, Berlemann and Nelson (2005) and Berlemann et al. (2005) set up a series of markets. However, in their field experiments participation is quite low and forecasts results are mixed but promising.
Experiment Setting: An Economic Indicator Exchange
In October 2009 we launched a play money prediction market designed to forecast economic indicators such as GDP, inflation, investments, export and unemployment figures in Germany. The goal was to forecast the indicators over longer time periods in advance and continuously aggregate economic information. The market called Economic Indicator Exchange (EIX) was launched in cooperation with the leading German economic newspaper’Handelsblatt’. The cooperation aimed at reaching a wide and well-informed audience interested in financial markets and economic development. We thus expected no problems understanding the indicators and the concept of trading. The market was publicly available over the Internet and readers were actively invited to join. Besides online documentation there was no further training on how to use the system.
Market and Contract Design
The market design features a continuous double auction without a designated market maker. After registration participants are endowed with 1,000 stocks of each contract and 100,000 play money units. Participants are allowed to submit marketable limit orders with 0.01 play money units increments through the web-based interface. The economic indicators tradable on the market are a mix of leading (forecasting the economy, e.g. investments) and lagging (describing the state of the economy, e.g. unemployment numbers) economic indicators. To facilitate longer forecast horizons every indicator is represented by three independent stocks each representing the next three data releases (e.g., “Inv Jan 2010”, “Inv Feb 2010”, “Inv Mar 2010, in case of investments). As a consequence, the initial forecast periods vary from 1 month for monthly released indicators up to 3 quarters for quarterly released variables. One day before the release date the trading in the concerned stock is stopped. As soon as the trading in one stock stops a new stock of the same indicator (e.g., “Inv Apr 2010”, when “Inv Jan 2010” is released) is introduced into the market. This means that participants receive 1,000 stocks of the respective indicator. As soon as the final value for the recently closed indicator is released, it is paid out. Consequently, all in all participants are able to continuously trade 18 stocks at all times. Due to the legal restrictions on gambling, the EIX prediction market could not be operated with real money and can thus be described as a free to join play money market. As previous research in the field of prediction markets has shown that play money perform as well as real money markets predicting future events (Servan-Schreiber et al. 2004; Wolfers and Zitzewitz 2004), we did not expect negative implications on predictive performance due to that design choice. To increase participants’ motivation and to provide incentives to truly reveal information we hand out prizes worth € 36,000. Prices were shuffled among participants rewarding good performance.
The registration at the EIX-market is free and requires just minimal personal information. However, participants had to report their full name, address, gender, and birthday to qualify for prizes. Thus, 1180 of 1235 registered participants decided to report these dates (96 %). They are predominantly male (93 %) which could bias the results of this study. Their average age is 53 years. Additionally, upon registration we asked participants to self-assess their market knowledge and their knowledge of the German economy. Furthermore, participants indicate if they are working in areas related to exchanges or economic forecasting; with 37.16 % indicating that they do. The user input is highly correlated (ρ = 0.74), meaning that participants indicating high knowledge in the market domain do the same in the economic domain.
Participants are able to customize their trading interface individually throughout the market runtime. By clicking on the small arrows on the upper left of each box, the seven information panels open and close. In the default setting, only the trading mask and the seven headlines are visible. After each submitted order the chosen interface is saved with the order as well as in the user profile. On user’s return, the system opens the previously used interface elements by default. The advantage is twofold; firstly, users have a convenient option to customize their trading experience, secondly we can assess which self-selected information pieces have influenced the participants’ decision processes. Additionally, we did not have to form treatment groups with different interfaces and assign users to certain groups, since such a setting would possibly create an unfair experience.
In a second step, we present how the self-chosen interface influences participants’ trading behaviour. As all traders have the same portfolio at the beginning, the size of a trade is a proxy for the trader’s confidence perception (Harris 2002). Assuming that participants using more information (a high number of open information elements) are more confident about how to trade, it seems reasonable that the resulting order size is on average higher (H2). Another individual market behaviour is how participants submit their orders. We distinguish between market orders (which trade instantaneously against a standing limit order) and limit orders. Therefore, the trader submitting a market order pays the effective spread in order to execute directly, knowing that the order will be executed. A less confident trader wants to keep the effective spread and posts limit orders. We hypothesize that traders using more information are more confident about their decision and submit market orders (H2).
Finally and most importantly, we analyse how the self-chosen interface influences the participants’ trading performance (H3). Starting from a ‘base load’ of information, we try to identify the influence of additional information, i.e. the number of open interface elements as well as the interface type (I1-I7). The intuitive assumption is: the more information, the better the decision accuracy. Hence, a higher number of open information elements leads to a better trading performance. As presented, previous work suggests that decision accuracy might suffer if the information load is too high or the control of information distracts from the problem’s solution (Malhotra 1982). Therefore, alternatively too much information reduces decision accuracy and thus trading performance.
In combination, these three steps provide a first recognition of a market interface’s impact on trader behaviour. Moreover they provide insight, how a market interface affects individual trading behaviour and subsequently trading performance.
Data and Methodology
The following section first presents some descriptive market statistics and then details the methods to systematically analyse the effect of different trading interfaces on trading behaviour.
The following data includes the timespan from 2009/10/30 until 2011/10/31. In total 1,235 participants registered at the EIX market, of those 824 submitted at least one order. We discard all stocks with less than 50 transactions. Altogether participants submitted 79,334 orders resulting in 34,028 executed transactions. Previous work showed that the market-generated forecasts performed well in comparison to the’Bloomberg’-survey forecasts, the industry standard (Teschner et al. 2011a; Teschner et al. 2011b). For every order we record the open interfaces (I1-I7, see Fig. 1). An interface variable is 1 when the element (I1-I7 Fig. 1) is open when submitting an order otherwise it is 0. In our field experiment we asked participants to self-assess their market knowledge on a scale from 1 (very good) to 6 (very poor). According to their rating we cluster the participants P by median into two groups: the good (MKp = 1) and the not good (MKp = 0) market knowledge groups.
Measuring Trading Behaviour and Performance
We relate the quantity of a specific order o to the number of open interfaces Ii,o. Please note, that we cannot assess, whether that information was actually used in the particular trading decision. However, the results won’t suffer as this research focuses primary on the presentation of information. By introducing one market dummy (M1 – M5) for each group of related indicators (e.g. one market dummy for export indicators, one for inflation indicators, etc.), we control for the different historic variances, similar groups of indicators exhibited in the past (e.g. exports are much more volatile than inflation). Similarly to control for the self-assessed market knowledge we add a market knowledge dummy (MK). The control variables are included in all presented regressions.
In order to calculate the influence of each individual interface on an order’s type, we use a Logit-regression (see Eq. 4).
In this section we will show that participants choose different information elements to support their trading. Moreover, individual behaviour differs depending on the interface elements used. Controlling for different trading behaviour, we find that market participants using a lower number of open interfaces are more likely to submit profitable orders.
Customizing the Trading Interface
Market Knowledge and Interface Choice
- Result 1:
Participants make use of the functionality to customize their market interfaces.
- Result 2:
Experienced participants use more information to inform their trading decision.
The Influence of the Number of Open Interfaces
Number of Interfaces
The Effect of Customizing Interfaces
I1 (order book)
I2 (price chart)
I3 (account information)
I4 (market information)
I6 (historic values)
I7 (previous orders)
- Result 3:
A high number of open interfaces lead to orders with decreased confidence (lower volume and more limit orders)
We suggested two dissenting outcomes regarding the interface influence on decision accuracy (H3). One might intuitively suspect that a higher number of information panels correlates with better trading decisions. Turning to Table 2, Model C reveals that the chance of submitting a profitable order is lower with an increasing number of information panels. Interestingly looking at the interface elements supporting successful trading, it turns out that the order book, market information and the previous order element have a positive effect (Table 3, Model F). In contrast the price chart does not actually help in the decision-making process but seems to distract participants. A possible explanation might be, that participants over-rate the past price process and under-rate their current information about the state of the economy. This is in line with Mussweiler and Schneller (2003) who show that charts depicting past stock prices influence investing decisions. While their work showed the effect in a closed laboratory setting our results generalizes the effect to a multifaceted field experiment. Moreover these results coincide with Fama (1991), that it is not possible to consistently increase trading performance by using information open to the public. Even though our market setting is a play money prediction market his prediction holds. Participants using certain elements (price chart) anchor their trading decision on irrelevant information and loose on average. Nevertheless, this result might not generalize to more sophisticated markets or market participants that are used to carefully attribute importance to a price chart.
- Result 4:
A high number of open interfaces lead to orders with decreased trading performance.
- Result 5:
If participants see the price chart their trading performance suffers.
Conclusion and Implications
Various allocation problems call for market based solutions. However, market complexities impose high entry barriers for non-sophisticated users. One reason is that in markets preferences are usually communicated through bids and offers, which require participants to adapt to a different mental model. Recently, researchers proposed the idea of hidden market design, which merges the fields of market design with user interface design in order to make complex markets accessible to a broader audience. As more trading decisions are facilitated through (web-based)’trading support systems’ one of the most urging questions is how to design such interfaces. Moreover, it is important to design such interfaces without reducing market efficiency and individual trading performance. In our field experiment participants trade in a complex prediction market, which closely resembles trading in financial markets. As the outcome of events in prediction markets is finally known, we can ex post measure the participants’ trading performance. In our market setting, participants can individually customize their interface. We show that participants choose different information elements depending on their self-assessed market knowledge. Apparently, market participants try to fit the interface to their individual internal problem representation. Yet the individual motivation of the interface choice remains unclear.
We show that presenting more information does not improve decision making but rather decreases trading performance. From the results presented, one might follow that information accessible on the interface does not help forecasting economic variables and hence the participants’ decision-making. Another interpretation, which is in line with previous work, can be found in cognitive theory. As Malhotra (1982) shows, “too much” information leads to information overload, which reduces decision accuracy and may also reduce decision confidence. Thus a high number of interface elements (a lot of displayed information) increases complexity and distracts from good decision making. As participants are able to customize their interface they seem unaware of the negative influence of the interface on their decisions. As a consequence, market designers should not only limit the amount of presented information but also make a validated guess about which information is useful. One clear market design implication is that the chart depicting past prices does distract participants. Even though it is one of the most common features of market interfaces (retail broker interfaces or trading desks) we see that participants actively choosing to see the chart lose on average. On the other hand, participants viewing the chart use an increased order size. Market operators who gain by the higher volume should hence clearly make the chart a prominent, default feature of the trading experience. One element that is “good” for both market operators and market participants is the order book. Participants using the order book gain above average and they submit more volume as well.
This work provides insight into the interplay between interface, information and trading behaviour. However, the research setting in this work has known drawbacks. First of all, the participants’ self-assessed market knowledge is a subjective measure. Secondly, participants were allowed to customize their interface by themselves. As a next step, one should randomize participants in pre-defined interface groups to validate our findings. Another promising approach is to rerun the presented field study in as a slightly modified laboratory experiment, where participants cannot modify the interfaces themself. By stepwise iterating different combinations of information elements and with the aid of eye-tracking systems, participants’ actual information usage, and thus a proxy for participants’ information-processing effort, could be derived. Moreover, future research might address the non-linear relationship between information load and decision making pointed out by Streufert et al. (1967). However, we hope this work provides a good starting point for practitioners and researchers designing markets and their interfaces.