Variance of entropy for testing time-varying regimes with an application to meme stocks

Shannon entropy is the most common metric to measure the degree of randomness of time series in many fields, ranging from physics and finance to medicine and biology. Real-world systems may be in general non stationary, with an entropy value that is not constant in time. The goal of this paper is to propose a hypothesis testing procedure to test the null hypothesis of constant Shannon entropy for time series, against the alternative of a significant variation of the entropy between two subsequent periods. To this end, we find an unbiased approximation of the variance of the Shannon entropy's estimator, up to the order O(n^(-4)) with n the sample size. In order to characterize the variance of the estimator, we first obtain the explicit formulas of the central moments for both the binomial and the multinomial distributions, which describe the distribution of the Shannon entropy. Second, we find the optimal length of the rolling window used for estimating the time-varying Shannon entropy by optimizing a novel self-consistent criterion based on the counting of significant variations of entropy within a time window. We corroborate our findings by using the novel methodology to test for time-varying regimes of entropy for stock price dynamics, in particular considering the case of meme stocks in 2020 and 2021. We empirically show the existence of periods of market inefficiency for meme stocks. In particular, sharp increases of prices and trading volumes correspond to statistically significant drops of Shannon entropy.


Introduction
The Shannon entropy is widely used as a measure of randomness in many fields, such as finance, physics, medicine, and biology [1,2,3,4,5,6]. One of the main applications of entropy estimation in finance is to measure the randomness of price returns. When the price incorporates all relevant information, the market is called efficient and the price dynamics is a martingale [7,8]. As such, the Shannon entropy takes the maximum allowed value and can then be used as a measure of market efficiency. For the review of methods for testing martingale property and nonlinear dynamics in financial time series, we refer to the article [9]. The drivers of market dynamics are however the result of the complex process of matching the supply and the demand of a large number of investors. It is easy to imagine that the market does not necessarily reflect all relevant information at certain times, because of the complex nature of the price formation mechanism. Moreover, feedback loops, irrational agents, market panic and speculation, or coordination of retail investors driven by non-economic reasons (like for the GameStop case) are just a few examples of mechanisms which can potentially create booms and busts [10,11], thus driving the price far away from its fundamental value. In such cases, the price dynamics may display some level of predictability and, as such, the market is inefficient. This can be captured by some low value of the Shannon entropy.
In a number of works, the hypothesis of market efficiency is relaxed, by accounting for the possibility of periods of inefficiencies. In order to capture such an effect, Shannon entropy is considered as time-varying and it is computed by using a window rolling over the period of interest, see, e.g., [12,13,14,15]. It is important to notice that many patterns of price dynamics may jeopardize the estimate of entropy. For example, long memory of volatility can be the result of a regime switching behavior [16,17,18] and, as a consequence, the estimate of any other dynamic pattern can be affected. For this reason, it is important to filter out any known pattern of market predictability, such as heteroscedasticity or seasonality, before using the estimate of entropy as measure of market efficiency. Interestingly, even after such filtering, the price dynamics often continues to display some predictability that is captured by low values of entropy estimates, signaling the not complete efficiency of the market [19,20]. A crucial question is whether a drop in the entropy estimate at some period is statistically significant or it is only a fluctuation consistent with the null hypothesis of market efficiency.
The goal of this paper is to propose a rigorous methodology to identify significant changes in the value of the Shannon entropy. We define a rigorous procedure to test the hypothesis that the entropy value associated with two different sequences (defined on the same finite alphabet) is statistically the same. To this end, we need to solve two problems. The first problem is to find the variance of the Shannon entropy's estimator obtained by the Empirical Frequencies method [21]. The second problem is to find the optimal length of a rolling window used to estimate the time-varying entropy of a time series.
The variance of the Shannon entropy is the key quantity to use in order to determine if two entropy estimates associated with two different sequences are statistically equal. In fact, it is possible to define a z-score given the variance of the Shannon entropy, thus determining the corresponding p-value of statistically equal estimates.
G. P. Basharin [22] obtained the first order approximation of the variance of the entropy estimatorĤ calculated using the Empirical Frequencies method.
where pj are the set of the probabilities of possible events and n is the length of the sequence of events. The same result was later obtained in [23]. However, Eq. 1 holds in the asymptotic regime, that is when the length of the sequence becomes arbitrarily large, i.e. n → ∞. Here, we aim to estimate the time-varying entropy for finite samples, i.e. using a finite length n of the sequence. Moreover, Eq. 1 is not a consistent estimator of the variance in the case of equal probabilities. When all probabilities pj are equal, D1(Ĥ) = 0. Thus, a more accurate approximation for the variance is needed. 1 Here, we obtain a formula for the variance of the estimator of the Shannon entropy as a sum of central moments of binomial and multinomial distributions. The central moments are calculated by a new recursive approach. This helps us to find the approximation of the variance with an accuracy of order O(n −4 ). 2 Interestingly, Ricci et al. [25] found that the naive estimator of the variance approximation D1(Ĥ) in Eq. 1 has a bias term of order O(n −2 ). On the contrary, we show that our proposed estimation for the variance of entropy is unbiased. Finally, by leveraging on the explicit formulation of the variance of the estimator of entropy, we are able to define a statistical test for entropy variation. In [26], the author suggests a statistical test for independence of symbolic dynamics. The test statistics is related to the permutation entropy. In our research, we are not restricted to the case when the benchmark value of the entropy is its maximum. A rejection of the null hypothesis signals statistically significant variation between any two possible values of entropy.
The second problem relies on finding the optimal length, i.e. the bandwidth, of the time window when testing for entropy variation between two subsequent time series. This is a problem of bias-variance tradeoff: reducing the length of the window allows to obtain a timely estimate of entropy, i.e. small bias, at the expense of increasing the variance of the estimator, and vice versa.
Here, we introduce a novel self-consistent criterion to select (in-sample) the optimal bandwidth ω. Given a sample of size T , we first define a counting function of the percentage of entropy variations for non-overlapping time series of length ω within such sample. Under the assumption of a finite number of true entropy variations, the percentage of estimated variations becomes negligible when ω is close to the minimum (ω = 1), while it is zero by definition when ω attains its maximum (ω = T ). Then, we show by simulations that such counting function has one maximum corresponding to the optimal bandwidth.
We use the novel methodology to find significant changes in entropy on simulated and real data. We investigate changes in the efficiency of the New York Stock Exchange with a special focus on meme stocks. The lower the entropy of the price return time series, the higher the price predictability. In particular, we examine the GameStop case, whose price increased significantly in January 2021.

The GameStop case
In early 2021, the eyes of everyone were on the New York Stock Exchange because of one of the most terrific surges in prices in its history. Starting from the very end of December 2020, GameStop and multiple other stocks belonging to the same sector experienced a dramatic increase in their share price and volume, driven by the long trades of a huge number of individual investors, who fomented such a coordinated action on the Reddit social platform. When the prices were hitting their all-time highs, the attention of everyone was focused on such shares, which became known as "meme stocks". Then, as the end of January 2021 approached, several retail brokerdealers temporarily limited certain operations in some of these stocks and options. As a consequence, the trend started to revert, even if such a turmoil period has had a permanent impact on GameStop and the other meme stocks. The stocks have maintained a higher price level with respect to the period before and have been characterized by higher volatility. A precise description of the GameStop case can be found in the report by the staff of the U.S. Securities and Exchange Commission [27]. There are two crucial aspects that have led to the sharp increase in meme stock prices. First, the aggregation of the orders sent to broker-dealers via online trading platforms, such as Robinhood, in the hands of a few off-exchange market makers permitted to negotiate good agreements, which have then resulted in incentives or no fee at all for the end customers. Such an absence of trading frictions driven by FinTech innovations has had a positive feedback effect on retail trading. The second and more important aspect is the coordination in long trades by a huge number of individual investors, which has been possible because of online social networks like Reddit. The coordination has shaped up to be an act of rebellion against short-selling professional investors who had allegedly targeted the meme stocks. The two effects combined have created a first sharp increase in the prices, which has been further amplified by the coverage of the short positions by professional investors. The increasing interest in buying meme stocks and the resulting feedback dynamics have then led the intermediaries to extraordinary operations. Without entering into the discussion about the manipulative nature of such trading behavior coordinated via social networks and the interpretation of the GameStop rally as a revenge against Wall Street for the 2007-2008 crisis, it is evident that the case of meme stocks has represented a period when the market was out of the ordinary. Here, the focus is on market (in)efficiency during such a period. In particular, we find that the clear predictability of the price pattern results in a signal of inefficiency in terms of the Shannon entropy and statistically significant variations of the Shannon entropy have an interpretation as early-warnings.

Structure of the paper
Section 2 presents the test of equality of entropies and a method for choosing the length of an interval for comparison. In particular, we give an estimation for the variance in Section 2.2. We propose a method for choosing the length of a sliding window in Section 2.4. We test the hypothesis about equal entropies on simulated data in Section 3. The hypothesis is tested for the price returns of meme stocks in Section 4. The Appendix Sections A, B, and C contain the proofs of propositions and theorems. Section 5 concludes the paper.

Methodology and Dataset
First, we discuss the Empirical Frequencies method for estimating the Shannon entropy. Then, we find the approximation of the variance of this estimation. We use the variance to make a hypothesis testing for equality of entropy values.

Shannon entropy
The Shannon entropy is defined as the average amount of information that a process transmits with each symbol [28].
Definition 1 Let X = {X1, X2, . . . } be a stationary random process with a finite alphabet A and a measure p. A k-th order entropy of X is where x k 1 are all sequences of length k with the convention 0 log 0 = 0. A process entropy of X is We use the Empirical Frequencies method [21] to estimate k-th order entropy from the sequence x n 1 . For each a k 1 ∈ A k empirical frequencies are defined as The estimation of the k-th order entropy iŝ The process entropy can be estimated asĤ k k .

Variance of the entropy's estimator
Let's assume that there are M events which can appear with probabilities p0, p1, pM−1, M −1 j=0 pj = 1. We assume that all pj are positive, because zero probabilities do not affect the entropy value H = − M −1 j=0 pj ln pj. If events appear independently n times in total, the frequencies of events f0, f1, fM−1 follow a multinomial distribution. Each frequency is distributed as Binomial B(pj, n). Therefore, the estimation of pj, pj = f j n , is distributed as B(pj, n)/n. The aim of this section is to find the variance of a random variableĤ = − M −1 j=0p j lnpj , where ln is the natural logarithm. The variablê H is a k-th order entropy from Eq. 2 with the base e of the logarithm assuming that blocks a k 1 ∈ A k are generated independently. If all blocks have a non-zero probability, M = |A| k , where |A| is the size of the alphabet A.
where H = − M −1 j=0 pj ln pj. The proof of the Theorem is in Appendix B. All useful propositions for proving the Theorem are in Appendix A. In particular, we present a recursive formula for the central moments of the multinomial distribution of (p1,p2) in Proposition 1. Also, we find the expression for the expectation ofp 2 ln 2 (p) that appears inĤ 2 using the Taylor expansion in Proposition 4. Using all needed central moments of binomial and multinomial distributions, we calculate V ar Theorem 2 (Estimation of variance) In addition to the assumption of Theorem 1, let's assume that all events appear at least once. Then, V ar(Ĥ) = E(V ar)+O(n −4 ), wherê The proof of Theorem 2 is in Appendix C. In the Theorem, we assume that all different events appear in the sequence, thus M is known as the number of all different events. We discuss this assumption in the Remark below.

Remark 1 Number of appearing different events,M , is defined asM
The error term M −1 j=0 (1 − pj) n attains its minimum when all pj = 1 M . It grows as a probability approaches 0; that is, when there is an event that can happen with a tiny but not zero probability, so that we may not observe it in a finite sequence of events. Such an event does not greatly affect the entropy value, since p ln p → 0 as p → 0. We need to fix the minimum length of a sequence to assume that all events appear in the sequence. To this aim, we introduce the following rule. The length of the sequence is taken such that the minimum error is less than 0.01. That is, one event with a non-zero probability does not appear in 1 case out of 100.

Hypothesis testing
Using the estimation of entropy, we aim to conclude if two entropy values significantly differ. Let's take two sequences with entropies H1 and H2. LetĤ1 andĤ2 be estimations of entropies with variances V ar1 and V ar2, respectively.
H0 : H1 = H2 Ha : H1 = H2 Let assume that H1 = H2. Then, a z-score is distributed with a zero mean and the variance equal to 1. We reject H0 if |z| is larger than a quantile corresponding to 99% of confidence. The quantile is defined empirically in Section 3.1.1.

Determining bandwidth
We assume that a time series into consideration can be not stationary, thus we need to choose a bandwidth w, that is the length of a rolling window where entropy is estimated. We aim to detect significant changes in entropy, thus we can not take w too large so that the window covers several intervals with different entropy values. On the other hand, if the process at some period is stationary, we aim to take w as large as possible to improve the accuracy of the entropy estimation 3 . In terms of the biasvariance trade-off, if the rolling window covers a period where the process is stationary (with constant entropy), the error from a bias is eliminated. However, taking w too small implies a large variance and thus it may be impossible to distinguish a change in entropy from estimation errors. Thus, our goal is to find an optimal parameter w.
We select the following approach to determine the optimal bandwidth. We test for all adjacent non-overlapping intervals. Then, we choose a bandwidth that allows us to detect the maximum z-score. More precisely, we aim to maximize an objective function f (w) below. 4 where q99 is a 99% quantile for the empirical distribution of z. The intuition for the objective function is that small values of w give small values of the z-score since the variance, V ar(Ĥ), is O(w −1 ). Large values of w may not be able to catch the largest change in entropy value. Finally, if we can not reject the hypothesis about constant entropy, we aim to choose w as large as possible, thus we maximize − 1 w . Since we are interested in detecting changes in entropy, max(|z|) is always non-negative and − 1 w is always negative. We give more intuition in Section 3.3 by plotting Fig. 3.
To make at least one test, we set the upper bound for the bandwidth as nmax = ⌊ n 2 ⌋, where n is the total length of the sequence. We may define the optimal bandwidth using a training set.

Dataset
We consider three meme stocks that became popular in late 2020 and 2021. We take a one-minute frequency from 9:00 to 15:59. We investigate stocks of the companies GameStop (GME), Bed Bath & Beyond (BBBY), and AMC Entertainment Holdings (AMC) 5 . In addition, we consider three well-known IT companies, expecting more persistent entropy for their stocks. These companies are Apple (AAPL), Salesforce (CRM), and Microsoft (MSFT). Using return time series, we define a 4-symbols alphabet as follows.
where Q1, Q2, Q3 are quartiles of the empirical distribution of returns rt. Here, rt are price returns time series after filtering out data regularities: intraday volatility pattern, heteroskedasticity, price staleness, and microstructure noise [20]. Price returns are defined as Rt = ln Pt P t−1 , where Pt is a price at time t. Data regularities are empirical properties of price returns [29] that make the price returns time series more predictable but do not imply any profitable trading strategy. For instance, intraday volatility pattern refers to the fact that the volatility of intraday returns has periodic behavior. It is higher near the opening and the closing of the market [30], because traders tend to trade less in the middle of a day.

Simulation study
For the further analysis of simulated and real data, we fix A = {0, 1, 2, 3}.

Determining quantile of entropy's distribution
In this section, we find the quantiles of the distribution of the z-score from Eq. 5. Intuitively, the larger M (the number of different events), the more the entropy estimator (as the sum of random variables) tends to be normally distributed. G. P. Basharin showed that the distribution of the entropy estimator is asymptotically normal [22]. However, if all probabilities are almost equal, the distribution of scaled entropy estimation converges to χ 2 -distribution [31]. That is, quantiles of normal distribution can be used if the entropy is not near the maximum. In case when entropy is close to the possible maximum, the normal distribution for testing equality of two entropies is inappropriate. 6 To show this, we set large values for the length of a sequence and the length of blocks.
We perform N = 2 × 10 4 Monte Carlo simulations. We simulate two sequences with length n = 2 × 10 5 of 4 symbols with equal probabilities. We set k = 7 so that M = 4 7 = 16384. When probabilities are equal, the expression for the variance (Eq. 3) becomes We use this value as given instead of estimating the variance from the sequence. Then, for two sequences we find the value of ∆Ĥ =Ĥ2 −Ĥ1. Since ∆Ĥ is the difference of two independent variables, its variance is 2V ar(Ĥ). An empirical 99% (95%) quantile of the empirical distribution of |z| is 3.30722 (2.54542). Thus, even if the length of a sequence and the amount of blocks are quite large, the tails of the empirical distribution are thicker than for the normal distribution. We use the empirical quantiles for further analysis. If an absolute value of z-score is larger than the quantile, the difference between entropy values is statistically significant.

Testing short independent sequences
For the further analysis, we fix k = 4. Therefore, the maximum of the k-th order entropy is k ln |A| = 4 ln 4, M = 256. Now, we aim to test the quantiles found in the previous section for shorter sequences. Here, we keep N = 2 × 10 4 and set n = 2 × 10 3 . Using quantiles from the previous section, the false positive rate is 1.025% (4.86%) for the level of significance α = 0.01 (α = 0.05). 7 We consider these results as acceptable for keeping the found values of quantiles for the rest of the paper.

Testing a fully random sequence
Here, we simulate one sequence with four equiprobable symbols and length N = 2×10 7 and divide it by overlapping intervals with the length n = 2 × 10 3 . We consider all differences with the gap equal to n, so that there are no common blocks between two intervals. The false positive rate according to the 99% (95%) empirical quantile is 0.9976% (4.4608%), that is quite close to the significance level. Thus, q99 from Eq. 6 is taken as 3.30722.

Power and size of the test
We consider a process with 4 symbols. The probability of repeating a symbol is τ . All other probabilities are equal, that is, the probability of having, for example, 0 after 1 is 1−τ 3 . Thus, the value of the 4th order entropy is given by The further τ from 1 4 , the lower the entropy. τ = 1 4 corresponds to equiprobable symbols and the maximum of entropy. First, we construct two sequences with H( 1 4 ) and length n = 10000. We estimate the variance of entropy's estimator using Equation 4 and perform hypothesis testing using Equation 5. The false positive rate (size) is 0.86% obtained by generating two sequences 2 × 10 4 times. Now, we simulate one 7 Empirical quantiles from this experiment are equal to 3.31684 and 2.52907, respectively.
sequence with H( 1 4 ) ≈ 5.54518 and the other with a different entropy value to measure the power of the test. The results are in Table 1.

Non-stationary process
Now, we consider a process with the 4-symbols alphabet constructed as follows. The length of a sequence is N = 30000. We set k = 4, M = 256. For this sequence, nmin = ⌈ln 0.01 M /ln M −1 M ⌉ = 2594 and nmax = ⌊ N−k+1 2 ⌋ = 14998. We divide the sequence into three parts as shown in Fig. 1. The first and the last parts are simulated as the processes with the maximum entropy and four equiprobable symbols. The first part has the length of 10000. The part with the length l in the middle has the entropy H(τ ) given by Eq. 8. The variance of the entropy's estimator becomes time-varying, thus we estimate it using Equation 4. We apply the method for determining an optimal bandwidth, wopt, from Section 2.4. First, we fix τ = 0.5 and take l from nmin to 10000 with the increment of 100. For each l, we calculate the optimal bandwidth and plot it in Fig. 2. The figure shows that the optimal bandwidth corresponds to the length of the interval in the middle. Fig. 3 shows the plot of the objective function (Eq. 6) for 4 random iterations with different values of l. All plots have one global maximum. The larger l, the larger the argument of the maxima corresponding to wopt. When l = 10000 and w is close to nmax, the percent of statistically significant changes in entropy is less than 1%, thus f (w) becomes negative. If the process generating the sequence is stationary (τ = 0.25), the range of the objective function may be negative. The objective function of one realization of the stationary process is given in Fig. 4. Since the function is monotonically increasing, the maximum is attained at nmax.   Finally, we fix l = 10000 and take τ from 0.25 to 0.5 with the increment of 0.01. For each τ , we calculate the optimal bandwidth and plot it in Fig. 5. τ = 0.25 corresponds to the case of a stationary process and thus w is close to the possible maximum. This corresponds to the intuition of choosing the bandwidth: if the process is stationary, the larger the bandwidth, the more accurate the entropy estimate. The deviation of wopt when τ = 0.25 is high because of the first type error in hypothesis testing. Even if the process is stationary, H0 may be rejected more than in 1% of cases that leads to a positive value of f (ω). Such a large deviation can be reduced using Bonferroni orŠidák corrections [33] if the distribution of a z-score is known. We note that when the entropy in the middle slightly differs from the maximum (τ = 0.26), the method may not detect the interval with different entropy and set wopt close to the maximum. The larger τ , the closer wopt to the length of the interval in the middle and the less the standard deviation of wopt.

Market efficiency
A market in which prices always fully reflect available information is called efficient [8]. In a weak form of the efficient market hypothesis, the information set is historical prices. Thus, if a market is efficient in the weak form, a future price can not be predicted better than its current value. Full uncertainty about future values of prices implies the maximum degree of randomness of the price returns time series. Thus, the entropy of return time series should attain its maximum if the market is efficient.
If a market is efficient and the entropy of the price returns is always at the maximum, the hypothesis H0 should be failed to reject. The rejection of H0 for two nonoverlapping intervals implies that the entropy is time-varying. Low values of entropy relatively to entropy estimated in the past indicate predictability of price returns.

Meme stocks
Here, we consider stocks GME, BBBY, and AMC. We set the year 2019 as a training set. The period from 01.01.2020 to 20.07.2021 is a testing set. First, we filter out the data regularities. We define an intraday volatility pattern, fit an autoregressive moving average (ARMA) model, and find an optimal bandwidth, wopt, using the training set. Volatility and the degree of price staleness are defined minute by minute. The ARMA model is needed to filter out microstructure noise; volatility estimation is needed to filter out heteroskedasticity. Quartiles Q1, Q2, Q3 used for discretization in Eq. 7 are defined using the return time series of the training set after filtering out the data regularities.
We calculate the Shannon entropy of the discretized sequence st of the testing set using the rolling window with length wopt. Comparing entropies of two adjacent intervals, we have three possible outputs: entropy decreases, entropy increases, entropy does not change significantly. We present results for each stock in Table 2.
We plot the entropy estimation, price, and trading volumes in Figure 6 for the stock GME. Plots of BBBY and AMC stocks are in Appendix D. We mark on the plots where entropy has significant changes. Red dots correspond to statistically significant decrease in entropy. Green dots stands for statistically significant increase in entropy. The dots at the same time are plotted in figures for the prices and trading volumes. We report below when entropy started to decrease first time or after an increase. We note that the entropy had been already defined as statistically significant low for the stock AMC when record volumes occurred and the price rose sharply for the first time in January 2021. Volumes of the GME stock volume increase decrease Figure 6: Entropy, price, volume of the GME stock. Dots correspond to statis-tically significant changes in entropy. 14 For GME, there are two series of decreases. They start from the entropies calculated at the periods [17.07.20 15:

IT stocks
We take the stocks of Apple, Salesforce, and Microsoft for the comparison. Calculated entropies are plotted in Fig. 7; the optimal bandwidths are recorded in Table 2. The entropy of the price returns of these stocks exhibits time-varying behavior. For each stock, statistically significant changes in entropy are detected. However, the entropy does not fall as much as in the case of meme stocks. The price returns time series for the stock CRM in 2019 is defined as stationary since f (wopt) is negative.  ) is calculated on the training set. The number of tests is the amount of adjacent time intervals with lengths w opt in the testing set. The increases and decreases are statistically significant changes in entropy between two adjacent intervals.

Quarterly training sets
The reason for the increase in predictability, when entropy falls, may be a change in the structure of data regularities. For instance, due to the growing popularity of meme stocks, the intraday volatility pattern could change from the year 2019 to the year 2021. If the behavior of traders has changed, the intraday volatility pattern from the training set does not filter out the data regularity in the testing set. In order to filter the data regularities more carefully, we update an estimation of intraday volatility pattern and a fitted ARMA model using quarterly intervals starting from the first quarter of 2019. This quarter is also used to filter out the data regularities in-sample. An optimal bandwidth is defined using the year 2019. The results are shown in Table  3 and Fig. 8.  ) is calculated on the training set. The number of tests is the amount of adjacent time intervals with lengths w opt in the testing set. The increases and decreases are statistically significant changes in entropy between two adjacent intervals.
In all six cases, entropy still exhibits time-varying behavior. Compared to the previous setting, where the training set is one year, the number of statistically significant changes in entropy for the stocks GME and AAPL increases. Also, in all cases, the maximum value of the objective function, indicating how much the entropies of two adjacent intervals differ, increases.
There are two sequences of statistically significant decreases in entropy value for the stock GME. They start from the entropies calculated at the periods [03.08.  We can conclude that a possible change in the structure of data regularities over time is not the cause of the changes in the values of entropy. As in the previous section, the entropies of IT companies do not drop as low as in the case of meme stocks. Therefore, there is more predictability in the prices of meme stocks than in the prices of IT companies.

Conclusions and discussion
We introduced a novel procedure of hypothesis testing to determine if two discrete sequences with the same alphabet have different entropy values. We use it to detect changes in the entropy value of a sequence. Since entropy can change over time, we cannot choose the length of the interval for measuring entropy as long as possible. For this reason, we have introduced three contributions. First, we have found the approximation of entropy's variance that is used in hypothesis testing. Our formula is more precise regarding the length of the interval compared to the formulas proposed by Basharin [22] and Harris [24]. Second, we have introduced an unbiased estimation of the approximation of the variance so that we can evaluate the variance from a sequence. Finally, we have proposed the method for finding an optimal length of the interval. We have shown that this method is suitable for determining how to split a sample into two time series that display statistically different values of Shannon entropy.
We applied the novel method to the return time series of meme stocks. We have found time intervals when entropy is statistically low with respect to other periods, thus signaling a high predictability of the price pattern. In particular, we have focused on three meme stocks, then comparing them with three more standard IT companies, namely GME, BBBY, AMC, AAPL, CRM, and MSFT. We have found that entropy changes over time for all six stocks in a period between January 2020 and July 2021. We can also say that the entropy was time-varying in the year 2019 for all stocks except CRM.
Low entropy values found for each stock are indicative of the predictability of return time series. We filtered data regularities in two ways, and therefore, we believe that the entropy values obtained are related to the measure of market inefficiency. The deviation of the U.S. stock market from efficiency is also detected in other articles, e.g. [34,35,12].
From the entropy plots, we notice that entropy for meme stocks falls lower than that for IT stocks. This indicates the existence of the periods of high predictability for meme stocks compared to IT stocks. For the GME stock, a low level of entropy was identified prior to the price spike. Moreover, a fall in entropy corresponds more to the growth in trading volumes than to the increase in the stock price according to the both series of low entropy values. For the AMC stock, entropy was low when the price went up in January 2021. Entropy fell again before the price attained its maximum in June 2021. For the BBBY stock, the period from December 2020 to the end of January 2021 is also characterized by low entropy, although it was discovered late enough to be considered a price increase warning. The low entropy values found for the BBBY stock are more likely to correspond first to the price drop in February 2021 and then to another spike in June 2021.
In the case of GameStop and AMC Entertainment Holdings, growth in prices and trading volumes is characterized by a drop in the entropy value. Interestingly, such a drop occurs before the boom observed in January 2021, already in the late December 2020. That is, some regularity pattern in the price dynamics, that appeared before all the news spread the market, leads to a statistically significant signal of market inefficiency. Given the observed timing, such a signal can also be interpreted as an early-warning of turmoil period for the stock.
Today, financial markets are inherently high-dimensional due to the plethora of instruments composing the portfolios of investors. At the same time, they are highly challenging to monitor, displaying more and more complex cycles, booms and bursts of prices, economic bubbles and so on, all of them representing severe risk factors for the portfolios. In this high-dimensional and complex context, the existence of online early-warnings of market inefficiency is key. In fact, such signals allow to anticipate the periods of turmoil, thus covering or, at least, mitigating the portfolio risk associated with such events, with potential stabilizing effects for the whole market.

Appendices A List of propositions
For the proof of Theorem 1 we need five propositions below.
Proposition 1 Central moments of multinomial distribution f M (p1, p2, n) divided by n (/n) can be defined recursively using the formulas below. Proof: where µ M m,k is the (m,k)-central moment of the multinomial distribution and q = 1 − p1 − p2. We can show that Solving the system for µ M m+1,k , we get that Taking into account that µ M m,k = n m+k µ m,k , we obtain the result and by symmetry Proposition 2 Central moments of the binomial distribution B(p, n)/n can be defined recursively using the formulas below.
This is a special case of the previous proposition where p2 = k = 0. It is known as the Renovsky formula [36].
Proposition 3 wherep ∼ B(p, n)/n and µm is its central m-moment.
The result of Proposition 3 was obtained in [22]. It is derived by using the Taylor expansion around p.