Controlling non-stationarity and periodicities in time series generation using conditional invertible neural networks

Generated synthetic time series aim to be both realistic by mirroring the characteristics of real-world time series and useful by including characteristics that are useful for subsequent applications, such as forecasting and missing value imputation. To generate such realistic and useful time series, we require generation methods capable of controlling the non-stationarity and periodicities of the generated time series. However, existing approaches do not consider such explicit control. Therefore, in the present paper, we present a novel approach to control non-stationarity and periodicities with calendar and statistical information when generating time series. We first define the requirements for methods to generate time series with non-stationarity and periodicities, which we show are not fulfilled by existing generation methods. Second, we formally describe the novel approach for controlling non-stationarity and periodicities in generated time series. Thirdly, we introduce an exemplary implementation of this approach using a conditional Invertible Neural Network (cINN). We evaluate this cINN empirically in experiments with real-world data sets and compare it to state-of-the-art time series generation methods. Our experiments show that the evaluated cINN can generate time series with controlled periodicities and non-stationarity, and it also generally outperforms the selected benchmarks.


Introduction
Using real-world time series is associated with issues such as lack of sufficient data, privacy concerns, or anomalies in various applications. Therefore, real-world time series are often replaced or complemented with synthetic time series to address these issues. For this reason, the generation of synthetic time series aims to provide realistic and useful time series. Generated time series are realistic if they have characteristics similar to real-world time series, such as seasonality and trend [13]. For example, an electricity consumption time series typically has reoccurring daily, weekly, and yearly patterns [10]. However, a realistic generated time series is not necessarily useful for all tasks. For example, to analyse the future growth of a startup company, it may be necessary to generate time series with Benedikt Heidrich benedikt.heidrich@kit.edu Extended author information available on the last page of the article. seasonal fluctuations and a trend when given only a small sample of stationary input data. 1 Therefore, to generate a useful time series, one must be able to influence or even design time series' characteristics during generation independently of the available input data.
Generating realistic and useful time series thus requires generation methods to control the generation of time series further. Firstly, generation methods have to control the non-stationarity of the generated time series to incorporate effects that change the value of the time series at different times. Examples of non-stationary time series are daily stock prices, the monthly beer production, or the annual number of strikes [13]. Secondly, these methods must be able to control the periodicities in the generated time series to represent regularly occurring patterns such as the patterns mentioned above in electricity consumption time series.
Despite generally promising results (e.g. [5,19,22,23]), existing time series generation methods do not explicitly control the non-stationarity and periodicities of 1 We further elaborate realistic and useful time series in Appendix A. the generated time series. Instead, these approaches learn a mapping from the latent space, which lacks temporal information, to the realisation space. As a result, all newly sampled data follow the same distribution, resulting in stationary time series. Furthermore, the periodicities in the generated time series are limited to those learned from the training sample.
In the present paper, we thus present a novel approach to control non-stationarity and periodicities with calendar and statistical information when generating time series. For this, we make the following contributions: Firstly, we define the requirements for generation methods to generate time series with non-stationarity and periodicities, which we show is not fulfilled by existing generation methods. Secondly, we formally describe the novel approach for controlling non-stationarity and periodicities in generated time series. Thirdly, we introduce an exemplary implementation of this approach using a conditional Invertible Neural Network (cINN) that preprocesses calendar and statistical information as conditional input with a conditioning network.
To evaluate the proposed cINN, we empirically examine its capabilities to generate time series with controlled nonstationarity and periodicities in experiments with real-world data sets. We also compare the general quality of the time series generated with the cINN to that of state-of-the-art time series generation methods and perform an ablation study to analyse the effects of the conditional information.

Related work
The present paper introduces a novel approach for generating realistic and useful time series by controlling nonstationarity and periodicities. In this section, we thus describe how recent approaches to generate realistic and useful time series consider the temporal structure.
The first group of approaches considers the temporal structure independent from a specific domain: Xu et al. [22], for example, propose COT-GAN. It considers causality in the generation process by utilising the causal optimal transport theory. With this, the output of a point of time depends only on inputs up to that point of time. Yoon et al. [23] also focus on the temporal dynamics of generated time series: By using a supervised loss, the proposed TimeGAN can better capture temporal dynamics. In addition to temporal structures, MuseGAN by Dong et al. [6] considers the interplay of different instruments. For this purpose, they integrate multiple generators that focus on different music characteristics. To generate time series with irregular steps, Ramponi et al. [20] present the T-CGAN that uses timestamps to learn the relationship between data and time.
The second group of approaches additionally uses domain-specific information to consider temporal structures. Esteban et al. [8] introduce the Recurrent Conditional GAN (RCGAN) for generating realistic synthetic medical data. Thanks to conditioning utilising specific labels, RCGAN can create time series that can also be used for training models without imposing privacy concerns. For performing high-quality speech synthesis, Prenger et al. [19] combine the melspectograms presented in WaveNet [17] with an Invertible Neural Network. Furthermore, to generate electrical consumption time series for scheduling and energy management, Lan et al. [16] condition their Wasserstein GAN on temporal and consumer information.
Altogether, existing works consider the temporal structure and domain-specific information when generating time series. However, to the best of our knowledge, no work exists that successfully generates realistic and useful time series by controlling non-stationarity and periodicities.

Problem formulation
To generate realistic and useful time series, generation methods must be able to control non-stationarity and periodicities during generation. This section firstly formalises the related requirements before highlighting why current generation methods cannot fulfil them.

Requirements of non-stationarity and periodicities in time series
This section briefly formalises the requirements of nonstationarity and periodicities in time series with calendar information based on phenomena observed in real-world time series. Requirement 1: non-stationarity Realistic time series can include phenomena such as trends or seasonality. Such characteristics can be represented as components of a time series. In time series analysis, a time series X t is typically decomposed into a seasonal component S t , a trend component T t , and a random (or remainder) component R t , i.e. X t = S t + T t + R t 2 [13]. Since these components can cause a time series to be non-stationary, we detail non-stationary time series. We regard non-stationary time series as realisations of a non-stationary stochastic process {X t }. However, we must consider stationary stochastic processes due to the lack of a formal definition of non-stationary stochastic processes. 2 A multiplicative decomposition X t = S t × T t × R t is also typical [13]. Regardless of considering the additive or multiplicative decomposition, we do not imply the independence of their components in the following.
Controlling non-stationarity and periodicities in time series generation using conditional invertible... 8827 For time series, it is sufficient to focus on the properties of weakly-stationary stochastic processes. According to Hyndman and Athanasopoulos [13], a weakly stationary stochastic process {X t } has the following properties: A time series is non-stationary if at least one of these properties is violated. That is, either the mean μ(t), the variance σ 2 (t), or the autocovariance (k, t) vary over time and thus are time-dependent. Requirement 2: periodicities Realistic time series can also contain the phenomenon of periodicities. In a trendfree time series without a random component Since the random component R t causes the time series to have constant unpredictable fluctuations, this definition is too strict. Therefore, we utilise the reoccurring autocovariance structure (η) between time series points separated by the period η (see [13]) to define a periodicity, i.e.
Furthermore, we expect a noticeably different autocovariance between time series observations separated by κ, whereby κ is not a multiple of the period η, i.e.
In the case of trend-free time series, observations separated by the period η are additionally similar to each other, i.e.
Therefore, a time series includes periodicities if a reoccurring autocovariance structure (η) is present.

Shortcomings of generation methods
This section explains why current generation methods cannot fulfil the requirements of non-stationarity and periodicities. We first explain the principles of generation methods before describing how to apply them to time series generation. We then point out shortcomings of these methods concerning non-stationarity and periodicities in generated time series.
Principles of generation methods Generation methods such as GANs [11], VAEs [15], and INNs [14] focus on describing a probability distribution P X of a random variable X : → X, with being a general probability space and X the realisation space. The underlying assumption is that the observed data x ∈ X are realisations of the random variable X ∼ P X . Since P X is often an intractable distribution, generation methods indirectly model the joint distribution P X,Z of X and a latent random variable Z : → Z in the latent space Z. Given the joint and latent distributions, P X can be expressed as where P X|Z is the likelihood and P Z the prior. If P X|Z and P Z are tractable, this expression allows an exact calculation of P X without knowledge of the intractable distribution P X .
To be able to make use of (4), generation methods learn mappings for the generative process. Given an intractable distribution P X in the realisation space, VAEs and INNs learn an encoding f (X; θ 1 ) from the sample distribution P X to the distribution P Z|X , where P Z|X is the probability distribution in the latent space given X and θ 1 are the trainable parameters. Thereby, regularisation is applied to ensure that P Z|X is a good approximation of P Z and that P Z is a tractable distribution in the latent space. Given P Z , generation methods then learn a second mapping g(Z; θ 2 ) from P Z to P X|Z , where θ 2 are the trainable parameters. 3 Based on the learned probability distribution P X|Z and the known tractable distribution P Z , generation methods finally apply (4) to determine an approximation of the sample distribution P X .
Generating time series Unfortunately, time series are not realisations of a probability distribution P X but of a timedependent stochastic process {X t }. Therefore, the aforementioned underlying assumption of generation methods does not hold. To still apply the principles of generation methods, one must account for the time-dependency of time series. One possibility to consider this time-dependency is to split a realised time series sample x ∈ X into, for example, N sequential segments x = (x 1 , x 2 , . . . , x N ) of arbitrary length, because a stochastic process is defined as a series of random variables. 4 Analogously, one can aggregate multiple generated time series segmentsx i , i ∈ [1, N] to include time-dependency in a generated time series longer than one segment. To generate these time series segmentsx i , i ∈ [1, N], one draws multiple samples z i , i ∈ [1, N] from P Z and uses the mapping g(z i ; θ 2 ) =x i .

Shortcoming 1: non-stationarity in generated time series
In the generative process, the used samples z i , i ∈ [1, N] from the latent space are realisations of the random variable Z with the known distribution P Z . Similarly, the generated time series segmentsx i , i ∈ [1, N] are realisations of the random variable transformation g(Z; θ 2 ). According to the so-called law of the unconscious statistician (LOTUS) [21], this transformed random variable g(Z; θ 2 ) has the expected value Assuming that the mapping g(Z; θ 2 ) only depends on the random variable Z and the fixed learned parameters θ 2 , all generated time series segments also have the same expected value, i.e.
The same argument applies to the variance. Since the variance of a random variable X is defined as 2 , one can again use LOTUS [21] to show that the variance is also the same for all generated time series segments (for details see Appendix B).
The equal variance across all generated time series segments has implications for the autocovariance. Since all generated time series segments are realisations of the same random variable transformation g(Z; θ 2 ), the autocovariance that is defined for two random variables simplifies to the variance, i.e. cov(g(Z; θ 2 ), g(Z; θ 2 )) = σ 2 g(Z;θ 2 ) , which, as previously shown, is the same for all generated time series segments. Altogether, existing generation methods cannot vary the statistical properties of generated time series segments. Therefore, these methods cannot control non-stationarity in generated time series 5 and do not fulfil the previously defined Requirement 1.
Shortcoming 2: periodicities in generated time series As previously shown, the autocovariance structure is the same for all generated time series segments in a generative process. As a result, existing generation methods cannot create reoccurring and different autocovariance structures. Therefore, these methods cannot control periodicities in generated time series and thus do not fulfil the previously defined Requirement 2. 5 Existing methods, such as a recurrent neural network with teacher forcing, can generate non-stationary time series segments of arbitrary length if trained on non-stationary data sets. However, such methods can only reproduce the non-stationarity available in the training data, and this non-stationarity cannot be controlled. Additionally, all of the time series segments generated by such a method contain only the non-stationarities identical to those in the training data. Therefore, it is impossible to control the non-stationarity across multiple time series segments to generate time series with desired non-stationarities.

Controlling non-stationarity and periodicities in generated time series
This section presents a novel approach for controlling non-stationarity and periodicities in generated time series. Firstly, we formally describe our approach to time series generation that fulfils Requirements 1 and 2. To show the practical viability of our approach, we then introduce a conditional Invertible Neural Network (cINN) as an exemplary implementation.

Formal solution
This section formally describes the novel approach for generating time series with controlled non-stationarity and periodicities whilst overcoming the previously presented Shortcomings 1 and 2 of existing generation methods. We firstly explain an assumption to guarantee the existence of time series segments with non-stationarity and periodicities. Afterwards, we detail how our approach uses and combines calendar and statistical information to control non-stationarity and periodicities in generated time series.
Existence guarantee To guarantee the existence of time series segments with controlled non-stationarity and periodicities, we assume the encoding f (X; θ 1 ) to be a bijective mapping, where g(Z; θ 2 ) is the inverse function of f (X; θ 1 ), i.e. f −1 (·; θ) := g(·; θ) and θ = θ 1 = θ 2 . This mapping guarantees that the image of the concatenation Im((f • f −1 )(X)) = X includes the entire realisation space X. Therefore, for all possible samples from the latent space, a corresponding time series segment in the realisation space exists, i.e.
Besides guaranteeing the existence of a corresponding time series segment for all possible samples, this bijective mapping allows us to include additional inputs in the mapping. With these inputs, we can vary the properties of each generated time series segment: Calendar information The first additional input is calendar information such as the hour, day of week, month, or year. This information is implicitly present in a time series as a realisation of a stochastic process {X t } but is currently not considered by generation methods. To include this information, we use the calendar information d as an additional input to our mapping, i.e.
f : Considering calendar information enables us to generate time series segments with varying calendar information, even though these segments are generated from samples Controlling non-stationarity and periodicities in time series generation using conditional invertible... 8829 z i that are realisations of the same random variable Z.
However, solely including calendar information does not allow us to vary the statistical properties of each generated time series segment.

Statistical information
To vary the statistical properties of each generated time series segment, we consider statistical information such as mean and variance as a second additional input. Therefore, we add statistical information s to our mapping, i.e.
f : Combining calendar and statistical information Based on calendar and statistical information as inputs, we are able to generate time series segments with varying statistical properties dependent on the calendar information. For example, as a result, the mean of the transformed random variable f −1 (Z; d, s, θ) is dependent on calendar and statistical information, i.e.
Similarly, the variance and the autocovariance also depend on the calendar and statistical information. Since the calendar and statistical information are included as additional inputs to the mapping, we can effectively control the statistical properties of the generated time series segments for the calendar information. This interplay between calendar and statistical information enables us to include and control nonstationarities and periodicities in the generated time series. Therefore, combining calendar and statistical information as additional inputs allows us to fulfil Requirements 1 and 2 mentioned above.

Exemplary implementation
This section presents the exemplary cINN-based implementation of our novel approach for generating time series with controlled non-stationarity and periodicities. After a brief overview of its architecture, we describe its training and generative process.
Architecture To realise the previously defined bijective mapping that considers calendar and statistical information, we use a conditional Invertible Neural Network (cINN).
To implement the exemplary cINN, we use FrEIA 6 and PyTorch [18]. As shown in Fig. 1, the used cINN comprises 15 subsequent invertible coupling layers, one conditioning network q, and their trainable parameters θ. 7 As coupling layers, we use the conditional affine coupling layer proposed by Ardizzone et al. [2], which extends RealNVP by Dinh et al. [4]. Each coupling layer contains two subnetworks. The architecture of the subnetworks is shown in Table 1. As inputs, each coupling layer takes the output of the previous coupling layer and the conditional information c. This conditional information is the calendar information d and the statistical information s mentioned above, both encoded by a separate conditioning network q.
The architecture of the conditioning network q is also described in Table 2. The input of q are calendar and statistical information. The calendar information d are information for all the time stamps for which a value should be generated. This information includes the hour of the day encoded as a sine function sin(π · hour/23) and cosine function cos(π · hour/23), the month of the year as sine function sin(π · month/11) and cosine function cos(π · month/11), and the weekend as a Boolean. The statistical information s for the training is the mean of the time-series sample, i.e.μ i = E[x i ]. In the generation, the mean is the desired mean of the generated sample.
Training To train the used cINN, we extend the training proposed by Ardizzone et al. [2] with a statistical loss. The training is generally based on a maximum likelihood optimisation using the change of variable formula with the Jacobian matrix ∂f/∂x [2]. To implement this optimisation, we select the standard normal distribution as the latent distribution, choose Gaussian priors, and apply Bayes' theorem. The result is the maximum likelihood loss, i.e.
where J i is the Jacobian corresponding to the i-th sample [2]. In addition to L ml and as an extension of Ardizzone et al. [2], we also minimise the difference between the desired mean and the mean of the generated time series segment, i.e.
wherex j is an entry of the generated time series segment and s μ is the desired mean in the statistical information. Therefore, the overall loss used to train the cINN is where λ is a hyperparameter weighting the influence of the statistical loss. We train the cINN for 200 epochs with this defined loss using the ADAM optimiser. Fig. 1 Our cINN realises a bijective mapping from the realisation space X to the latent space Z. It comprises 15 subsequent coupling layers and one conditioning network q. The conditioning network processes the calendar information d and statistical information s to provide the conditional input c for each coupling layer Generative process The cINN learns a mapping from the realisation space to the latent space in the described training. Since this mapping is bijective, we can use the inverse direction as a generative process: To generate a time series segmentx i , we firstly draw a sample z i from the random variable Z, choose the desired calendar and statistical information d i and s i , and apply the conditioning network to encode c i = q(d i , s i ). With these inputs, we use the trained cINN in the inverse direction and obtain a time series To create a time series longer than one segment, we utilise the calendar information previously included in the mapping to aggregate the generated time series segmentŝ x i . More specifically, we take advantage of the fact that the calendar information d i of adjacent time series segments overlap: For adjacent segments, similar and related calendar information form the input of the conditional network. This input ensures that the sample distribution is conditioned on similar and related calendar information. This way, generated time series segmentsx i with adjacent calendar information d i are related, and we can calculate the median over all entries of a certain time t witĥ where the condition d i j → t ensures that only entries of the time series segmentsx i with the same time t are aggregated.

Experiments
This section empirically evaluates the cINN mentioned above as an exemplary implementation of our proposed approach for generating time series with controlled nonstationarity and periodicities. After introducing the used data sets, we demonstrate how the cINN generates time series with controlled non-stationarity and periodicities. Finally, we compare the cINN to state-of-the-art time series generation benchmarks to assess the general quality of the generated time series. At this point, we also perform an ablation study to determine the influence of conditional information and the statistical loss.

Data sets
In order to comprehensively evaluate our proposed approach and the exemplary implementation, we aim to select data sets from different domains. Since data sets commonly used to evaluate time series generation methods are not publicly available or do not contain calendar information (e.g. [8,22,23]), we select three publicly available time series data sets.
The selected data sets 8 all contain univariate time series with calendar information but differ in their temporal resolution, variance, and periodicities: Energy The first data set has an hourly temporal resolution, a low variance, and contains daily, weekly, and yearly periodicities. It consists of the electricity consumption of one client from the UCI Electricity Load Dataset [7]. Information The second data set has a daily temporal resolution, a medium variance, and contains no periodicities. It comprises the number of daily views of the Wikipedia page "Hitchhiker's Guide to the Galaxy (video game)" from the Web Traffic Time Series Forecasting Dataset. 9 Mobility The third data set also has an hourly temporal resolution, a high variance, and contains daily and yearly periodicities. It contains the hourly records of rented bikes from the UCI Bikesharing Dataset [7,9].

Generated time series with controlled non-stationarity and periodicities
To demonstrate how the cINN creates time series with controlled non-stationarity and periodicities as described 8 Appendix C provides further information on these data sets and their preprocessing. 9 https://www.kaggle.com/c/web-traffic-time-series-forecasting/data in Requirements 1 and 2, we first generate a time series with controlled non-stationarity and second a time series with controlled periodicities for each data set. We first define the calendar and statistical information used in the generation for each time series. Afterwards, we evaluate the generated time series by visually inspecting them and calculating statistics corresponding to the requirements of non-stationarity and periodicities, respectively.

Controlled non-stationarity
To demonstrate controlled nonstationarity, we define calendar and statistical information for the cINN as follows. To determine the calendar information, we choose the years from 2011 to 2013 for the three data sets. As statistical information for the energy and information data sets, we specify a mean with a linear trend starting from 75 and ending at 125 and a yearly sinusoidal periodicity with an amplitude of 15. For the mobility data set, we specify a mean with a linear trend starting from 100 and ending at 250 and a yearly sinusoidal periodicity with an amplitude of 25.
The time series generated based on the selected calendar and statistical information are shown in Fig. 2 for the three data sets. For all data sets, the generated time series accurately reflect the specified mean, including the trend and the sinusoidal shape, while retaining the previously described original characteristics of the respective data set concerning variance and periodicities.
To further examine the generated time series according to the previously defined requirement of non-stationarity, we determine corresponding statistics. On four three-month cut-outs, we compare the mean, the variance, and the autocovariance with a fixed lag of half a year for two generated time series of each data set: one with a constant mean as statistical information and one with a controlled mean as statistical information to control the non-stationarity.
For both generated time series of each data set, Table 3 presents the mean, the variance, and the autocovariance of the four cut-outs. For all data sets, the calculated statistics of the generated time series with constant mean are similar for the considered cut-outs. In contrast, the calculated statistics of the generated time series with controlled non-stationarity are different for the considered cut-outs of each data set and thus are all time dependent. This shows that the generated time series is non-stationary according to Requirement 1.

Controlled periodicities
To demonstrate controlled periodicities, we also define calendar and statistical information for the cINN. To determine the calendar information, we again choose the years from 2011 to 2013. As statistical information for the three data sets, we specify a constant mean of 150 and a yearly sinusoidal periodicity with an amplitude of 50.
For the three data sets, Fig. 3 shows the resulting generated time series and the used mean. For all data sets, the generated time series follows the periodicity defined by the mean and retains the variance and periodicities of the respective data set.
To further analyse the generated time series according to the previously defined requirement of periodicities, we examine the autocovariance structure of the generated time series. More specifically, for the generated time series of each data set, we calculate the autocovariance of the first three months and the rest of the time series. Figure 4 shows the yearly and the daily autocovariance of the generated time series of each data set. Note that no daily autocovariance structure exists for the information Fig. 2 To demonstrate controlled non-stationarity, a time series is generated using defined calendar and statistical information. For the energy, the information, and the mobility data sets, the generated time series is shown in blue and the used controlled mean in orange B. Heidrich et al. 8832 Table 3 Statistics according to Requirement 1, i.e. mean, variance, and autocovariance, for the generated time series with constant mean as statistical information and with a controlled mean as statistical information to control the non-stationarity data set because this data set has a daily resolution and thus no daily periodicities. The autocovariance has a daily and yearly reoccurring structure for all generated time series. According to Requirement 2, the generated time series thus contains periodicities. These periodicities are more regular for the energy and mobility data sets than for the information data set.

Quality of generated time series
To assess the general quality of the generated time series, we compare the cINN to state-of-the-art benchmarks on the three selected data sets and perform an ablation study regarding the influence of the conditional information and the statistical loss. We first introduce the three used evaluation metrics and the six benchmarks before presenting the benchmarking and the ablation study results. For this evaluation, we run each generation method three times. The cINN uses the calendar information and a calculated rolling mean of the respective data set for the generation.

Metrics
To assess the quality of the generated time series segments, we use three metrics. Firstly, we apply the train-synthetictest-real evaluation [8] to obtain a predictive score. The predictive score measures the usefulness of the generated time series. Secondly, we make use of a discriminator to obtain a discriminative score. With the discriminative score, we examine the distinguishability of the generated and the original time series [23]. Thirdly, we measure the training time of the generation methods to assess their computational cost. In the following, we detail all metrics.
Predictive score For the train-on-synthetic-test-on-real evaluation [8], we train a predictive model on the generated Fig. 3 To demonstrate controlled periodicities, a time series is generated using defined calendar and statistical information. For the energy, the information, and the mobility data sets, the generated time series is shown in blue and the used controlled mean in orange Controlling non-stationarity and periodicities in time series generation using conditional invertible... 8833 Fig. 4 The yearly and daily autocovariance structure according to Requirement 2 of the generated time series with controlled periodicities based on the three data sets. The autocovariance is calculated between the first three months and all other three months segments of the time series. Note that the information data has a daily resolution and thus we only calculate the yearly autocovariance time series and test the model on the original time series to obtain a predictive score. As the predictive score, we use the mean absolute error (MAE), the mean absolute scaled error (MASE), and the root mean squared error (RMSE). The architecture of the predictive model is a threelayered fully connected neural network with ten hidden neurons. The model is designed to work with time series segments of 24 hours. Thereby, the first 23 hours are used to forecast the last value. We use a ReLU activation function for the hidden layers and a linear activation function for the output layer.
To implement the predictive model, we use pyWATTS 10 [12] with Keras [3]. We train the implemented predictive model for 100 epochs and apply early stopping during the training process. To obtain more robust results, we train the predictive model five times on the generated time series of each data set.
Discriminative score For the evaluation by the discriminator [23], we merge the generated and the original time series. We label the generated time series with 0 and the original time series with 1. Afterwards, the merged data set is split into a training (70%) and test set (30%). The 10 https://github.com/KIT-IAI/pyWATTS discriminative model is then trained on the training set and the discriminative score is calculated on the test set. We use | Accuracy − 0.5 | as discriminative score, where Accuracy refers to the performance of the discriminative model on the test set.
The architecture of the discriminative model is a threelayered fully connected neural network. The network uses tanh as an activation function for the hidden layers and softmax for the output layer.
To implement the discriminative model, we also use pyWATTS [12] with Keras [3]. We train the implemented discriminative model on the CPU for ten epochs using the ADAM optimiser and the binary cross-entropy loss. We run the training five times on each generated time series to obtain more robust results.
Computational cost To evaluate the computational cost, we measure the training time of the evaluated generation methods in seconds. For this, we measure the time required for training of all generation methods three times and calculate the respective average to obtain robust results. For comparable results, we perform the training on the same hardware. As hardware, we use an off-the-shelf Lenovo Thinkpad T490 laptop with an Intel i7-8565U processor and 16 GB of RAM.

Benchmarks
As benchmarks, we select six generation methods. As state-of-the-art benchmark generation methods, we consider COT-GAN [22], RGAN and RCGAN [8], and TimeGAN [23]. As baseline generation methods, we additionally consider a simple GAN and a simple VAE.
COT-GAN For the implementation of COT-GAN [22], we use the publicly available source code. 11 We only adapt the data loading functionality to apply COT-GAN to our data.

RGAN and RCGAN
For the implementation of RGAN and RCGAN [8], we use the publicly available source code. 12 For our experiments, we only adapt the setting file test.txt and the data utils.py to load our data and ensure the training of RGAN and RCGAN on our data is robust. For RCGAN, we use the same calendar and statistical information as the implemented cINN.
TimeGAN To implement TimeGAN [23], we use the publicly available source code. 13 Compared to the source code, we adapt the data loader of TimeGAN to apply it to our data sets.
GAN To implement the GAN, we use Keras [3] and Tensorflow [1]. Table 4 provides details on the generator and the discriminator of the implemented GAN. The dimension of the random noise input of the implemented generator is 32. We train the GAN for 200 epochs and use the binary cross-entropy as the loss function for the discriminator.
VAE For the implementation of the VAE, we use Keras [3] and Tensorflow [1]. Table 5 details the encoder and the decoder of the implemented VAE. The latent dimension of the VAE is 2. When training the model, we use the Kullback-Leibler distance and the RMSE as the loss functions. We train the VAE for 2000 epochs and apply an early stopping with a patience of 10 epochs.

Benchmarking results
With the cINN and the six selected benchmark methods, we generate time series segments for the three selected data sets. We compare the related predictive score, discriminative score, and training time in the following. Predictive score For all three data sets, the average, minimum, and maximum predictive scores of the cINN and the six benchmark methods are shown in Table 6. We also report a predictive model trained on the original time series as original data in the table, which corresponds to a train-on-real-test-on-real evaluation.
The cINN outperforms all benchmark methods except COT-GAN on the three selected data sets. While the cINN performs better than COT-GAN on the mobility data set, it is on par with COT-GAN on the energy and information data sets. The cINN is also on par with the GAN for the information data set. Moreover, the cINN is almost on par with the predictive model trained on the original time series. Considering the MASE, we also observe that the performance of the generation methods depends on the data set. More specifically, most generation methods obtain the best relative predictive score -measured by the MASE-on the mobility data set. However, most generation methods achieve the worst predictive score on the mobility data set compared to the original data.
Discriminative score The average, minimum, and maximum discriminative scores of the cINN and the six benchmark methods on the three data sets are shown in Table 7. We also observe that the cINN outperforms all benchmark methods except COT-GAN for the discriminative score. However, while the cINN performs better than COT-GAN on the information and mobility data sets, it performs slightly worse on the energy data set. Moreover, we observe that the discriminative score of each generation method is more similar across the different data sets than the predictive score.

Computational cost
The computational cost of the cINN and the six benchmark methods in terms of the average training time is presented in Table 8. Overall, we observe that the simple generation methods, namely the GAN and the VAE, have the lowest training times. However, we observe that the cINN has the lowest training times when only considering state-of-the-art generation methods. Note that the training times of the generation methods on the information data set are shorter due to the smaller length of the related time series segments.

Ablation study
To determine the influence of the conditional information comprising calendar and statistical information as well as the statistical loss defined in (14), we perform an ablation study for the predictive and the discriminative scores. Based on the three data sets, we compare the cINN using calendar and statistical information and the statistical loss (cINN) to a cINN using only the calendar and statistical information (cINN Stats + Cal). Additionally, we compare cINNs using only statistical information (cINN Stats), calendar information (cINN Cal), and no information (INN).  Predictive score The predictive scores of the different cINNs for the three data sets are shown in Table 9. We observe that the considered cINNs generally perform similarly on the three data sets. While the INN barely achieves the best performance on the energy data set and the cINN using only calendar information achieves the best results on the mobility data set, all cINNs are on par on the information data set.
Discriminative score The discriminative scores of the different cINNs for the three data sets are presented in Table 10. Our observation is that the considered cINNs perform similarly on the energy and information data set. However, the cINNs using statistical information perform better on the mobility data set than the cINNs that do not consider this information.

Discussion
This section discusses the previously reported results of the experiments, limitations, and benefits of the cINN as the exemplary implementation of the proposed approach.
In the experiments, we observe that, based on defined conditional information, the cINN can generate time series with controlled non-stationarity and periodicities while retaining the characteristics of the original data set. Furthermore, the cINN outperforms or is on par with the selected benchmark generation methods regarding the predictive and discriminative scores. Also, the cINN requires the lowest training time of the considered state-of-the-art generation methods, probably due to the cINN's lower number of parameters or non-recurrent architecture. Additionally, in the ablation study, we observe that considering calendar and statistical information as conditional information only partly influences the predictive and discriminative score. From these observations, we conclude that the cINN, as an exemplary implementation of the proposed approach, can control the non-stationarity and the periodicities of generated high-quality time series with arbitrary length. Despite these promising results, we note that the performed experiments may have limitations. One limitation could be that the selected data sets only have an hourly or daily resolution and only contain moderate variations. Another limitation might be that we do not evaluate how the described method for aggregating generated time series segments affects the performance of our approach. Furthermore, we only use univariate time series in our evaluation, even though our approach can be extended to multivariate time series.
Concerning the proposed approach, one limitation could also be the required bijective mapping realised by the cINN in the exemplary implementation. While using the bijective mapping guarantees the existence of all generated time series segments, we assume that it is not a necessary requirement for the proposed approach. Therefore, the proposed approach should also be effective with other generative mappings without the guaranteed existence of the generated time series. These mappings could approximate the inverse function, as in VAEs, or be trained by a discriminator, as in GANs. Extending our method to these generative mappings could lead to a more general framework to control non-stationarity and periodicities when generating time series. Another limitation could be that the proposed approach requires calendar information. Although this calendar information is present in a wide range of real-world time series, some time series, such as audio time series, do not contain such information. In addition to calendar information, the proposed approach also considers statistical information to generate time series with desired properties. In real-world applications, these desired properties may only be partially known and thus may need to be approximated. However, given calendar and statistical information, the proposed approach enables controlling non-stationarity and periodicities in generated time series in many realworld applications, especially where real-world time series are non-existent, only partly available, not usable due to privacy concerns, or expensive to measure. In these cases, the proposed approach allows the generation of specific and diverse scenarios with non-stationarity and periodicities in time series. These scenarios could then be used to investigate unusual phenomena and various applications such as forecasting and imputation. Hence, our approach noticeably extends the capabilities of existing time series generation methods and offers new opportunities for purposeful time series generation and analysis.

Conclusion
The present paper presents a novel approach to control nonstationarity and periodicities with calendar and statistical information when generating time series. For this purpose, we first define the requirements for generation methods to generate time series with non-stationarity and periodicities, which we show is not fulfilled by existing generation methods. Secondly, we formally describe the novel approach for controlling non-stationarity and periodicities in generated time series. Thirdly, we introduce an exemplary implementation of this approach using a conditional Invertible Neural Network (cINN) that preprocesses calendar and statistical information as conditional input with a conditioning network.
To evaluate the proposed cINN, we examine its capabilities to generate time series with controlled non-stationarity and periodicities in experiments with real-world data sets. We also compare the general quality of its generated time series to state-of-the-art benchmark generation methods and perform an ablation study to analyse the effects of the conditional information. The presented experiments show that the cINN can generate time series with controlled non-stationarity and periodicities while retaining the characteristics of the original data set. Furthermore, the cINN outperforms or is on par with the selected benchmark generation methods regarding the predictive and discriminative scores. The cINN also requires the lowest training time of the considered state-of-the-art generation methods.
Future work could relax the assumption of a bijective mapping by applying the proposed approach to other generative models to control non-stationarity and periodicities during time series generation. This way, relaxing this assumption could enable a more general framework to control non-stationarity and periodicities during time series generation. Moreover, future work could extend the proposed approach to multivariate time series and time series without calendar information. Similarly, future work could focus on identifying additional controllable properties of time series and incorporating them into the proposed approach. Furthermore, we scale all selected time series using a Standard-Scaler to make the training more stable. Based on the selected and scaled time series, we create samples containing values of one day for the energy and the mobility data set and one week for the information data set as inputs for the cINN and the benchmarks. The resulting samples thus contain 24 values for the energy and the mobility data sets and seven values for the information data set. For all data sets, we took the first year for training.

C.3 Data use rights and data privacy
The associated license allows the reuse of the selected energy and mobility data sets, and the associated donation policy implies consent. Regarding the information data set, the competition guidelines of Kaggle 14 as data host limit the use of the data set to the competition itself. Nevertheless, the data set is commonly used in scientific publications. However, concerning the information data set, we cannot give further information on the consent because Google as a data provider, does not provide any information on this aspect. Regarding privacy, none of the selected data sets contains personally identifiable information because the energy data set is pseudonymised, and the information and mobility data sets are aggregated.

Declarations
Competing interests The authors have no competing interests to declare that are relevant to the content of this article.

Data Availability
The datasets analysed during the current study are available in the UCI repository, https://archive.ics.uci.edu/ ml/datasets/ElectricityLoadDiagrams20112014. The datasets generated during the current study can be replicated with the code made available in our GitHub repository, https://github.com/KITIAI/ ControllableTimeSeriesGeneration.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4.0/.