Introduction

The probability density function (PDF) of the state estimation error (uncertainty) of space objects is utilised in a variety of applications including collision probability estimation, manoeuvre detection, track association, multi-target tracking and sensor scheduling. Each of these areas has its own accuracy and realism requirements of the uncertainty. Traditional orbit determination methods produce a state fit to observations based on root-mean-square minimisation [28]. Although the transformation from observation to state space is non-linear, the state estimation error PDF corresponding to the estimated state is typically given as a normal distribution (Gaussian) in the state parameters and expressed in the form of the covariance matrix. The point in time for which the state estimate is given is simply referred to as ‘epoch’ or ‘at epoch’ in the following. It seems to be generally accepted that the PDF of the state estimation error may be assumed to remain close to normal for a time after epoch, and many methods have been developed based on this assumption [5, 9, 18, 23, 24]. While some methods require the PDF to be normal on the entire \(\pm \infty \) range [23], others operate on a portion of the PDF surrounding the covariance mean [24]. Over time, some features attributed to normality of the uncertainty PDF dissolve, however, so that methods operating on the traditional covariance matrix may become unreliable. As an example, symmetry and the distinct bell-shape are two well-known features of the normal distribution. Changing the weight equally in the two tails of the distribution will lead to a non-normal distribution which retains the feature of symmetry. There is no common agreement on how the departure from one feature should be quantified in relation to others or indeed how it should be determined. This merits the existence of different methods and also helps to better understand where differences in Type I and Type II errors in each may come from. An extensive comparison of methods applied to samples drawn from well known functions is presented in [21]. Some publications that apply these tests to the case of state error estimates in astrodynamics are [10, 27, 29]. The methods for Multivariate (Multi-dimensional) Normality (MVN) detection used in the current paper assess the time frame until a bounded volume surrounding the mean becomes non-normal. The tails of the PDF are not evaluated, so that the results should not be used for methods requiring the PDF to be normal in its entirety.

The Space Environment Research Centre (SERC Limited) is in the process of setting up a catalogue of space objects based on passive and active optical observations. The aim of the work presented here is to augment orbit determination (OD) results with information on the time frame until departure from normality is detected in the uncertainty PDF. Methods for collision likelihood estimation as well as others mentioned before, take the state and uncertainty data supplied in the catalogue as starting points for extrapolation to other points in time. The time frame until MVN departure is of particular importance when selecting appropriate methods that require extrapolation outside the time frame of available observations.

In the current paper, two fundamentally different approaches are used to determine departure from normality in state uncertainty. Commonality between the methods exists in that both operate on a set of states given at a common point in time. State propagation is limited to two-body motion and assumes no process noise. Proper modelling of process noise in itself can easily become quite a complex topic, if sources are to be properly taken into account. Although this topic is important, the current analysis instead concentrates on the assessment of the fundamental capabilities and limitations in the absence of these further complexities. As neither method requires knowledge on how the states were generated, at least formally, they can be applied regardless of the propagation method. In Section “Multivariate Normality Test Baseline” the first of the two methods, the Henze-Zirkler test [11], is discussed. This test belongs to the family of consistent approaches and operates on a random particle sample. Examples of other tests that have been used in astrodynamics include the Kolmogorov-Smirnov, Royston and Doornik and Hansen tests [10, 29]. The test is applied to a set of constructed test cases and the time evolution is discussed. The second test introduced in Section “Efficient MVN Testing” relies solely on the unscented transform. The offset of the state mean from the covariance mean in whitened space is used to test for normality. Within the section, the two methods are compared based on the previously assessed test cases. The paper concludes by applying both approaches to a select number of orbit determination results from SERC’s own catalogue.

Multivariate Normality Test Baseline

The random sample approach was chosen to create a baseline for the assessment of the departure from normality in state uncertainty predictions. The approach involves the approximation of the continuous uncertainty PDF by a random set of discrete states. Most states will be generated in the vicinity of the highest probability density. Each state could be an exact representation of the true state and may be propagated through time independent of all other generated states. In this paper, the collection of states that are generated from the initial covariance in this fashion are called “sample”. The number of states within the sample describe the “sample size” and is denoted as np. An MVN test takes into account all states within a sample and evaluates the likelihood that these could have been drawn from a multivariate normal distribution. Although relying on few assumptions, these approaches suffer from two major drawbacks: 1) Different MVN tests exist and each gives a different result for a given a sample, in particular when samples are ‘near’ normally distributed. 2) The outcome depends to some quantifiable degree on the particular random sample drawn from the PDF. The latter issue can be overcome by repeating the test by drawing new samples from the same PDF and looking at all outcomes. This is the basis of Monte-Carlo assessment and the number of samples used are denoted nmc. Monte-Carlo analysis can be computationally expensive. The former issue requires a good understanding of available statistical tests to chose one or several appropriate methods.

The section starts out with an outline of MVN tests presented in literature from which a suitable test is selected (Section “Random Sample MVN Test”). The second (Section “Henze-Zirkler Test for Normality”) contains several sub-topics: First, important aspects of the chosen test statistic are outlined. The evolution of this test statistic for an example case is then detailed to give some insight into its behaviour in describing the transition into non-normality (Section “Example Evolution of Henze-Zirkler MVN Test Values”). This is followed by an assessment of the impact of the sample size as well as the definition of an appropriate one for the given task (Section “Sample Size”). The central limit theorem (Section “Monte-Carlo Runs & Central Limit Theorem”) is used to assess how the variability of sample-by-sample results at each epoch translates into a time window. In the last Section “Constructed Test Cases” a set of constructed test scenarios is introduced and the transition into non-normality is assessed using the outlined method.

Random Sample MVN TestFootnote 1

Results from large scale comparisons of different MVN tests which operate on random samples are published, for instance, in [21] and [6]. A grouping of MVN tests is outlined in [21]:

  • Graphical and correlational approaches

  • Skewness and kurtosis approaches

  • Goodness-of-fit approaches, and

  • Consistent approaches.

The well known Kolmogorov-Smirnov, Cramér-von Mises, and Anderson-Darling tests all fall into the “Goodness-of-fit approach” category. Among the selected methods was an extension of the Anderson-Darling test published by [26] which is given the name Romeu–Ozturk test. The Henze–Zirkler [11] test falls into the category of “Consistent approaches”. Consistent is used to indicate that it has been mathematically shown that the test will – at least in theory – consistently reject all non-MVN distributions. An example of tests of the category “Skewness and kurtosis approaches” are approaches based on the work of [20]. Among the criteria which the authors used to assess the tests was the rate of Type IFootnote 2 and Type IIFootnote 3 errors as well as feasibility for implementation and desirable mathematical properties. The two major conclusions of this analysis were: a) No single test for MVN delivered perfect results and it was recommended to employ multiple methods for testing of MVN where possible; and, b) If only one test is used, the Henze–Zirkler was recommended. The Romeu–Ozturk was rejected early on in the study due to high Type I type error rates which in some cases exceeded 10 %. These results are supported by a later study by [6]. In these publications, particle sample sizes were varied between 25 and 250 and sample space dimensions of up to 10 were considered. Based on these results, the Henze–Zirkler test was selected as baseline method.

Henze-Zirkler Test for Normality

The following equation calculates the test metric:

$$ HZ = \left[\frac{1}{n} \sum\limits_{i=1}^{n} \sum\limits_{j=1}^{n} e^{-\frac{\beta^{2}}{2}D_{ij}}\right] - \left[2\left( 1 + \beta^{2}\right)^{-\frac{d}{2}} \sum\limits_{i=1}^{n} e^{-\frac{\beta^{2}}{2\left( 1+\beta^{2}\right)}D_{i}}\right] + \left[n\left( 1+2\beta^{2}\right)^{-\frac{d}{2}}\right] . $$
(1)

Herein Di is the squared Mahalanobis distance of the ith observation to the centroid and Dij gives the squared Mahalanobis distance between ith and jth observations. The authors give a relation for β which uses the particle sample size nFootnote 4 and sample space dimensions d:

$$ \beta =\frac{1}{\sqrt{2}} \left( \frac{n\left( 2d + 1\right)}{4}\right)^{\frac{1}{d+4}} . $$
(2)

This value is used to test the null-hypothesis H0 which is that the sample is indeed MVN distributed. In general, the test statistic HZ is small when the particles are MVN distributed and increases with deviation from MVN. To ease assessment, the HZ value is normalised so that any value HZ∗ above 1 rejects the null-hypothesis and the distribution is considered to be non-normal. For a complete description of its implementation, the reader is referred to [8].

Example Evolution of Henze-Zirkler MVN Test ValuesFootnote 5

Using the fast Mikkola/Halley-4 single step solution for two-body motion as detailed in [22], a common structure is observed in all simulations performed to date. This is not a general proof of the structure being true for all cases. Such an assessment would be extremely valuable but has not yet been attempted. An example of the observed structure is given in Fig. 1. So far, the evolution could always be split roughly into four stages: i) fully normal, ii) transition iii) fully non-normal and iv) deep non-normal.

Fig. 1
figure 1

Example evolution of normalised Henze-Zirkler test for multivariate normality (HZ∗). Two random samples were drawn from position & velocity covariance. Test is performed on position uncertainty only

Full size image

Phase I: Fully Normal

During the initial phase following the state epoch, the uncertainty remains normal throughout an entire revolution. During this stage, Type I error causes some samples to still flag non-normality.

Phase II: Transition

The uncertainty volume then becomes non-normal during parts of one revolution. During this phase, a common structure materialises between the time-evolution of all initial samples.

Phase III: Fully Non-Normal

In the final phase, all samples exhibit the same time evolution and the uncertainty volume is non-normal throughout the entire revolution.

Phase IV: Deep Non-Normal

The simple structure observed in the fully non-normal phase gives way to some more complex behaviour.

An interesting observation is a half-period periodicity in all simulations. This has some likeness to debris clouds from on-orbit fragmentations. In position space and in early stages after a breakup, the debris accumulates into a ‘pinch-point’ at the point on orbit where the fragmentation occurred, and into a ‘pinch-line’ at the opposite side of the orbit. The debris cloud is smallest at these two points and spreads out in between. Similar to the observed behaviour in the position uncertainty, two somewhat similar states are thus obtained at half-period intervals. It must be noted however that unlike particles generated from an uncertainty at epoch, fragmentation debris has no – or very little – initial position offset.

Sample Size

Figure 2 visualises the importance of proper sample size selection. The left hand plot has been produced by predicting an object’s position using only uncertainty in two dimensions of the position in ECI coordinates. The blue lines are made up of individual particles created at the σ-lines 1 to 10 at the initial epoch where normality is given by definition. Although the technical significance at the outskirts of the chosen scale is questionable, the same relation can be observed on any other scale as well and depends on the propagation time and the magnitude of initial uncertainties. The red lines are made up of the same particles after propagation to t1 = t0 + Δt. To ease comparison of the two samples, the particle positions are transformed into Mahalanobis space. Within this space, particle distances from the mean are given in multiples of standard deviations. In the plot, particles close to the state mean describe concentric circles at t0 as well as t1. Particles at the outskirts however exhibit marked deformation in two-dimensional position space at t1. In two-dimensional space, this may be taken as one feature required to allow classifying a PDF to be “normal”. Other features such as the area contained in the tails are not well captured in this representation. The amount of ‘bending’ which is observed to increase with distance from the mean indicates that normality may remain intact for longer periods of time close to the mean state compared to regions that are further removed.

Fig. 2
figure 2

Left: Particles taken at the 1-σ to 10-σ lines before propagation (t0) and after propagation (t1). [8]. Right: Example of effect using different sample sizes from a 6-d position & velocity covariance. Plot shows mean value of nmc = 5 000 Monte-Carlo runs for initial detection of non-normality

Full size image

The right hand figure aims at demonstrating by example that the observed qualitative effect is measurable in simulations and must be accounted for appropriately. It displays the evolution of the Henze-Zirkler MVN test statistic over time for samples of different sizes drawn from a 6-d initial position & velocity covariance matrix. Only the 3-d position sub-space is tested for normality. Data points and trend fits are shown for the highest on orbit values only. The figure thus shows the epoch at which normality evolution enters the Transition Regime detailed in Section “Example Evolution of Henze-Zirkler MVN Test Values”. Each data point represents the mean value of nmc = 5000 independently propagated samples where each sample consists of the np states. It can be observed that an increase in sample size, the MVN departure advances in time. However, the trend is not uniform with sample size steps, indicating that here already further effects may be present which are not

figure a

To define a suitable sample size, a relation is first required between the sample size np and the number of particles outside a defined confidence interval p:

$$ n_{p} \left( \nu > \nu_{0}\right) = n_{p} \left( 1 - p(\nu_{0}, k=1)\right) . $$
(3)

The p-value is the area under the integrated χ2(ν, k) function with degrees of freedom k that is outside the defined limit variance ν0. For k = 1, the χ2 function describes the distribution of residuals of the one-dimensional normal distribution. For the current purpose, the 3-σ bound (99.7 % confidence level) is selected as the limit beyond which no significant particles are required. In addition, a significant number of particles should still be present between 98.7% ≤ p ≤ 99.7% confidence levels. The significant number of particles is set to 30. The number ‘30’ is borrowed from the central limit theorem (see Section “Monte-Carlo Runs & Central Limit Theorem”) without further investigation and may be revised in future studies. From Table 1 it can be seen that any sample size between 3 000 and 11 000 satisfies the above restrictions. With the understanding visualised in Fig. 2, larger sample sizes lead to earlier detection of non-MVN. For the current purpose it is decided to err on the conservative side and a sample size of np = 10000 is chosen. This is the same number as was selected in previous studies.

Table 1 Sample sizes and the number of particles in the outskirts of the distribution
Full size table

Monte-Carlo Runs & Central Limit Theorem

It is in the nature of the approach that for any given epoch, not all samples lead to the same MVN-test result. Often, a decision is based on their mean value. A different set of samples would create a slightly different mean. The PDF of many such means follows a normal distribution when each mean used more than 25 - 30 samples. If fewer samples were averaged over, the PDF is expressed by the “Student’s T” distribution [12, pp. 200].

In the following, the decision will be based on nmc = 50 so that the normal distribution applies. In this case, the standard deviation of the PDF of means σCLT can be estimated from the standard deviation of the distribution of a single set of 50 results s(nmc = 50). This is the basis of the Central Limit Theorem (CLT) whose general relation is:

$$ \sigma_{CLT}\left( n_{mc}\right) = \frac{s\left( n_{mc}\right)}{\sqrt{n_{mc}}} . $$
(4)

We want to know two things at this point. Firstly: What is the true state of the population at a given point in time? As we are dealing with the random particle approach, this information is only useful in combination with its variability. Secondly: How does this variability from epoch to epoch translate into a time window for the transition to non-normality. Such a window is only a further measure of understanding the variability of the results that are inherent to the particle method. They have no relation to the underlying population that we assume represents reality under the simplified physical properties of two-body motion and no process noise. In the following, we derive a continuous function by fitting to the discrete data from the Henze-Zirkler test that allows us to estimate such a time window.

MVN Departure Onset and Conclusion Mean Epochs

Once MVN starts breaking down, it initially occurs only briefly during a revolution either at one or several locations. Section “Example Evolution of Henze-Zirkler MVN Test Values” has details on this. To identify the state of the uncertainty at any epoch, one must therefore evaluate the entire revolution. To decrease computational burden, departure onset and conclusion epochs are iterated towards. This is possible due to the purely analytic description of orbital motion. For the transition onset, the epoch is recorded which marks the start of the first revolution during which the MVN test has failed at least once. The departure conclusion epoch marks the start of the first revolution during which the MVN test fails at all epochs. Evaluating for MVN at set epochs allows sharp peaks to be missed however. Here, circular orbits are tested at 17 epochs and eccentric orbits at 35 epochs. The higher sampling rate for eccentric orbits is required as the higher dynamic behaviour causes MVN values to spike sharply during departure onset especially. Under-sampling causes peaks and therefore early stages of MVN departure onset to be missed.

Continuous Function Fits to Discrete Henze-Zirkler Values

The CLT can now be employed to estimate a time frame around the determined mean MVN departure epochs. To achieve this, first, the highest and lowest mean values, \(\bar {x}\), as well as sample standard deviation values, s, within each assessed revolution are identified. Continuous functions \(\bar {x}(t)\) and s(t) are then fit to the mean and sample standard deviation results independently. An example of this is shown in Fig. 3. Values near the MVN decision threshold are given high weights in the fit. This ensures that the variability in the departure epoch is reflected well.

Fig. 3
figure 3

Simulation results from two-body propagation for Alfano’s case 6-1 (see Table 2). Mean values from 50 Monte-Carlo runs evaluated at up to 17 equidistant epochs within a revolution. Top Left: Fits to mean values for transition start (blue line) and conclusion (red line). Top Right: Fits to standard deviation values for transition start (blue line) and conclusion (red line). Light blue crosses are results at epochs which don’t yield revolution minimum or maximum MVN values. Bottom: Results for mean and 3σCLT as per relation (4) are superimposed

Full size image

The time dependent sample standard deviation s(t) can be translated into the standard deviation of the PDF of the mean value σCLT(nmc) as per Eq. 4. With this information, a continuous function can now be established which describes the time evolution of the Henze-Zirkler test statistic, given a user definable confidence level R:

$$ 0 = \bar{x}(t) + R \left[\frac{s(t)}{\sqrt{n_{mc}}}\right] - 1 . $$
(5)

Confidence level values R according to the cumulative standard normal distribution are:

R

− 3

− 2

− 1

0

1

2

3

1 − α [%]

0.1

2.3

15.9

50.0

84.1

97.7

99.9

With R = 0, the relation simplifies back to \(0 = \bar {x}(t) - 1\) and the threshold epoch as per the mean function fit \(\bar {x}(t)\) is obtained. This relates to the 50 % confidence level (1 − α).

MVN Departure Onset and Conclusion Windows

The inversion of Eq. 5 to yield t would allow finding the epoch for MVN departure as function of confidence, R. For the ensuing analysis, R is fixed and t is solved for numerically.

Constructed Test Cases

The assessment of collision probability methods performed by Alfano [1] contains detailed information on states and uncertainties for twelve different close encounter cases. These cases create the basis for the current chapter. Table 2 gives an overview of the simulated scenarios. For more detail, the reader is referred to the original publication. The first number in each case ID is the case number. The second number references the respective object from the close conjunction case. Cases number 3, 11 and 12 differ from other cases only in parameters not pertaining to the state and covariance and are thus neglected here. Instead the GEO cases have been duplicated with all terms in the covariance matrices multiplied by a factor of 100 (IDs ending with ‘a’).

It should be noted that none of the cases in this section have position-velocity co-variances. This is a crucial difference to the state estimation error covariance matrices contained in SERC’s catalogue at the time of writing and which are used in Section “Departure from Normality for Catalogue OD Results”.

Table 2 Overview of calibration cases based on [1]. Nomenclature is scenario No.-object No.
Full size table

Results for the transition onset and conclusion epochs based on the Henze-Zirkler Method and using 50 Monte-Carlo runs at a sample size of 10 000 are visualised in Fig. 4. The results are characterised in terms of the mean value of each Monte-Carlo analysis and its variability based on the CLT applying the method outlined in Section “Monte-Carlo Runs & Central Limit Theorem”. The variability of results using the CLT is determined for the 0.1 % and 99.9 % (± 3σ) confidence levels. For the GEO cases with enlarged uncertainties, the departure from normality simply advances in time. Although this may seem correct intuitively, it has been observed that not all terms have the same effect; e.g. an appropriate selection of covariance terms for instance have been observed to allow normality to break down later, even though variances may be higher in position as well as velocity.

Fig. 4
figure 4

Beginning and end of transition into non-normality of position uncertainty. Scenario identifiers refer to those detailed in Table 2. Error bars are for the CLT standard deviation ± 3σCLT. Blue is for MVN departure onset epoch. Red is for MVN departure conclusion epoch

Full size image

Efficient MVN Testing

The transition start and end epochs described in Section “Example Evolution of Henze-Zirkler MVN Test Values” have different implications for different methods of covariance propagation. An error-state transition matrix approach uses a prior covariance matrix to produce an update. As no information on non-normally distributed uncertainties is contained in this representation, any covariance matrix produced after the transition phase is entered may no longer be reliable. The covariance matrix propagation based on the unscented transform on the other hand allows each particle to move independently through time and space. The covariance matrix generated from these at any epoch requires no prior and so will accurately portray normally distributed uncertainties within the transition region. The key here is to identify at each epoch, whether the underlying uncertainty space is normally distributed or not.

Unscented Transform based Normalised Offset (UNO)

In astrodynamics, non linear two-body motion causes any normal uncertainty volume to bend in two dimensional space after some time. This deformation from normality is often used in the literature to visualise an important effect of non-linear motion in astrodynamics on state uncertainty divergence from normality [2, 13, 14, 17]. For any covariance extrapolation method that retains information on this bending, the deformation should be quantifiable simply by the offset of the propagated state z(0) from the propagated covariance mean state \(\bar {\mathbf {z}}\). Apart from the bending, this offset should also be able to detect non-symmetry in one-dimensional normal distributions (skewing). Kurtosis on the other hand, which describes a symmetrical deviation pertaining only to the tails of the PDF may remain undetected.

Here, the unscented transform is chosen for its efficiency and ease of implementation as covariance extrapolation method. More specifically, the symmetric set extended by the mean as per Equation (12) of [15, 16] is applied. The sample covariance mean state is determined when the covariance is estimated from the given set of sigma points. Reference [15] defines it as “the weighted average of the transformed points”:

$$ \bar{\mathbf{z}} = \sum\limits_{i = 0}^{p} W^{(i)} \mathbf{z}^{(i)} . $$
(6)

Herein, p is the number of states generated by the Unscented Transform,Footnote 6z(i) is the ith state and W(i) is the corresponding weight. The offset used to measure the deviation from normality is labelled NED for normalised Euclidean distance in this paper:

$$ \text{NED} = \sqrt{\mathbf{D}} = \large|\large| \mathbf{L}^{-1} \left( \mathbf{z}^{(0)} - \bar{\mathbf{z}}\right)\large|\large| . $$
(7)

L− 1 is the inverse lower triangular – obtained through the Cholesky decomposition – of the sample covariance matrix. The multiplication of the vector difference with the inverse lower triangular matrix retrieves the relative vector in whitened space. Its magnitude is the Mahalanobis distance. This normalisation is required to decouple the offset from orbit regime and covariance size. Figure 5 visualises the method.

Fig. 5
figure 5

Sketch of the idea behind the Unscented Transform based Normalised Offset (UNO) MVN test metric

Full size image

Calibration

Unlike the Henze-Zirkler test, the NED is not subject to a hypothesis test. Thus, we must determine which NED we believe to yield a reasonable estimate for detection of significant non-normality in the propagated PDF of the state estimation error. Rather than choosing an arbitrary value, here we attempt a calibration by comparison to the Henze-Zirkler test results presented in the previous section. If a strong relationship exists between the two methods, then it should be possible to associate each Henze-Zirkler value exactly one NED. Ideally, this should be possible regardless of the initial covariance or orbit type. Some variability in the results is expected due to the established variability in the Henze-Zirkler mean values. More pronounced deviations may however occur due to the fundamental differences between the two tests as well as short comings in the UNO test to detect certain deviations from normality as outlined in the previous section.

The left hand plot in Fig. 6 plots the mean of the Henze-Zirkler test statistic vs. NED for all cases of Section “Constructed Test Cases”. A strong relation between the two can be seen up to a Henze-Zirkler test value of about 2. Beyond, the relation is no longer independent of orbit or covariance. As we are only interested in transition between normal and non-normal state uncertainties, a trend is fit to the region below the Henze-Zirkler Test value of 1.5. Its intersection with the normality boundary given by the Henze-Zirkler test yields the calibrated decision threshold for UNO which is: NED = 0.0501. This value is rounded to 0.05 in the ensuing computations.

Fig. 6
figure 6

Henze-Zirkler test statistic vs. NED for all performed baseline simulations. Accounting for uncertainties in terms of the central limit theorem, excellent agreement between all values is observed in the vicinity of the MVN departure threshold

Full size image

Comparing UNO and Henze-Zirkler Test for Constructed Cases

Table 3 contains the epochs for transition onset and conclusion calculated with UNO as well as the Henze-Zirkler test. It can be seen that for all orbit types including highly eccentric, the results of the Unscented Transform based test are well within the ± 3σCLT window of the Henze-Zirkler test.

Table 3 Beginning and end of transition into non-normality of position uncertainty
Full size table

Departure from Normality for Catalogue OD Results

Measurements from passive and active optical measurements of objects in all Earth orbits are being processed in a Batch-Least-Squares orbit determination and ingested into a catalogue. An account of the status of this catalogue is made in the papers by Dr. Michael Lachut et al. [19] and James C.S. Bennett et al. [4]. The OD results assessed here are from mid 2018 and include passive observations only.

The current section should be seen as a proof of concept as at the time of writing the SERC catalogue is still in its build-up phase and the presented efficient method for MVN testing is still in its early testing stage.

Overview of Cases

An overview of example orbit determination results which are assessed in the following is given in Table 4.

Table 4 Overview of orbit determination example cases
Full size table

Comparing UNO and Henze-Zirkler for OD Cases

Table 5 compares the MVN validity period of OD cases from SERC’s catalogue given in Table 4. The Henze-Zirkler test statistic is averaged from 50 propagated samples.

Table 5 Beginning and end of transition into non-normality of position uncertainty using Henze-Zirkler test with nmc = 50 and UNO with MVN departure decision threshold of 0.05. MVN is evaluated at 35 epochs within each revolution
Full size table

The results for the transition start show good agreement. The highest deviations between the two methods are observed for cases with short time frames until onset of MVN departure. As departure initially occurs only once or twice a revolution, a difference of one revolution between methods yields a higher relative deviation than for cases where the same difference is obtained but at a later departure onset epoch.

The predicted transition end epoch in contrast shows very high deviations in at least three of the nine test cases. The highest deviation is observed for OD ID 3778, but also IDs 3783 and 3769 show deviations which are unlikely to be due to variability in the Henze-Zirkler results, given the markedly better agreement across the other cases. Important differences of the current cases compared to cases of the previous section used for calibration include i) different ratios of position-velocity variances as well as ii) non-zero co-variance terms in the initial covariance matrix. In Section “Unscented Transform based Normalised Offset (UNO)”, the lacking capability in the current version of UNO in detecting kurtosis has already been identified as a short-coming. Whether the observed differences are due to such an effect has yet to be determined.

Looking at how the results of the two methods differ between constructed test cases (Section “Comparing UNO and Henze-Zirkler Test for Constructed Cases”) and actual OD result cases (this Section), the importance of gaining a sound understanding of capabilities and limitations of any method is highlighted.

A Note on Implementation and Computational Efficiency

The software is implemented in C++ with multithreading support via OpenMP. Compilation is performed with the GNU Compiler Collection release (gcc-9-1.5) for openSUSE Tumbleweed. The majority of underlying mathematical functions required for the MVN tests are implemented based on [25]. The NOVAS library Version C3.1 [3] is used for reference frame conversions. SERC’s internal library libSERC introduced and maintained by Dr. Marek Möckel (SERC) provides additional functionality for reference frame conversions, mathematical functions as well as I/O handling. The hardware used for the following comparison relies on an Intel(R) Core(TM) i7-8700T CPU. The given execution times are the total program run time which includes input file reading and output file writing activities. The comparison is based on evaluation of the state estimation error at a single epoch for the 3D position only. The Henze-Zirkler test is performed using nmc = 50 Monte-Carlo runs at a sample size of np = 10 000.

Under these conditions, the Henze-Zirkler test requires just under four minutes in single-thread mode and just over two minutes using 10 threads; this suggests, that the code has room for optimization. The more efficient UNO test takes about seven milliseconds on a single thread; no multithreading support is implemented for this test.

Conclusions

The Henze-Zirkler test for Multivariate Normality is used to test random samples drawn from the covariance matrix representing the state estimation error and propagated through time using two-body motion. A structure is observed in the time evolution of this parameter when Monte-Carlo analysis is employed which underlies all tested LEO, MEO, HEO and GEO cases. After a period of remaining normally distributed over one entire revolution, the state uncertainty enters a phase of transition wherein it will become non-normally distributed for increasing parts throughout a revolution. Finally, the state uncertainty remains non-normally distributed throughout the entire revolution. During this phase, results between samples are most aligned and show a clear structure. After some time, the structure starts to dissolve. Non-normality however remains over the entire revolution.

A more efficient method which operates solely on the states computed using an Unscented Transform has been introduced. Both methods have been applied to state and covariances detailed in a Monte-Carlo study of probability of collision methods by Alfano in 2009 [1] as a baseline as well as orbit determination results from SERC’s own catalogue.

A strong relation between this method and the Henze-Zirkler test has been demonstrated in the relevant region surrounding the state normality decision boundary for all baseline cases and some, but not all, orbit determination result cases. Significant deviations in three out of nine cases in particular in the detection of the MVN departure conclusion exist. Given the fundamental difference of the two methods, deviations are to be expected. The source of these deviations is not yet fully understood however and requires further investigation.