Quantitative Neuroscience pp 251-262 | Cite as
Testing Whether a Prediction Scheme is Better than Guess
Abstract
To be able to predict a future event is the most convincing side of science. Usually in the beginning of an investigation, the prediction is not perfect, i.e., an event may be missing or a prediction turns out to be a false alarm. When past prediction records are available, can we determine whether the prediction scheme is promising? In this paper, we use the naive optimal prediction as a yard stick, i.e., testing whether the new prediction scheme is better than the naive optimal scheme through statistical hypothesis testing. Here the naive scheme is defined as we know only the distribution of the inter-arrival times without any other auxiliary information. We use the trade off curve between false alarm rate and sensitivity to measure the prediction performance and the bootstrap method to compute the p-value. Real data and simulation examples are presented.
Keywords
False Alarm Lyapunov Exponent False Alarm Rate Epileptic Seizure Auxiliary InformationPreview
Unable to display preview. Download preview PDF.
References
- [1]S.B. Cantor and M.W. Kattan. Determining the area under the curve under the roc curve for a binary diagnostic test. Med Decls making, 20: 488–470, 2000.Google Scholar
- [2]D.L. Fairclough. Design and Analysis of Quality of Life Studies in Clinical Trials. Chapman & Hall/CRC, Boca Raton, 2002.Google Scholar
- [3]W.A. Hauser and L.T. Kurland. The epidemiology of epilepsy in rochester, minnessota, 1935 through 1967. Epilepsia, 16: 1–66, 1975.CrossRefGoogle Scholar
- [4]L.D. Iasemidis, P.M. Pardalos, J.C. Sackellares, and D.-S. Shiau. Quadratic binary programming and dynamical system approach to determine the predictability of epileptic seizures. Journal of Combinatorial Optimization, 5: 9–26, 2001.MathSciNetMATHCrossRefGoogle Scholar
- [5]L.D. Iasemidis and J.C. Sackellares. Chaos theory and epilepsy. The Neuroscientist, 2: 118–126, 1996.CrossRefGoogle Scholar
- [б]P. Joensen. Prevalence, incidence and classification of epilepsy in the faroes. Acta Neurologica Scandinavica, 76: 150–155, 1986.CrossRefGoogle Scholar
- [7]K. Lehnertz and C.E. Elger. Can epileptic seizures be predicted? evidence from nonlinear time series analysis of brain electrical activity. Phys. Rev. Lett., 80: 5019–5022, 1998.CrossRefGoogle Scholar
- [8]T. Maiwald. The seizure prediction characteristic: A new terminology and assessment criterion for epileptic seizure prediction methods. Presented at the Conference on Quantitative Neuroscience, University of Florida, February 2003.Google Scholar
- [9]H.L.V. Quyen et al. Anticipation of epileptic seizures from standard eeg recordings. Lancet, 357: 183–188, 2001.CrossRefGoogle Scholar
- [10]S.M. Ross. Stochastic Processes. John Wiley & Sons, New York, 1983.MATHGoogle Scholar
- [11]E.F. Schisterman, D. Faraggi, B. Reiser, and M. Trevisan. Statistical inference for the area under the receiver operating characteristic curve in the presence of random measurement error. Am JEpidemoil, 154 (2): 174–179, 2001.CrossRefGoogle Scholar
- [12]D. Vere-Jones. Forecasting earthquakes and earthquake risk. Internal J Forecasting, 11: 530–538, 1995.Google Scholar