On the Theory and Applications of Sequence Based Estimation of Independent Binomial Random Variables

  • B. John Oommen
  • Sang-Woon Kim
  • Geir Horn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4109)


We re-visit the age-old problem of estimating the parameters of a distribution from its observations. Traditionally, scientists and statisticians have attempted to obtain strong estimates by “extracting” the information contained in the observations taken as a set. However, generally speaking, the information contained in the sequence in which the observations have appeared, has been ignored – i.e., except to consider dependence information as in the case of Markov models and n-gram statistics. In this paper, we present results which, to the best of our knowledge, are the first reported results, which consider how estimation can be enhanced by utilizing both the information in the observations and in their sequence of appearance. The strategy, known as Sequence Based Estimation (SBE) works as follows. We first quickly allude to the results pertaining to computing the Maximum Likelihood Estimates (MLE) of the data when the samples are taken individually. We then derive the corresponding MLE results when the samples are taken two-at-a-time, and then extend these for the cases when they are processed three-at-a-time, four-at-a-time etc. In each case, we also experimentally demonstrate the convergence of the corresponding estimates. We then suggest various avenues for future research, including those by which these estimates can be fused to yield a superior overall cumulative estimate of the parameter of the distribution. We believe that our new estimates have great potential for practitioners, especially when the cardinality of the observation set is small.


Maximum Likelihood Estimate Real Root Transmission Control Protocol Internet Protocol Arithmetic Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bickel, P., Doksum, K.: Mathematical Statistics: Basic Ideas and Selected Topics, 2nd edn., vol. I. Prentice Hall, Englewood Cliffs (2000)Google Scholar
  2. 2.
    Bunke, H.: Structural and Syntactic Pattern Recognition. Handbook of Pattern Recognition and Computer Vision, p. 25. World Scientific, Singapore (1993)Google Scholar
  3. 3.
    Casella, G., Berger, R.: Statistical Inference, 2nd edn. Brooks/Cole Pub. Co. (2001)Google Scholar
  4. 4.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley and Sons, Inc., New York (2000)Google Scholar
  5. 5.
    Feng, W.-C., Kandlur, D.D., Saha, D., Shin, K.G.: Adaptive packet marking for maintaining end-to-end throughput in a differentiated-services internet. IEEE/ACM Transactions on Networking 7, 685–697 (1999)CrossRefGoogle Scholar
  6. 6.
    Glen, J., Langdon, G.: Arithmetic coding. IBM Journal of Research and Development 28, 135–149 (1984)MATHCrossRefGoogle Scholar
  7. 7.
    Herbrich, R.: Learning Kernel Classifiers: Theory and Algorithms. MIT Press, Cambridge (2001)Google Scholar
  8. 8.
    Jones, B., Garthwaite, P., Jolliffe, I.: Statistical Inference, 2nd edn. Oxford University Press, Oxford (2002)MATHGoogle Scholar
  9. 9.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. and Machine Intell. 20(3), 226–239 (1998)CrossRefGoogle Scholar
  10. 10.
    Kuncheva, L.I.: A theoretical study on six classifier fusion strategies. IEEE Trans. Pattern Anal. and Machine Intell. 24(2), 281–286 (2002)CrossRefGoogle Scholar
  11. 11.
    Oommen, B.J., Kim, S.-W., Horn, G.: On the Estimation of Binomial Random Variables Using Occurrence and Sequential Information (Unabridged version of this paper)Google Scholar
  12. 12.
    Rissanen, J., Glen, J., Langdon, G.: Arithmetic coding. IBM Journal of Research and Development 23, 149–162 (1979)MATHCrossRefGoogle Scholar
  13. 13.
    Tanenbaum, A.S.: Computer Networks, 4th edn. Prentice-Hall, New Jersey (2003)Google Scholar
  14. 14.
    U.o.S.C. Information Science Institute, Transmission control protocol (TCP) (September 1981),; Clarifications, corrections, extensions and a textbook introduction can be found in [13]
  15. 15.
    White, P.P.: RSVP and integrated services in the internet: A tutorial. IEEE Communications Magazine, 100–106 (May 1997)Google Scholar
  16. 16.
    Witten, I.H., Neal, R.M., Cleary, J.G.: Arithmetic coding for data compression. Communications of the ACM 30, 520–540 (1987)CrossRefGoogle Scholar
  17. 17.
    Xiao, X., Ni, L.M.: Internet QoS: A big picture. IEEE Network 13, 8–18 (1999)CrossRefGoogle Scholar
  18. 18.
    Webb, A.: Statistical Pattern Recognition, 2nd edn. John Wiley & Sons, New York (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • B. John Oommen
    • 1
  • Sang-Woon Kim
    • 2
  • Geir Horn
    • 3
  1. 1.Professor and Fellow of the IEEE, School of Computer ScienceCarleton UniversityOttawaCanada
  2. 2.Dept. of Computer Science and EngineeringMyongji UniversityYonginKorea
  3. 3.SIMULA Research LaboratoryFornebuNorway

Personalised recommendations