Advertisement

Impact of Outlier Detection on Neural Networks Based Property Value Prediction

  • Sayali SandbhorEmail author
  • N. B. Chaphalkar
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 862)

Abstract

Detecting outliers is an important step in data mining. Outliers not only hamper data quality but also affect the output in case of prediction models. Prediction tools like Neural Networks (NN) need outlier free dataset in order to achieve better generalization of the network as errors in the dataset hinder the modelling process and produce misleading results. Thus, range of the dataset needs to be curbed in order to make it fit for generating better prediction results. However, outlier detection faces one difficulty. There is no standard framework for the treatment of outliers found in the literature. The present study is an effort to identify the most suited outlier detection method for a specific problem, which deals with the use of NN for prediction of real property value. 3094 cases of property sale instances are presented to various univariate outlier detection methods like Tukey’s method, Standard Deviation (SD) method, median method, Z score method, MAD method, modified Z score method, etc. The datasets prepared after removing outliers marked for respective methods are used for prediction using NN. Comparison of results show that the median method is the best-suited outlier detection method for the present study.

Keywords

Outliers Data mining Neural networks Valuation Real property Data preprocessing 

References

  1. 1.
    R. Johnson, D.W. Wichern, Applied Multivariate Statistical Analysis, 6th edn. (Prentice Hall, Englewood Cliffs, 1992)zbMATHGoogle Scholar
  2. 2.
    V. Chandola, A. Banerjee, V. Kumar, Anomaly detection: a survey. ACM Comput. Surv. 41(3), Article 15 (2009)CrossRefGoogle Scholar
  3. 3.
    V. Barnett, T. Lewis, Outliers in Statistical Data, 3rd edn. (Wiley, New York, 1994). (ISBN 978-0-471-93094-5)zbMATHGoogle Scholar
  4. 4.
    I. Ben Gal, Outlier detection, in Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, ed. by O. Maimon, L. Rockach (Kluwer Academic Publishers, Dordrecht, 2005)Google Scholar
  5. 5.
    R.J. Bullen, D. Cornforda, I.T. Nabneya, Outlier detection in scatterometer data: neural network approaches. Neural Netw. Spec. Issue 16, 419–426 (2003)CrossRefGoogle Scholar
  6. 6.
    G.J. Williams, R.A. Baxter, H.X. He, S. Hawkins, L. Gu, A comparative study of RNN for outlier detection in data mining, in IEEE International Conference on Data-Mining (ICDM’02), Maebashi City, Japan, CSIRO Technical Report CMIS-02/102 (2002)Google Scholar
  7. 7.
    H. Liu, S. Shah, W. Jiang, On-line outlier detection and data cleaning. Comput. Chem. Eng. 28, 1635–1647 (2004)CrossRefGoogle Scholar
  8. 8.
    S. Seo, A review and comparison of methods for detecting outliers in univariate data sets, Dissertation, Graduate School of Public Health, Kyunghee University, University of Pittsburgh (2006). Accessed at: http://d-scholarship.pitt.edu/7948/. Accessed on 20 Apr 2017
  9. 9.
    D. Cousineau, S. Chartier, Outliers detection and treatment: a review. Int. J. Psychol. Res. 3(1), 58–67 (2010)CrossRefGoogle Scholar
  10. 10.
    N. Upasania, H. Omb, Evolving fuzzy min-max neural network for outlier detection, in International Conference on Advanced Computing Technologies and Applications (ICACTA-2015) Proceedia Computer Science, vol. 45, pp. 753–761 (2015)CrossRefGoogle Scholar
  11. 11.
    A.M. Rajeswari, M. Sridevi, C. Deisy, Outliers detection on educational data using fuzzy association rule mining, in International Conference on Advance in Computer Communication and Information Science (ACCIS-14), pp. 1–9 (2014)Google Scholar
  12. 12.
    M. Nkurunziza, L. Vermeire, A comparison of outlier labelling criteria in univariate measurements, Sustainability in statistics education, in Proceedings of the Ninth International Conference on Teaching Statistics (ICOTS9, July, 2014), Flagstaff, Arizona, USA (International Statistical Institute, Voorburg, The Netherlands), pp. 1–4 (2014)Google Scholar
  13. 13.
    M. Markou, S. Singh, Novelty detection: a review-part 1: statistical approaches. Sig. Process. 83(12), 2481–2497 (2003)CrossRefGoogle Scholar
  14. 14.
    M. Markou, S. Singh, Novelty detection: a review-part 2: neural network based approaches. Sig. Process. 83(12), 2499–2521 (2003)CrossRefGoogle Scholar
  15. 15.
    V. Hodge, J. Austin, A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85–126 (2004)CrossRefGoogle Scholar
  16. 16.
    M. Agyemang, K. Barker, R. Alhajj, A comprehensive survey of numeric and symbolic outlier mining techniques. Intell. Data Anal. 10(6), 521–538 (2006)CrossRefGoogle Scholar
  17. 17.
    A. Patcha, J.M. Park, An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput. Netw. 51(12), 3448–3470 (2007)CrossRefGoogle Scholar
  18. 18.
    P. Gogoi, D.K. Bhattacharyya, B. Borah, J.K. Kalita, A survey of outlier detection methods in network anomaly identification. Comput. J. 54(4), 570–588 (2011)CrossRefGoogle Scholar
  19. 19.
    M. Daszykowski, K. Kaczmarek, Y. Vander Heyden, B. Walczak, Robust statistics in data analysis-a review basic concepts. Chemometr. Intell. Lab. Syst. 85, 203–219 (2007)CrossRefGoogle Scholar
  20. 20.
    J.W. Tukey, Exploratory Data Analysis, 1st edn. (Addison-Wesely Publishers, Reading, MA, 1977). (ISBN-13: 978-0201076165)zbMATHGoogle Scholar
  21. 21.
    S.S. Tripathy, R.K. Saxena, P.K. Gupta, Comparison of statistical methods for outlier detection in proficiency testing data on analysis of lead in aqueous solution. Am. J. Theor. Appl. Stat. 2(6), 233–242 (2013)CrossRefGoogle Scholar
  22. 22.
    A. Gautam, V. Bhateja, A. Tiwari, S.C. Satapathy, An improved mammogram classification approach using back propagation neural network, in Data Engineering and Intelligent Computing (Springer, Singapore, 2018), pp. 369–376Google Scholar
  23. 23.
    S. Sandbhor, N.B. Chaphalkar, Effect of training sample and network characteristics in neural network based real property value prediction, Proceedings of 2nd International Conference on Data Engineering and Communication Technology (ICDECT) 2017, 15th–16th December 2017, ‘Advances in Intelligent Systems and Computing (AISC) Series of Springer’ (SCOPUS), 303–313 (2018)Google Scholar
  24. 24.
    D. Tay, D. Ho, Artificial intelligence and the mass appraisal of residential apartments. J. Prop. Valuat. Invest. 10, 525–539 (1991)CrossRefGoogle Scholar
  25. 25.
    A. Do, G. Grudnitski, A neural network approach to residential property appraisal. R. Estate Apprais. 58, 38–45 (1992)Google Scholar
  26. 26.
    N. Nghiep, Al Cripps, Predicting housing value: a comparison of multiple regression analysis and artificial neural networks. J. R. Estate Res. 22(3), 313–336 (2001)Google Scholar
  27. 27.
    A. Khamis, Z. Ismail, K. Haron, A. Muhammed, The effects of outliers data on neural network performance. J. Appl. Sci. 5(8), 1394–1398 (2005)Google Scholar
  28. 28.
    S. Sandbhor, N.B. Chaphalkar, Determining attributes of Indian real property valuation using principal component analysis. J. Eng. Technol. 6(2), 483–495 (2017)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.SIT, Symbiosis International (Deemed University)PuneIndia
  2. 2.JSPM’s RSCOEPuneIndia

Personalised recommendations