Missing Data Approaches to Economic Modeling: Optimization Approach

  • Tshilidzi Marwala
Part of the Advanced Information and Knowledge Processing book series (AI&KP)


This chapter introduces an auto-associative network with optimization methods for modelling economic data. This resulting architecture is a missing data estimation technique, and this is used to predict the production volume by treating it as a missing variable. The autoassociative network is created using a multi-layered perceptron network, while the optimization techniques which are implemented are particle swarm optimization, genetic algorithms and simulated annealing. The results obtained are then compared.


Genetic Algorithm Particle Swarm Optimization Simulated Annealing Markov Chain Monte Carlo Markov Chain Monte Carlo Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Abdella M (2005) The use of genetic algorithms and neural networks to approximate missing data in database. Master’s thesis, University of the Witwatersrand, JohannesburgGoogle Scholar
  2. Abdella M, Marwala T (2005) Treatment of missing data using neural networks. In: Proceedings of the IEEE international joint conference on neural networks, Montreal, 2005, pp 598–603Google Scholar
  3. Abdella M, Marwala T (2006) The use of genetic algorithms and neural networks to approximate missing data in database. Comput Inf 24:1001–1013Google Scholar
  4. Arya LD, Choube SC, Arya R, Tiwary A (2012) Evaluation of reliability indices accounting omission of random repair time for distribution systems using Monte Carlo simulation. Int J Electr Power Energy Syst 42:533–541CrossRefGoogle Scholar
  5. Banzhaf W, Nordin P, Keller R, Francone F (1998) Genetic programming-an introduction: on the automatic evolution of computer programs and its applications. Morgan Kaufmann, San FranciscoMATHGoogle Scholar
  6. Botlani-Esfahani M, Toroghinejad MR (2012) Application of a Bayesian artificial neural network and the reversible jump Markov chain Monte Carlo method to predict the grain size of hot strip low carbon steels. J Serb Chem Soc 77:937–944CrossRefGoogle Scholar
  7. Černý V (1985) Thermodynamical approach to the traveling salesman problem: an efficient simulation algorithm. J Optim Theory Appl 45:41–51MathSciNetMATHCrossRefGoogle Scholar
  8. Crossingham B, Marwala T (2008) Using genetic algorithms to optimise rough set partition sizes for HIV data analysis. Adv Intell Distrib Comput Stud Comput Intell 78:245–250CrossRefGoogle Scholar
  9. Dervilis N, Barthorpe R, Antoniadou I, Staszewski WJ, Worden K (2012) Damage detection in carbon composite material typical of wind turbine blades using auto-associative neural networks. In: Proceedings of SPIE – the international society for optical engineering, San Diego, 2012, art. no. 834806Google Scholar
  10. Dindar ZA, Marwala T (2004) Option pricing using a committee of neural networks. Proc IEEE Int Conf Syst Man Cybern 1:434–438Google Scholar
  11. Dong J, Yang J, Lei W, Shi R, Guo Y (2012) Antenna array design in MIMO radar using cyclic difference sets and simulated annealing. In: Proceedings of the international conference on microwave and millimeter wave technology, Shenzhen, China, pp 237–240Google Scholar
  12. Engebrecht AP (2005) Fundamentals of computational swarm intelligence. Wiley, New YorkGoogle Scholar
  13. Fang S-E, Ren W-X, Perera R (2012) A stochastic model updating method for parameter variability quantification based on response surface models and Monte Carlo simulation. Mech Syst Signal Process 33:83–96CrossRefGoogle Scholar
  14. Farzi S, Shavazi AR, Pandari AR (2013) Using quantum-behaved particle swarm optimization for portfolio selection problem. Int Arab J Inf Technol 10:art. no. 2/7-2761Google Scholar
  15. Fishman GS (2012) Counting contingency tables via multistage Markov chain Monte Carlo. J Comput Graph Stat 21:713–738MathSciNetCrossRefGoogle Scholar
  16. Fonseca GHG, Brito SS, Santos HG (2012) A simulated annealing based approach to the high school timetabling problem. Lect Note Comput Sci 7435:540–549CrossRefGoogle Scholar
  17. Gholizadeh S, Fattahi F (2012) Design optimization of tall steel buildings by a modified particle swarm algorithm. Struct Design Tall Spec Build. doi: 10.1002/tal.1042 Google Scholar
  18. Giraleas D, Emrouznejad A, Thanassoulis E (2012) Productivity change using growth accounting and frontier-based approaches – evidence from a Monte Carlo analysis. Eur J Oper Res 222:673–683CrossRefGoogle Scholar
  19. Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, ReadingMATHGoogle Scholar
  20. Goldberg DE (2002) The design of innovation: lessons from and for competent genetic algorithms. Addison-Wesley, ReadingMATHGoogle Scholar
  21. Granville V, Krivanek M, Rasson J-P (1994) Simulated annealing: a proof of convergence. IEEE Trans Pattern Anal Mach Intell 16:652–656CrossRefGoogle Scholar
  22. Hettiarachchi I, Mohamed S, Nahavandi S (2012) A marginalised Markov chain Monte Carlo approach for model based analysis of EEG data. In: Proceedings of the international symposium on biomedical imaging, Barcelona, 2012, pp 1539–1542Google Scholar
  23. Hlalele N, Nelwamondo FV, Marwala T (2009) Imputation of missing data using PCA, neuro-fuzzy and genetic algorithms. Lect Note Comput Sci 5507:485–492CrossRefGoogle Scholar
  24. Holland JH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann ArborGoogle Scholar
  25. Hulley G, Marwala T (2007) Genetic algorithm based incremental learning for optimal weight and classifier selection. Comput Model Life Sci Am Inst Phys Ser 952:258–267Google Scholar
  26. Jeremiah E, Sisson SA, Sharma A, Marshall L (2012) Efficient hydrological model parameter optimization with sequential Monte Carlo sampling. Environ Model Softw 38:283–295CrossRefGoogle Scholar
  27. Kalatehjari R, Ali N, Hajihassani M, Kholghi Fard M (2012) The application of particle swarm optimization in slope stability analysis of homogeneous soil slopes. Int Rev Model Simul 5:458–465Google Scholar
  28. Karabulut M, Ibrikci T (2012) A Bayesian scoring scheme based particle swarm optimization algorithm to identify transcription factor binding sites. Appl Soft Comput J 12:2846–2855CrossRefGoogle Scholar
  29. Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of IEEE international conference on evolutionary computation, Piscataway, 1997, pp 303–308Google Scholar
  30. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, Piscataway, 1995, pp 1942–1948Google Scholar
  31. Kennedy J, Eberhart RC (2001) Swarm intelligence. Morgan Kaufmann, San FranciscoGoogle Scholar
  32. Kim JK, Shin DW (2012) The factoring likelihood method for non-monotone missing data. J Korean Stat Soc 41:375–386CrossRefGoogle Scholar
  33. Kirkpatrick S, Gelatt CD, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680MathSciNetMATHCrossRefGoogle Scholar
  34. Klopfer M, Wolowiec T, Satchouk V, Alivov Y, Molloi S (2012) Characterization and optimization of pyroelectric X-ray sources using Monte Carlo spectral models. Nucl Instrum Method Phys Res Sec A Accel Spectrom Detect Assoc Equip 689:47–51CrossRefGoogle Scholar
  35. Kramer MA (1992) Autoassociative neural networks. Comput Chem Eng 16:313–328CrossRefGoogle Scholar
  36. Laloy E, Linde N, Vrugt JA (2012) Mass conservative three-dimensional water tracer distribution from MCMC inversion of time-lapse GPR data. Water Resour Res. doi: 10.1029/2011WR011238 (in press)
  37. Lee C-H, Xu X, Eun DY (2012) Beyond random walk and metropolis-hastings samplers: why you should not backtrack for unbiased graph sampling. Perform Eval Rev 40:319–330CrossRefGoogle Scholar
  38. Leke B, Marwala T, Tim T, Lagazio M (2006) Using genetic algorithms versus line search optimization for HIV predictions. Trans Inf Sci Appl 4:684–690Google Scholar
  39. Lemma TA, Hashim FM (2012) Wavelet analysis and auto-associative neural network based fault detection and diagnosis in an industrial gas turbine. In: Proceedings of the IEEE business, engineering and industrial applications colloquium, Kuala Lumpur, 2012, pp 103–108Google Scholar
  40. Liu L, Mu H, Luo H, Li X (2012) A simulated annealing for multi-criteria network path problems. Comput Oper Res 39:3119–3135MathSciNetCrossRefGoogle Scholar
  41. Makki B, Hosseini MN (2012) Some refinements of the standard autoassociative neural network. Neural Comput Appl. doi: 10.1007/s00521-012-0825-5 (in press)
  42. Markowitz H (1952) Portfolio selection. J Finance 7:77–91Google Scholar
  43. Martins TDC, De Camargo EDLB, Lima RG, Amato MBP, Tsuzuki MDSG (2012) Image reconstruction using interval simulated annealing in electrical impedance tomography. IEEE Trans Biomed Eng 59:1861–1870CrossRefGoogle Scholar
  44. Marwala T (2002) Finite element updating using wavelet data and genetic algorithm. Am Inst Aeronaut Astronaut J Aircraft 39:709–711CrossRefGoogle Scholar
  45. Marwala T (2004) Control of complex systems using Bayesian neural networks and genetic algorithm. Int J Eng Simul 5:28–37Google Scholar
  46. Marwala T (2005) Finite element model updating using particle swarm optimization. Int J Eng Simul 6:25–30Google Scholar
  47. Marwala T (2007) Bayesian training of neural network using genetic programming. Pattern Recognit Lett 28:452–1458CrossRefGoogle Scholar
  48. Marwala T (2009) Computational intelligence for missing data imputation, estimation and management: knowledge optimization techniques. IGI Global Publications, New YorkCrossRefGoogle Scholar
  49. Marwala T (2010) Finite element model updating using computational intelligence techniques. Springer, LondonMATHCrossRefGoogle Scholar
  50. Marwala T (2012) Condition monitoring using computational intelligence methods. Springer, LondonCrossRefGoogle Scholar
  51. Marwala T, Chakraverty S (2006) Fault classification in structures with incomplete measured data using autoassociative neural networks and genetic algorithm. Curr Sci 90:542–548Google Scholar
  52. Marwala T, Lagazio M (2011) Militarized conflict modeling using computational intelligence techniques. Springer, LondonCrossRefGoogle Scholar
  53. Marwala T, de Wilde P, Correia L, Mariano P, Ribeiro R, Abramov V, Szirbik N, Goossenaerts J (2001) Scalability and optimisation of a committee of agents using genetic algorithm. In: Proceedings of the international symposium on soft computing and intelligent systems for industry, Florida, USA, arXiv 0705.1757Google Scholar
  54. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21:1087CrossRefGoogle Scholar
  55. Miki M, Hiroyasu T, Wako J, Yoshida T (2003) Adaptive temperature schedule determined by genetic algorithm for parallel simulated annealing. In: Proceedings of the congress on evolutionary computation, Canberra, Australia, pp 459–466Google Scholar
  56. Milenkovic M, Bojovic N, Ribeiro RA, Glisovic N (2012) A fuzzy simulated annealing approach for project time–cost tradeoff. J Intell Fuzzy Syst 23:203–215Google Scholar
  57. Miranda V, Castro ARG, Lima S (2012) Diagnosing faults in power transformers with autoassociative neural networks and mean shift. IEEE Trans Power Deliv 27:1350–1357CrossRefGoogle Scholar
  58. Mohamed S, Tettey T, Marwala T (2006) An extension neural network and genetic algorithm for bearing fault classification. In: Proceedings of the IEEE international joint conference on neural networks, Vancouver, Canada, pp 7673–7679Google Scholar
  59. Moreau V, Bage G, Marcotte D, Samson R (2012) Statistical estimation of missing data in life cycle inventory: an application to hydroelectric power plants. J Clean Prod 37:335–341CrossRefGoogle Scholar
  60. Muthukaruppan S, Er MJ (2012) A hybrid particle swarm optimization based fuzzy expert system for the diagnosis of coronary artery disease. Expert Syst Appl 39:11657–11665CrossRefGoogle Scholar
  61. Nasir M, Das S, Maity D, Sengupta S, Halder U, Suganthan PN (2012) A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization. Inf Sci 209:16–36MathSciNetCrossRefGoogle Scholar
  62. Nelwamondo FV (2008) Computational intelligence techniques for missing data imputation. Ph.D. thesis, University of the Witwatersrand, JohannesburgGoogle Scholar
  63. Nelwamondo FV, Marwala T (2007) Rough set theory for the treatment of incomplete data. In: Proceedings of the IEEE conference on fuzzy systems, London, UK, pp 338–343Google Scholar
  64. Palmé T, Breuhaus P, Assadi M, Klein A, Kim M (2011) Early warning of gas turbine failure by nonlinear feature extraction using an auto-associative neural network approach. Proc ASME Turbo Expo 3:293–304Google Scholar
  65. Patel PB, Marwala T (2009) Genetic algorithms, neural networks, fuzzy inference system, support vector machines for call performance classification. In: Proceedings of the IEEE internationall conference on machine learning and applications, Florida, USA, pp 415–420Google Scholar
  66. Ransome TM, Rubin DM, Marwala T, de Kok EA (2005) Optimising the verification of patient positioning in proton beam therapy. In: Proceedings of the IEEE 3rd international conference on computational cybernetics, Mauritius, 2005, pp 279–284Google Scholar
  67. Rey-del-Castillo P, Cardeñosa J (2012) Fuzzy min-max neural networks for categorical data: application to missing data imputation. Neural Comput Appl 21:1349–1362CrossRefGoogle Scholar
  68. Salazar R, Toral R (1997) Simulated annealing using hybrid Monte Carlo. J Stat Phys 89(5/6):1047–1060MATHCrossRefGoogle Scholar
  69. Shao W, Zuo Y (2012) Simulated annealing for higher dimensional projection depth. Comput Stat Data Anal 56:4026–4036MathSciNetMATHCrossRefGoogle Scholar
  70. Shao W, Guo G, Meng F, Jia S (2012) An efficient proposal distribution for metropolis-hastings using a B-splines technique. Comput Stat Data Anal 57:465–478MathSciNetCrossRefGoogle Scholar
  71. Sheridan P, Yagahara Y, Shimodaira H (2012) Measuring preferential attachment in growing networks with missing-timelines using Markov chain Monte Carlo. Physica A Stat Mech Appl 391:5031–5040CrossRefGoogle Scholar
  72. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of IEEE international conference on evolutionary computation, Anchorage, Alaska, pp 69–73Google Scholar
  73. Sivaram GSVS, Ganapathy S, Hermansky H (2010) Sparse auto-associative neural networks: theory and application to speech recognition. In: Proceedings of the 11th annual conference of the international speech communication association, Florence, Italy, pp 2270–2273Google Scholar
  74. Stander J, Silverman BW (1994) Temperature schedules for simulated annealing. Stat Comput 4:21–32CrossRefGoogle Scholar
  75. Stošić BD, Santos Silva JR, Filho MC, Barros Cantalice JR (2012) Optimizing river discharge measurements using Monte Carlo Markov chain. J Hydrol 450–451:199–205Google Scholar
  76. Tettey T, Marwala T (2006) Controlling interstate conflict using neuro-fuzzy modeling and genetic algorithms. In: Proceedings of the 10th IEEE international conference on intelligence engineering systems, London, UK, pp 30–44Google Scholar
  77. Tian L, Chen H-G, Zhu J, Zhang L-S, Chen W-H (2012) A study of optimal sensor placement based on the improved adaptive simulated annealing genetic algorithms. J Vib Eng 25:238–243Google Scholar
  78. Tsai LT, Yang C-C (2012) Improving measurement invariance assessments in survey research with missing data by novel artificial neural networks. Expert Syst Appl 39:10456–10464CrossRefGoogle Scholar
  79. Vihola M (2012) Robust adaptive Metropolis algorithm with coerced acceptance rate. Stat Comput 22:997–1008MathSciNetMATHCrossRefGoogle Scholar
  80. Wan S, Yang K, Zhou H (2012) Efficient bitstream extraction for scalable video based on simulated annealing. Concurr Comput Pract Exp 24:1223–1230CrossRefGoogle Scholar
  81. Wang C, Yang Y (2011) Robust face recognition from single training image per person via auto-associative memory neural network. In: Proceedings of the international conference on electrical and control engineering, Beijing, China, pp 4947–4950Google Scholar
  82. Weinberger E (1990) Correlated and uncorrelated fitness landscapes and how to tell the difference. Biol Cybern 63:325–336MATHCrossRefGoogle Scholar
  83. Zhou L, Liang Y, Pan Q (2012) System error registration based on Metropolis-Hastings sampling. Syst Eng Electron 34:433–438Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Tshilidzi Marwala
    • 1
  1. 1.Faculty of Engineering and the Built EnvironmentUniversity of JohannesburgJohannesburgSouth Africa

Personalised recommendations