Computational Economics

, Volume 35, Issue 4, pp 371–394 | Cite as

Should Economists Use Open Source Software for Doing Research?

Article

Abstract

We survey the literature on the accuracy of econometric software. We also assess the advantages of open source software from the point of view of reliability and discuss its potential in applied economics, which has now become fully dependent on computers. As a case study, we apply various accuracy tests on GNU Regression, Econometrics and Time-series Library (gretl) and demonstrate that the open source nature of the program made it possible to see the cause, facilitated a rapid fix, and enabled verifying the correction of a number of flaws that we uncovered. We also run the same tests on four widely-used proprietary econometric packages and observe the known accuracy errors that remained uncorrected for more than 5 years.

Keywords

Open source Econometric software Gretl Accuracy Software reliability 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Altman M., McDonald M. P. (2001) Choosing reliable scientific software. PS: Political Science and Politics 34: 681–687Google Scholar
  2. Altman M., McDonald M. P. (2003) Replication with attention to numerical accuracy. Political Analysis 11: 302–307CrossRefGoogle Scholar
  3. Altman M., Gill J., McDonald M. P. (2004) Numerical issues in statistical computing for the social scientist (1st ed.). Wiley, New JerseyGoogle Scholar
  4. Anderson R. G., Greene W. H., McCullough B. D., Vinod H. D. (2008) The role of data-code archives in the future of economic research. Journal of Economic Methodology 15: 99–119CrossRefGoogle Scholar
  5. Baiocchi G. (2007) Reproducible research in computational economics: Guidelines, integrated approaches, and open source software. Computational Economics 30: 19–40CrossRefGoogle Scholar
  6. Baiocchi G., Distaso W. (2003) GRETL: Econometric software for the GNU generation. Journal of Applied Econometrics 18: 105–110CrossRefGoogle Scholar
  7. Bankhofer U., Hilbert A. (1997) Statistical software packages for windows—a market survey. Statistical Papers 38: 377–471CrossRefGoogle Scholar
  8. Boehm B. W. (1981) Software engineering economics (1st ed.). Prentice Hall, New JerseyGoogle Scholar
  9. Boehm B. W., Abts C., Brown A. W., Chulani S., Clark B. K., Horowitz E., Madachy R., Reifer D. J., Steece B. (2000) Software cost estimation with Cocomo II (1st ed.). Prentice Hall, New JerseyGoogle Scholar
  10. Bonaccorsi A., Giannangeli S., Rossi C. (2006) Entry strategies under competing standards: Hybrid business models in the open source software industry. Management Science 52: 1085–1098CrossRefGoogle Scholar
  11. Brooks C., Burke S. P., Persand G. (2001) Benchmarks and the accuracy of GARCH model estimation. International Journal of Forecasting 17: 45–56CrossRefGoogle Scholar
  12. Brooks C., Burke S. P., Persand G. (2003) Multivariate GARCH models: Software choice and estimation issues. Journal of Applied Econometrics 18: 725–734CrossRefGoogle Scholar
  13. Bruno G., Bonis R. D. (2004) A comparative study of alternative econometric packages with an application to Italian deposit interest rates. Journal of Economic and Social Measurement 29: 271–295Google Scholar
  14. Choi H. S., Kiefer N. M. (2005) Software evaluation: EasyReg international. International Journal of Forecasting 21: 609–616CrossRefGoogle Scholar
  15. Clary E. G., Ridge R. D., Stukas A. A., Snyder M., Copeland J., Haugen J., Miene P. (1998) Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology 74: 1516–1530CrossRefGoogle Scholar
  16. Cryer J. D., Chan K. S. (2008) Time series analysis: With applications in R (2nd ed.). Springer, New York, NYGoogle Scholar
  17. Haefliger S., von Krogh G., Spaeth S. (2008) Code reuse in open source software. Management Science 54: 180–195CrossRefGoogle Scholar
  18. Heiser, D. A. (2006). Microsoft Excel 2000 and 2003 faults, problems, workarounds and fixes. Computational Statistics and Data Analysis, 51, 1442–1443. http://www.daheiser.info/excel/frontpage.html.
  19. Hertel G., Niedner S., Herrmann S. (2003) Motivation of software developers in open source projects: An internet-based survey of contributors to the Linux kernel. Research Policy 32: 1159–1177CrossRefGoogle Scholar
  20. Johnson J. P. (2006) Collaboration, peer review and open source software. Information Economics and Policy 18: 477–497CrossRefGoogle Scholar
  21. Keeling K. B., Pavur R. J. (2007) A comparative study of the reliability of nine statistical software packages. Computational Statistics and Data Analysis 51: 3811–3831CrossRefGoogle Scholar
  22. Kitchen A. M., Drachenberg R., Symanzik J. (2003) Assessing the reliability of web-based statistical software. Computational Statistics 18: 107–122Google Scholar
  23. Kleiber C., Zeileis A. (2008) Applied Econometrics with R (1st ed.). Springer, New York, NYGoogle Scholar
  24. Knüsel L. (1995) On the accuracy of the statistical distributions in GAUSS. Computational Statistics and Data Analysis 20: 699–702CrossRefGoogle Scholar
  25. Knüsel L. (1996) Telegrams. Computational Statistics and Data Analysis 21: 116Google Scholar
  26. Knüsel L (1998) On the accuracy of statistical distributions in Microsoft Excel 97. Computational Statistics and Data Analysis 26: 375–377CrossRefGoogle Scholar
  27. Knüsel L. (2002) On the reliability of Microsoft Excel XP for statistical purposes. Computational Statistics and Data Analysis 39: 109–110Google Scholar
  28. Knüsel L. (2005) On the accuracy of statistical distributions in Microsoft Excel 2003. Computational Statistics and Data Analysis 48: 445–449CrossRefGoogle Scholar
  29. Koch, A., & Haag, U. (1995). The statistical software guide ‘94/95. A. Koch & U. Haag (Eds.). Computational Statistics and Data Analysis, 19, 237–261.Google Scholar
  30. Lakhani K. R., von Hippel E. (2003) How open source software works: “Free” user-to-user assistance. Research Policy 32: 923–943CrossRefGoogle Scholar
  31. Lerner J., Tirole J. (2001) The open source movement: Key research questions. European Economic Review 45: 819–826CrossRefGoogle Scholar
  32. Lerner J., Tirole J. (2002) Some simple economics of open source. Journal of Industrial Economics 50: 197–234CrossRefGoogle Scholar
  33. Lerner J., Tirole J. (2005a) The economics of technology sharing: Open source and beyond. Journal of Economic Perspectives 19: 99–120CrossRefGoogle Scholar
  34. Lerner J., Tirole J. (2005b) The scope of open source licensing. Journal of Law Economics and Organization 21: 20–56CrossRefGoogle Scholar
  35. Lerner J., Pathak P. A., Tirole J. (2006) The dynamics of open-source contributors. American Economic Review 96: 114–118CrossRefGoogle Scholar
  36. Lovell M. C., Selover D. D. (1994) Econometric software accidents. The Economic Journal 104: 713–725CrossRefGoogle Scholar
  37. Lucchetti, R. (2009). Who uses gretl? An analysis of the SourceForge download data. In Econometrics with gretl. Proceedings of the gretl conference 2009, Bilbao, Spain (pp. 45–55).Google Scholar
  38. Maurer S. M., Scotchmer S. (2006) Open source software: The new IP paradigm. In: Hendershott T. (eds) Handbook of economics and information systems. Elsevier, Amsterdam, pp 285–319Google Scholar
  39. McCullough B. D. (1998) Assessing the reliability of statistical software: Part I. American Statistician 52: 358–366CrossRefGoogle Scholar
  40. McCullough B. D. (1999a) Assessing the reliability of statistical software: Part II. American Statistician 53: 149–159CrossRefGoogle Scholar
  41. McCullough B. D. (1999b) Econometric software reliability: Eviews, LIMDEP, CHASM and TSP. Journal of Applied Econometrics 14: 191–202CrossRefGoogle Scholar
  42. McCullough B. D. (2000) Is it safe to assume that software is accurate. International Journal of Forecasting 16: 349–357CrossRefGoogle Scholar
  43. McCullough, B. D. (2001). Does Microsoft fix errors in Excel? In Proceedings of the 2001 joint statistical meetings. Alexandria, VA: American Statistical Association.Google Scholar
  44. McCullough, B. D. (2004a). Fixing statistical errors in spreadsheet software: The cases of Gnumeric and Excel. CSDA Statistical Software Newsletter. Retrieved December 10, 2008, from http://www.csdassn.org/software_reports/gnumeric.pdf.
  45. McCullough B. D. (2004b) Wilkinson’s tests and econometric software. Journal of Economic and Social Measurement 29: 261–270Google Scholar
  46. McCullough B. D. (2008) Microsoft Excel’s ‘not the Wichmann–Hill’ random number generators. Computational Statistics and Data Analysis 52: 4587–4593CrossRefGoogle Scholar
  47. McCullough B. D. (2009) Testing econometric software. In: Mills T. C., Patterson K. (eds) Handbook of econometrics. Pargrave, New York, pp 1293–1320Google Scholar
  48. McCullough B. D., Heiser D. A. (2008) On the accuracy of statistical procedures in Microsoft Excel 2007. Computational Statistics and Data Analysis 52: 4570–4578CrossRefGoogle Scholar
  49. McCullough B. D., Renfro C. G. (1998) Benchmarks and software standards: A case study of GARCH procedures. Journal of Economic and Social Measurement 25: 59–71Google Scholar
  50. McCullough B. D., Vinod H. D. (1999) The numerical reliability of econometric software. Journal of Economic Literature 37: 633–655Google Scholar
  51. McCullough B. D., Vinod H. D. (2003a) Comment: Econometrics and software. Journal of Economic Perspectives 17: 223–224CrossRefGoogle Scholar
  52. McCullough B. D., Vinod H. D. (2003b) Verifying the solution from a nonlinear solver: A case study. The American Economic Review 93: 873–892CrossRefGoogle Scholar
  53. McCullough B. D., Wilson B. (1999) On the accuracy of statistical procedures in Microsoft EXCEL 97. Computational Statistics and Data Analysis 31: 27–37CrossRefGoogle Scholar
  54. McCullough B. D., Wilson B. (2002) On the accuracy of statistical procedures in Microsoft Excel 2000 and Excel XP. Computational Statistics and Data Analysis 40: 713–721CrossRefGoogle Scholar
  55. McCullough B. D., Wilson B. (2005) On the accuracy of statistical procedures in Microsoft Excel 2003. Computational Statistics and Data Analysis 49: 1244–1252CrossRefGoogle Scholar
  56. McKenzie C. R., Takaoka S. (2002) 2002: A LIMDEP odyssey. Journal of Applied Econometrics 18: 241–247CrossRefGoogle Scholar
  57. McKenzie C. R., Takaoka S. (2007) Eviews 5.1. Journal of Applied Econometrics 22: 1145–1152CrossRefGoogle Scholar
  58. Mixon J. W., Smith R. J. (2006) Teaching undergraduate econometrics with GRETL. Journal of Applied Econometrics 21: 1103–1107CrossRefGoogle Scholar
  59. Nerlove M. (2004) Programming languages: A short history for economists. Journal of Economic and Social Measurement 29: 189–203Google Scholar
  60. Newbold P., Agiakloglou C., Miller J. (1994) Adventures with ARIMA software. International Journal of Forecasting 10: 573–581CrossRefGoogle Scholar
  61. Renfro C. G. (2004a) A compendium of existing econometric software packages. Journal of Economic and Social Measurement 29: 359–409Google Scholar
  62. Renfro C. G. (2004b) Econometric software: The first fifty years in perspective. Journal of Economic and Social Measurement 29: 9–107Google Scholar
  63. Roberts J. A., Hann I. H., Slaughter S. A. (2006) Understanding the motivations, participation and performance of open source software developers: A longitudinal study of the Apache projects. Management Science 52: 984–999CrossRefGoogle Scholar
  64. Rosenblad, A. (2008). Gretl 1.7.3. Journal of Statistical Software, 25, 19. http://www.jstatsoft.org/v25/s01.
  65. Sawitzki G. (1994a) Report on the numerical reliability of data analysis systems. Computational Statistics and Data Analysis 18: 289–301CrossRefGoogle Scholar
  66. Sawitzki G. (1994b) Testing numerical reliability of data analysis systems. Computational Statistics and Data Analysis 18: 269–286CrossRefGoogle Scholar
  67. Schwarz, M., & Takhteyev, Y. (to appear). Half a century of public software institutions: Open source as a solution to the hold-up problem. Journal of Public Economic Theory.Google Scholar
  68. Silk J. (1996) Systems estimation: A comparison of SAS, Shazam and TSP. Journal of Applied Econometrics 11: 437–450CrossRefGoogle Scholar
  69. Stokes H. H. (2004) On the advantage of using two or more econometric software systems to solve the same problem. Journal of Economic and Social Measurement 29: 307–320Google Scholar
  70. Vinod H. D. (2000) Review of GAUSS for Windows, including its numerical accuracy. Journal of Applied Econometrics 15: 211–220CrossRefGoogle Scholar
  71. Vinod H. D. (2001) Care and feeding of reproducible econometrics. Journal of Econometrics 100: 87–88CrossRefGoogle Scholar
  72. Vinod H. D. (2008) Hands-on intermediate econometrics using R (1st ed.). World Scientific Publishing Company, New JerseyGoogle Scholar
  73. von Krogh G., Spaeth S. (2007) The open source software phenomenon: Characteristics that promote research. Journal of Strategic Information Systems 16: 236–253CrossRefGoogle Scholar
  74. von Krogh G., Spaeth S., Lakhani K. R. (2003) Community, joining, and specialization in open source software innovation: A case study. Research Policy 32: 1217–1241CrossRefGoogle Scholar
  75. West J. (2003) How open is open enough? melding proprietary and open source platform strategies. Reseach Policy 32: 1258–1286Google Scholar
  76. Wichmann B. A., Hill I. D. (1982) Algorithm AS 183: An efficient and portable pseudo-random number generator. Applied Statistics 31: 188–190CrossRefGoogle Scholar
  77. Wilkinson L. (1985) Statistics quiz: Problems which reveal deficiencies in statistical programs (1st ed.). SYSTAT, Evanston, ILGoogle Scholar
  78. Wilkinson, L. (1994). Practical guidelines for testing statistical software. In P. Dirschedl & R. Ostermann (Eds.), Computational statistics, 25th conference on statistical computing at Schloss Reisenburg. Physica, Verlag.Google Scholar
  79. Yalta A. T. (2007) The numerical reliability of GAUSS 8.0. The American Statistician 61: 262–268CrossRefGoogle Scholar
  80. Yalta A. T. (2008) The accuracy of statistical distributions in Microsoft ® Excel 2007. Computational Statistics and Data Analysis 52: 4579–4586CrossRefGoogle Scholar
  81. Yalta A. T., Jenal O. (2009) On the importance of verifying forecasting results. International Journal of Forecasting 25: 62–73CrossRefGoogle Scholar
  82. Yalta A. T., Lucchetti R. (2008) The GNU/Linux platform and freedom respecting software for economists. Journal of Applied Econometrics 23: 279–286CrossRefGoogle Scholar
  83. Yalta A. T., Yalta A. Y. (2007) GRETL 1.6.0 and its numerical accuracy. Journal of Applied Econometrics 22: 849–854CrossRefGoogle Scholar
  84. Zeileis A., Kleiber C. (2005) Validating multiple structural change models—a case study. Journal of Applied Econometrics 20: 685–690CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC. 2010

Authors and Affiliations

  1. 1.Department of EconomicsTOBB University of Economics and TechnologyAnkaraTurkey
  2. 2.Department of EconomicsHacettepe UniversityAnkaraTurkey

Personalised recommendations