Advertisement

An Information Quality (InfoQ) Framework for Ex-Ante and Ex-Post Evaluation of Empirical Studies

  • Galit Shmueli
  • Ron Kenett
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)

Abstract

Numbers are not data and data analysis does not necessarily produce information and knowledge. Statistics, data mining, and artificial intelligence are disciplines focused on extracting knowledge from data. They provide tools for testing hypotheses, predicting new observations, quantifying population effects, and summarizing data efficiently. In these fields, measurable data is used to derive knowledge. However, a clean, exact and complete dataset, which is analyzed professionally, might contain no useful information for the problem under investigation. The term Information Quality (InfoQ) was coined by Ref. [15] as the potential of a dataset to achieve a specific (scientific or practical) goal using a given data analysis method. InfoQ is a function of goal, data, data analysis, and utility. Eight dimensions that relate to these components help assess InfoQ: Data Resolution, Data Structure, Data Integration, Temporal Relevance, Generalizability, Chronology of Data and Goal, Construct Operationalization, and Communication. The eight dimensions can be used for developing streamlined evaluation metrics of InfoQ. We describe two studies where InfoQ was integrated into research methods courses, guiding students in evaluating InfoQ of prospective and retrospective studies. The results and feedback indicate the importance and usefulness of InfoQ and its eight dimensions for evaluating empirical studies.

Keywords

Data analytics Data mining Statistical modeling Study goal Empirical study evaluation quality 

Notes

Acknowledgments

We thank Professors Joel Greenhouse (Carnegie Mellon University), Shirley Coleman (Newcastle University), and Irena Ograjenek (University of Ljubljana) for their support of integrating InfoQ into graduate courses at CMU and University of Ljubljana, and helping assess its impact.

References

  1. 1.
    Angst CM, Agarwal R, Kuruzovich J (2008) Bid or buy? Individual shopping traits as predictors of strategic exit in on-line auctions. Int J Electron Commer 13:59–84CrossRefGoogle Scholar
  2. 2.
    Bapna R, Goes P, Gupta A, Jin Y (2004) User heterogeneity and its impact on electronic auction market design: an empirical exploration. MIS Quarterly, 28(1):21Google Scholar
  3. 3.
    Bapna R, Jank W, Shmueli G (2008) Price formation and its dynamics in online auctions. Decis Support Syst 44:641–656CrossRefGoogle Scholar
  4. 4.
    Berthold MR, Borgelt C, Hoppner F, Klawonn F (2010) Guide to intelligent data analysis. Springer, LondonzbMATHCrossRefGoogle Scholar
  5. 5.
    Borle S, Boatwright P, Kadane JB (2006) The timing of bid placement and extent of multiple bidding: an empirical investigation using eBay online auctions. Stat Sci 21:194–205MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Deming WE (1953) On the distinction between enumerative and analytic studies. J Am Stat Assoc 48:244–255CrossRefGoogle Scholar
  7. 7.
    Figini S, Kenett RS, Salini S (2010) Integrating operational and financial risk assessments. Qual Reliab Eng Int 26(8):887–897Google Scholar
  8. 8.
    Ghani R, Simmons H (2004) Predicting the end-price of online auctions. Pisa, ItalyGoogle Scholar
  9. 9.
    Giovanni E (2008) Understanding economic statistics. Technical reportGoogle Scholar
  10. 10.
    Godfrey AB (2008) Eye on data quality. Six Sigma Forum Magazine, pp 5–6Google Scholar
  11. 11.
    Hand DJ (2008) Statistics: a very short introduction. Oxford University Press, OxfordGoogle Scholar
  12. 12.
    Jank W, Shmueli G (2010) Modeling online auctions. Wiley, HobokenzbMATHCrossRefGoogle Scholar
  13. 13.
    Katkar R, Reiley DH (2006) Public versus secret reserve prices in eBay auctions: results from a pokémon field experiment. Advances in Econc Analysis and Policy, 6(2), Article 7, 1–23Google Scholar
  14. 14.
    Kenett RS, Coleman S, Ograjenšek I (2010) On quality research: an application of InfoQ to the Phd research process. In: Proceedings of the European network for business and industrial statistics (ENBIS) 10th annual conference on business and industrial statistics, Antwerp, Belgium, September 2010Google Scholar
  15. 15.
    Kenett RS, Shmueli G (2013) On information quality. J Roy Stat Soc Ser A, forthcomingGoogle Scholar
  16. 16.
    Kenett RS, Thyregod P (2006) Aspects of statistical consulting not taught by academia. Stat Neerl 60:396–412MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Lucking-Reiley D, Bryan D, Prasad N, Reeves D (2007) Pennies from eBay: the determinants of price in online auctions. J Ind Econ 55:223–233CrossRefGoogle Scholar
  18. 18.
    Mallows C (1998) The zeroth problem. Am Stat 52:1–9MathSciNetGoogle Scholar
  19. 19.
    Patzer GL (2005) Using secondary data in marketing research. Praeger, WestportGoogle Scholar
  20. 20.
    Russom P (2011) Big data analytics. Technical report, Q4Google Scholar
  21. 21.
    Shmueli G (2010) To explain or to predict? Stat Sci 25:289–310MathSciNetCrossRefGoogle Scholar
  22. 22.
    Shmueli G, Koppius OR (2011) Predictive analytics in information systems research. Manag Inf Syst Q 35:553–572Google Scholar
  23. 23.
    Tukey JW (1977) Exploratory data analysis. Addison Wesley, New YorkGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2013

Authors and Affiliations

  1. 1.Srini Raju Centre for IT and the Networked EconomyIndian School of BusinessHyderabadIndia
  2. 2.Rigsum Institute of IT and ManagementThimphuBhutan
  3. 3.KPA Ltd., Raanana Israel Dept of Statistics & Applied MathematicsUniversity of TorinoTurinItaly
  4. 4.Center for Finance and Risk EngineeringNYU-PolyBrooklynUSA

Personalised recommendations