Abstract
Missing values and sparse data often challenge the reliability of statistical analysis in terms of biased parameter estimates and degraded confidence intervals, thereby leading to false inferences and suboptimal business decisions. To managers in the consumer data analytics field, the challenge faced by missing and limited data is nothing novel, and many powerful techniques of analysis and data management are available to them. However, the choice of adequate management practices is far from optimal. This chapter proposes an integrated approach by jointly treating the missing data and sparse data problems, using approximate Bayesian bootstrap (ABB) and Bayesian (HB) modeling. Therefore, the chapter addresses these two key challenges and corrects the bias formed, by extrapolating information from the sparse and missing data onto a large sample. The proposed method is illustrated by computation of price elasticity models for a leading consumer finance business on data that suffers from both missing and sparsity issues. The results presented illustrate the superiority of the model in taking better decisions in consumer data analytics. In contrast to the point estimate generated using traditional price elasticity models, the proposed model helps to make a better inference on the price elasticity estimates through a probability density function as it generates a distribution of price elasticity. Further expansion of the principle illustrated here will auger a powerful business optimization possibility and should be a fruitful area of future research.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
All these approaches assume that the variation in the estimates of the common parameter is due to sampling variation. This is the statistical justification of the Claycamp and Liddy approach; Lilien, Rao, and Kalish do the same thing by their restriction to “similar products.” In many cases, however, data to support this assumption are not available, or similar products, with respect to sales rates, are hard to identify. For example, the parameters of marketing effectiveness may be functions of product characteristics. Rao and Yamada (1988) have studied the situation when the parameters are functions of perceived product attributes; see also Srivastava et al. (1985), Sultan et al. (1996), steckel and Vanhonacker (1988) and Batra and Vanhonacker (1988) for methods of using past cases for forecasting the diffusion of new products.
- 2.
This is done using the “PROC BGENMOD” procedure in SAS. In the PROC BGENMOD analysis, if no prior is specified by the user, a flat prior distribution is assumed on the regression coefficients which reflects ignorance of the location of the parameter, placing equal likelihood on all possible values the regression coefficient can take.
- 3.
It is important to note that many diagnostic tools are designed to verify a necessary but not sufficient condition for convergence. There are no conclusive tests that can tell you when the Markov chain has converged to its stationary distribution. Also, it is important to check the convergence of all parameters, and not just those of interest, before proceeding to make an inference. With some models, certain parameters can appear to have very good convergence behavior, but that could be misleading due to the slow convergence of other parameters.
References
Afifi AA, Elashoff RM (1966) Missing observations in multivariate statistics I. J Am Stat Assoc 61(315):595–605
Allenby GM, Ginter JL (1995) Using extremes to design products and segment markets. J Market Res 32:392–403
Batra R, Vanhonacker WR (1988) Falsifying laboratory results through field tests: a time-series methodology and some results. J Bus Res 16(4):281–300
Claycamp HJ, Liddy LE (1969) Prediction of new product performance: an analytical approach. J Market Res 6:414–420
Dempster AP, Laird LM, Rubin DB (1977) J Royal Stat Soc Seri B (Methodological) 39(1):1–38
Efron B, Tibshirani RJ (1993) An Introduction to the Bootstrap. Chapman & Hall, CRC Monographs on Statistics & Applied Probability
Geweke J (1992) Evaluating the accuracy of sampling‐based approaches to the calculation of posterior moments. In: Bernardo JM, Berger JO, Dawid AP, Smith AFM (eds) Bayesian Statistics 4. pp 169–193. Oxford University Press, Oxford
Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. TT Addison-Wesley Publishing Company Inc.
Hartley HO, Hocking RR (1971) The analysis of incomplete data. Biometrics 27:783–808
Hedeker D, Gibbons RD (1997) Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychol Methods 2(1):64–78
Lenk PJ, Rao AG (1990) New models from old: forecasting product adoption by hierarchical bayes procedures. Market Sci 9(1):42–53
Lilien G, Rao A, Kalish S (1981) Bayesian estimation and control of detailing effort in a repeat purchase diffusion environment. Manage Sci 27:493–506
Little RJA (1982) Models for nonresponse in sample surveys. J Am Stat Assoc 77:237–250
Little RJA, Rubin DB (1987) Statistical analysis with missing data. John Wiley and Sons, New York
Mitchell M (1998) An introduction to genetic algorithms. MIT Press, Cambridge, Massachusetts
Neelamegham R, Chintagunta P (1999) A Bayesian model to forecast new product performance in domestic and international markets. Market Sci 18(2):115–136
Nie NH, Hadlai CH, Jean GJ, Karin S, Bent DH (1975) Statistical package for the social sciences, 2nd edn. McGraw-Hill, New York
Orchard T, Woodbury MA (1972) A missing information principle: theory and applications. Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, vol 1, pp 697–715
Rao AG, Yamada M (1988) Forecasting with a repeat purchase diffusion model. Manage Sci 34(6):734–752
Rosenbaum PR, Rubin DB (1983) Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J Royal Stat Soc: Ser B 45:212–218
Rosenbaum PR, Rubin DB (1984) Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc 79:516–524
Rosenbaum PR, Rubin DB (1985) Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am Stat 39:33–38
Roth PL (1994) Missing data: a conceptual review for applied psychologists. Pers Psychol 47(3):537–560
Rubin DB (1996) Multiple imputation after 18+ years. J Am stat Assoc 91(434):437–489
Schafer JL, Olsen MK (1998) Multiple imputation for multivariate missing data problems: a data analyst’s perspective. Multivar Behav Res 33:545–571
Spector L, Barnum H, Bernstein HJ (1998) Genetic programming for quantum computers. In: Koza JR, Banzhaf W, Chellapilla K, Deb K, Dorigo M, Fogel DB, Garzon MH, Goldberg DE, Iba H, Riolo RL (eds) Genetic Programming 1998: Proceedings of the Third Annual Conference, pp 365–374, Morgan Kaufmann
Srivastava RK, Mahajan V, Ramaswami SN, Cherian J (1985) A multi-attribute diffusion model for forecasting the adoption of investment alternatives for consumers. Technol Forecast Social Change 28(4):325–333
Steckel JH, Vanhonacker WR (1988) A heterogeneous conditional logit model of choice. J Bus Econ Stat 6(3):391–398
Sultan F, Farley JU, Lehmann DR (1996) Reflections on a meta-analysis of applications of diffusion models. J Market Res 247–249
Talukdar D, Sudhir K, Ainslie A (2002) Investigating new product diffusion across products and countries. Market Sci 21(1):97–114
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Bhaduri, S.N., Fogarty, D. (2016). Estimating Price Elasticity with Sparse Data: A Bayesian Approach. In: Advanced Business Analytics. Springer, Singapore. https://doi.org/10.1007/978-981-10-0727-9_9
Download citation
DOI: https://doi.org/10.1007/978-981-10-0727-9_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-0726-2
Online ISBN: 978-981-10-0727-9
eBook Packages: Business and ManagementBusiness and Management (R0)