Skip to main content
SpringerLink
Log in

A Multiple Imputation Framework for Massive Multivariate Data of Different Variable Types: A Monte-Carlo Technique

  • Chapter
  • First Online:
Monte-Carlo Simulation-Based Statistical Modeling

Part of the book series: ICSA Book Series in Statistics ((ICSABSS))

  • 3839 Accesses

Abstract

The purpose of this chapter is to build theoretical, algorithmic, and implementation-based components of a unified, general-purpose multiple imputation framework for intensive multivariate data sets that are collected via increasingly popular real-time data capture methods . Such data typically include all major types of variables that are incomplete due to planned missingness designs, which have been developed to reduce respondent burden and lower the cost associated with data collection. The imputation approach presented herein complements the methods available for incomplete data analysis via richer and more flexible modeling procedures, and can easily generalize to a variety of research areas that involve internet studies and processes that are designed to collect continuous streams of real-time data. Planned missingness designs are highly useful and will likely increase in popularity in the future. For this reason, the proposed multiple imputation framework represents an important and refined addition to the existing methods, and has potential to advance scientific knowledge and research in a meaningful way. Capability of accommodating many incomplete variables of different distributional nature, types, and dependence structures could be a contributing factor for better comprehending the operational characteristics of today’s massive data trends. It offers promising potential for building enhanced statistical computing infrastructure for education and research in the sense of providing principled, useful, general, and flexible set of computational tools for handling incomplete data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Amatya, A., & Demirtas, H. (2015a). Simultaneous generation of multivariate mixed data with Poisson and normal marginals. Journal of Statistical Computation and Simulation, 85, 3129–3139.

    Article  MathSciNet  Google Scholar 

  • Amatya, A., & Demirtas, H. (2015b). Concurrent generation of ordinal and normal data with given correlation matrix and marginal distributions, R package OrdNor. http://CRAN.R-project.org/package=OrdNor.

  • Amatya, A., & Demirtas, H. (2016a). Simultaneous generation of multivariate binary and normal variates, R package BinNor. http://CRAN.R-project.org/package=BinNor.

  • Amatya, A., & Demirtas, H. (2016b). Generation of multivariate ordinal variates, R package MultiOrd. http://CRAN.R-project.org/package=MultiOrd.

  • Amatya, A., & Demirtas, H. (2016c). Simultaneous generation of multivariate data with Poisson and normal marginals, R package PoisNor. http://CRAN.R-project.org/package=PoisNor.

  • Demirtas, H. (2004). Simulation-driven inferences for multiply imputed longitudinal datasets. Statistica Neerlandica, 58, 466–482.

    Google Scholar 

  • Demirtas, H. (2005). Multiple imputation under Bayesianly smoothed pattern-mixture models for non-ignorable drop-out. Statistics in Medicine, 24, 2345–2363.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H. (2006). A method for multivariate ordinal data generation given marginal distributions and correlations. Journal of Statistical Computation and Simulation, 76, 1017–1025.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H. (2007). Practical advice on how to impute continuous data when the ultimate interest centers on dichotomized outcomes through pre-specified thresholds. Communications in Statistics-Simulation and Computation, 36, 871–889.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H. (2008). On imputing continuous data when the eventual interest pertains to ordinalized outcomes via threshold concept. Computational Statistics and Data Analysis, 52, 2261–2271.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H. (2009). Rounding strategies for multiply imputed binary data. Biometrical Journal, 51, 677–688.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H. (2010). A distance-based rounding strategy for post-imputation ordinal data. Journal of Applied Statistics, 37, 489–500.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H. (2014). Joint generation of binary and nonnormal continuous data. Journal of Biometrics and Biostatistics, 5, 1–9.

    Google Scholar 

  • Demirtas, H. (2016). A note on the relationship between the phi coefficient and the tetrachoric correlation under nonnormal underlying distributions. American Statistician, 70, 143–148.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H. (2017a). Concurrent generation of binary and nonnormal continuous data through fifth order power polynomials, Communications in Statistics—Simulation and Computation, 46, 344–357.

    Google Scholar 

  • Demirtas, H. (2017b). On accurate and precise generation of generalized Poisson variates, Communications in Statistics—Simulation and Computation, 46, 489–499.

    Google Scholar 

  • Demirtas, H., Ahmadian, R., Atis, S., Can, F. E., & Ercan, I. (2016a). A nonnormal look at polychoric correlations: Modeling the change in correlations before and after discretization. Computational Statistics, 31, 1385–1401.

    Article  MATH  Google Scholar 

  • Demirtas, H., Arguelles, L. M., Chung, H., & Hedeker, D. (2007). On the performance of bias-reduction techniques for variance estimation in approximate Bayesian bootstrap imputation. Computational Statistics and Data Analysis, 51, 4064–4068.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., & Doganay, B. (2012). Simultaneous generation of binary and normal data with specified marginal and association structures. Journal of Biopharmaceutical Statistics, 22, 223–236.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H., Freels, S. A., & Yucel, R. M. (2008). Plausibility of multivariate normality assumption when multiply imputing non-Gaussian continuous outcomes: A simulation assessment. Journal of Statistical Computation and Simulation, 78, 69–84.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2007). Gaussianization-based quasi-imputation and expansion strategies for incomplete correlated binary responses. Statistics in Medicine, 26, 782–799.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2008a). Multiple imputation under power polynomials. Communications in Statistics–Simulation and Computation, 37, 1682–1695.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2008b). An imputation strategy for incomplete longitudinal ordinal data. Statistics in Medicine, 27, 4086–4093.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2008c). Imputing continuous data under some non-Gaussian distributions. Statistica Neerlandica, 62, 193–205.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2011). A practical way for computing approximate lower and upper correlation bounds. The American Statistician, 65, 104–109.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., & Hedeker, D. (2016). Computing the point-biserial correlation under any underlying continuous distribution. Communications in Statistics- Simulation and Computation, 45, 2744–2751.

    Article  MathSciNet  MATH  Google Scholar 

  • Demirtas, H., Hedeker, D., & Mermelstein, J. M. (2012). Simulation of massive public health data by power polynomials. Statistics in Medicine, 31, 3337–3346.

    Article  MathSciNet  Google Scholar 

  • Demirtas, H., Hu, Y., & Allozi, R. (2016b). Data generation with Poisson, binary, ordinal and normal components, R package PoisBinOrdNor. https://cran.r-project.org/web/packages/PoisBinOrdNor.

  • Demirtas, H., Nordgren, R., & Allozi, R. (2016c). Generation of up to four different types of variables, R package PoisBinOrdNonNor. https://cran.r-project.org/web/packages/PoisBinOrdNonNor.

  • Demirtas, H., & Schafer, J. L. (2003). On the performance of random-coefficient pattern-mixture models for non-ignorable drop-out. Statistics in Medicine, 22, 2553–2575.

    Article  Google Scholar 

  • Demirtas, H., Shi, Y., & Allozi, R. (2016d). Simultaneous generation of count and continuous data, R package PoisNonNor. https://cran.r-project.org/web/packages/PoisNonNor.

  • Demirtas, H., Wang, Y., & Allozi, R. (2016e). Concurrent generation of binary, ordinal and continuous data, R package BinOrdNonNor. https://cran.r-project.org/web/packages/BinOrdNonNor.

  • Demirtas, H., & Yavuz, Y. (2015). Concurrent generation of ordinal and normal data. Journal of Biopharmaceutical Statistics, 25, 635–650.

    Article  Google Scholar 

  • Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood estimation from incomplete data via the EM algorithm. Journal of Royal Statistical Society-Series B, 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Emrich, J. L., & Piedmonte, M. R. (1991). A method for generating high-dimensional multivariate binary variates. The American Statistician, 45, 302–304.

    Google Scholar 

  • Ferrari, P. A., & Barbiero, A. (2012). Simulating ordinal data. Multivariate Behavioral Research, 47, 566–589.

    Article  Google Scholar 

  • Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532.

    Article  MATH  Google Scholar 

  • Headrick, T. C. (2002). Fast fifth-order polynomial transforms for generating univariate and multivariate nonnormal distributions. Computational Statistics and Data Analysis, 40, 685–711.

    Article  MathSciNet  MATH  Google Scholar 

  • Headrick, T. C. (2010). Statistical simulation: Power method polynomials and other transformations. Boca Raton, FL: Chapman and Hall/CRC.

    MATH  Google Scholar 

  • Hedeker, D., Demirtas, H., & Mermelstein, R. J. (2008). An application of a mixed-effects location scale model for analysis of ecological momentary mssessment (EMA) data. Biometrics, 6, 627–634.

    Article  MATH  Google Scholar 

  • Hedeker, D., Demirtas, H., & Mermelstein, R. J. (2009). A mixed ordinal location scale model for analysis of ecological momentary assessment (EMA) data. Statistics and Its Interface, 2, 391–402.

    Article  MathSciNet  MATH  Google Scholar 

  • Hedeker, D., Mermelstein, R. J., & Demirtas, H. (2012). Modeling between- and within-subject variance in ecological momentary assessment (EMA) data using mixed-effects location scale models. Statistics in Medicine, 31, 3328–3336.

    Article  MathSciNet  Google Scholar 

  • Higham, N. J. (2002). Computing the nearest correlation matrix—a problem from finance. IMA Journal of Numerical Analysis, 22, 329–343.

    Article  MathSciNet  MATH  Google Scholar 

  • Inan, G., & Demirtas, H. (2016a). Data generation with binary and continuous non-normal components, R package BinNonNor. https://cran.r-project.org/web/packages/BinNonNor.

  • Inan, G., & Demirtas, H. (2016b). Data generation with Poisson, binary and ordinal components, R package PoisBinOrd. https://cran.r-project.org/web/packages/PoisBinOrd.

  • Inan, G., & Demirtas, H. (2016c). Data generation with Poisson, binary and continuous components, R package PoisBinNonNor. https://cran.r-project.org/web/packages/PoisBinNonNor.

  • Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley.

    MATH  Google Scholar 

  • Nelsen, R. B. (2006). An introduction to copulas. Berlin, Germany: Springer.

    MATH  Google Scholar 

  • Qaqish, B. F. (2003). A family of multivariate binary distributions for simulating correlated binary variables with specified marginal means and correlations. Biometrika, 90, 455–463.

    Article  MathSciNet  MATH  Google Scholar 

  • Raghunathan, T. E., Lepkowski, J. M., van Hoewyk, J., & Solenberger, P. A. (2001). Multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology, 27, 85–95.

    Google Scholar 

  • Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592.

    Article  MathSciNet  MATH  Google Scholar 

  • Rubin, D. B. (2004). Multiple imputation for nonresponse in surveys (2nd ed.). New York, NY: Wiley.

    MATH  Google Scholar 

  • Schafer, J. L. (1997). Analysis of incomplete multivariate data. London, UK: Chapman and Hall.

    Google Scholar 

  • Tanner, M. A., & Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of American Statistical Association, 82, 528–540.

    Article  MathSciNet  MATH  Google Scholar 

  • Vale, C. D., & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465–471.

    Article  MATH  Google Scholar 

  • Van Buuren, S. (2012). Flexible imputation of missing data. Boca Raton, Florida: CRC Press.

    Book  MATH  Google Scholar 

  • Walls, T. A., & Schafer, J. L. (2006). Models for intensive longitudinal data. New York, NY: Oxford University Press.

    Book  MATH  Google Scholar 

  • Yahav, I., & Shmueli, G. (2012). On generating multivariate Poisson data in management science applications. Applied Stochastic Models in Business and Industry, 28, 91–102.

    Article  MathSciNet  MATH  Google Scholar 

  • Yucel, R. M., & Demirtas, H. (2010). Impact of non-normal random effects on inference by multiple imputation: A simulation assessment. Computational Statistics and Data Analysis, 54, 790–801.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Epidemiology and Biostatistics (MC923), University of Illinois at Chicago, 1603 West Taylor Street, Chicago, IL, 60612, USA

    Hakan Demirtas

Authors
  1. Hakan Demirtas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hakan Demirtas .

Editor information

Editors and Affiliations

  1. Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA

    Ding-Geng (Din) Chen

  2. Risk Management, Credit Suisse Risk Management, New York, New York, USA

    John Dean Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this chapter

Cite this chapter

Demirtas, H. (2017). A Multiple Imputation Framework for Massive Multivariate Data of Different Variable Types: A Monte-Carlo Technique. In: Chen, DG., Chen, J. (eds) Monte-Carlo Simulation-Based Statistical Modeling . ICSA Book Series in Statistics. Springer, Singapore. https://doi.org/10.1007/978-981-10-3307-0_8

Download citation

Publish with us

Policies and ethics

Search

Navigation