Skip to main content
SpringerLink
Log in

From Scientific Research to Practical Implementations: Applications to Improve Data Quality in Child Welfare

  • Published:
The Journal of Behavioral Health Services & Research Aims and scope Submit manuscript

Abstract

Child welfare decisions have life-impacting consequences which, often times, are underpinned by limited or inadequate data and poor quality. Though research of data quality has gained popularity and made advancements in various practical areas, it has not made significant inroads for child welfare fields or data systems. Poor data quality can hinder service decision-making, impacting child behavioral health and well-being as well as increasing unnecessary expenditure of time and resources. Poor data quality can also undermine the validity of research and slow policymaking processes. The purpose of this commentary is to summarize the data quality research base in other fields, describe obstacles and uniqueness to improve data quality in child welfare, and propose necessary steps to scientific research and practical implementation that enables researchers and practitioners to improve the quality of child welfare services based on the enhanced quality of data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The current manuscript does not contain any data or data analysis.

Notes

  1. The evaluation process indicates the whole procedure includes defining data quality within a specific domain, establishing operational definitions for data quality dimensions, and assigning quantitative values to each record or individual based on these operational definitions. The details will be shown in the section “Strategies to improve data quality.”.

  2. Users are the people who use data to achieve their intended goals. Specifically, in child welfare, they are the professionals and practitioners who require data to investigate child welfare outcomes and evaluate the effectiveness of provided services. They are the primary end-users of the data and should play a pivotal role in determining the data “fitness for use.”.

  3. Within an organization, a single dimension of data quality means using only one aspect of predefined data quality dimensions to specify multiple data fields or variables. For instance, in evaluating demographic information such as race, gender, age, and ethnicity, the application of the single dimension of completeness to assess whether the inputted data values meet the desired quality standards would not be enough, as completeness will not provide the quality of accuracy or data consistency. For anyone who is not familiar with the term of dimensions in data quality, it sometimes can be easily paralleled to the term of measurement construct or domain in social and behavioral science.

References

  1. Comprehensive child welfare information system (CCWIS) technical bulletin #6: CCWIS data quality plan. Washington, DC: Children’s Bureau. Available at https://www.acf.hhs.gov/cb/training-technical-assistance/ccwis-technical-bulletin-6. Accessed 30 November, 2022.

  2. Data and Samples. NSF-funded National Ecological Observatory Network. Available at https://www.neonscience.org/. Accessed 30 November, 2022.

  3. Data release 18. Sloan Digital Sky Survey(SDSS). Available at https://www.sdss.org/dr18/. Accessed 30 November, 2022.

  4. Wang Z, Talburt JR, Wu N,et al. A rule-based data quality assessment system for electronic health record data. Applied Clinical Informatics 2020;11(04):622–634. Available at https://doi.org/10.1055/s-0040-1715567. Accessed 2 February, 2023.

  5. Behavioral health & wellness. Washington, DC: Child Welfare Information Gateway. Available at https://www.childwelfare.gov/topics/systemwide/bhw. Accessed 2 February, 2023.

  6. Tayi GK, Ballou DP. Examining data quality. Communications of the Association for Computing Machinery 1998;41(2):54–57. Available at https://doi.org/10.1145/269012.269021. Accessed 2 February, 2023.

  7. Olson JE. Data Quality: The Accuracy Dimension. Morgan Kaufmann, San Francisco: Elsevier, 2003. Available at https://doi.org/10.1016/B978-1-55860-891-7.X5000-8. Accessed 2 February, 2023.

  8. Parssian A, Sarkar S, Jacob VS. Assessing data quality for information products: impact of selection, projection, and cartesian product. Management Science 2004;50(7):967–982. Available at https://doi.org/10.1287/mnsc.1040.0237. Accessed 2 February, 2023.

  9. Heinrich B, Klier M, Kaiser MA. Procedure to develop metrics for currency and its application in CRM. Journal of Data and Information Quality (JDIQ) 2009;1(1):1–28. Available at https://doi.org/10.1145/1515693.1515697. Accessed 2 February, 2023.

  10. Watts S, Shankaranarayanan G, Even A. Data quality assessment in context: a cognitive perspective. Decision Support Systems 2009;48(1):202–211. Available at https://doi.org/10.1016/j.dss.2009.07.012. Accessed 2 February, 2023.

  11. Lederman R, Shanks G. Gibbs MR. Meeting Privacy Obligations: The Implications for Information Systems Development. Paper presented at the 11th European Conference on Information Systems (ECIS). Naples, Italy. June, 2003.

  12. Huang J, Liu M, Bowling N. Insufficient effort responding: examining an insidious confound in survey data. Journal of Applied Psychology 2014; 100(3): 828–845. Available at https://doi.org/10.1037/a0038510. Accessed 1 February, 2023.

  13. Arias VB, Garrido LE, Jenaro C,et al. A little garbage in, lots of garbage out: assessing the impact of careless responding in personality survey data. Behavior Research Methods 2020;52(6):2489–2505. Available at https://doi.org/10.3758/s13428-020-01401-8. 1 March, 2023.

  14. Eppler M, Helfert M. A classification and analysis of data quality costs. Paper presented at the Ninth International Conference on Information Quality, Cambridge, MA, February, 2004.

  15. Haug A, Zachariassen F, Liempd D. The costs of poor data quality. Journal of Industrial Engineering and Management 2011;4:168–193. Available at https://doi.org/10.3926/jiem.v4n2.p168-193. Accessed 1 February, 2023.

  16. Kahn MG, Callahan TJ, Barnard J, et al. A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data. The Journal for Electronic Health Data and Methods 2016;4(1):1244. Available at https://doi.org/10.13063/2327-9214.1244. Accessed 2 February, 2023.

  17. Redman TC. Data Quality: The Field Guide. Boston, MA: Digital Press, 2001. Available at https://dl.acm.org/doi/book/https://doi.org/10.5555/362427. Accessed 1 March, 2022.

  18. Kahn BK, Strong DM, Wang RY. Information quality benchmarks: product and service performance. Communications of the Association for Computing Machinery 2002;45(4):184–192. Available at https://doi.org/10.1145/505248.506007. Accessed 2 February, 2023.

  19. Leo L, Pipino L, Yang W, et al. Data quality assessment. Communications of the Association for Computing Machinery 2002;45(4):211. Available at https://doi.org/10.1145/505248.506010. Accessed 15 February, 2023.

  20. Hänel T, Felden C. Applying operational business intelligence in production environments. Paper presented at the 25th International Conference on Information Systems Development. Katowice, Poland, August, 2016.

  21. BIDQI: The Business Impacts of Data Quality Interdependencies Model. Utrecht, Netherlands: Utrecht University. Available at http://www.cs.uu.nl/research/techreps/repo/CS-2019/2019-001.pdf. Accessed 22 September, 2022.

  22. Choi TM, Luo S. Data quality challenges for sustainable fashion supply chain operations in emerging markets: roles of blockchain, government sponsors and environment taxes. Transportation Research Part E: Logistics and Transportation Review 2019;131:139-152. https://doi.org/10.1016/j.tre.2019.09.019. Accessed 22 September, 2022.

    Article  Google Scholar 

  23. Pezoulas VC, Kourou KD, Kalatzis F, et al. Medical data quality assessment: on the development of an automated framework for medical data curation. Computers in Biology and Medicine. 2019;107:270-283. https://doi.org/10.1016/j.compbiomed.2019.03.001. Accessed 22 January, 2023.

    Article  PubMed  Google Scholar 

  24. Terry AL, Stewart M, Cejic S, et al. A basic model for assessing primary health care electronic medical record data quality. BMC medical informatics and decision making 2019;19(1):30. https://doi.org/10.1186/s12911-019-0740-0. Accessed 22 January, 2023.

  25. Shepperd M. Data quality: cinderella at the software metrics ball? Paper presented at the 2nd International Workshop on Emerging Trends in Software Metrics. Honolulu HI. May 2011.

  26. Chen H. Measuring quality of data collection process to ensure data quality for public health information systems. Wollongong, Australia: University of Wollongong, 2020. Available at https://ro.uow.edu.au/theses1/994. Accessed 18 December, 2022.

  27. Chen H, Yu P, Hailey D, et al. Identification of the essential components of quality in the data collection process for public health information systems. Health Informatics Journal 2020;26(1):664-682. https://doi.org/10.1177/1460458219848622. Accessed 22 September, 2022

    Article  PubMed  Google Scholar 

  28. Shirai Y, Nichols W, Kasunic M. Initial evaluation of data quality in a TSP software engineering project data repository. Paper presented at the 2014 International Conference on Software and System Process. Nanjing, China. May, 2014.

  29. Wand Y, Wang RY. Anchoring data quality dimensions in ontological foundations. Communications of the Association for Computing Machinery 1996;39(11):86–95. https://doi.org/10.1145/240455.240479. Accessed 22 January,2023.

  30. Schmidt CO, Struckmann S, Enzenbach C, et al. Facilitating harmonized data quality assessments. A data quality framework for observational health research data collections with software implementations in R. BMC Medical Research Methodology 2021;21(1):63. https://doi.org/10.1186/s12874-021-01252-7. Accessed 22 January, 2023.

  31. Haug A. Understanding the differences across data quality classifications: a literature review and guidelines for future research. Industrial Management & Data Systems 2021;121(12):2651–2671. https://doi.org/10.1108/IMDS-12-2020-0756. Accessed 22 September, 2022.

  32. Forsgren N, Durcikova A, Clay PF, et al. The integrated user satisfaction model: assessing information quality and system quality as second-order constructs in system administration. Communications of the Association for Information Systems 2016;38:803–839. Available at http://aisel.aisnet.org/cais/vol38/iss1/39. Accessed 22 September, 2022.

  33. Zhang R, Indulska M, Sadiq S. Discovering data quality problems: the case of repurposed data. Business & Information Systems Engineering 2019;61(5):575–593. Available at https://doi.org/10.1007/s12599-019-00608-0. Accessed 22 September, 2022.

  34. Hassenstein MJ, Vanella P. Data quality—concepts and problems. Encyclopedia 2022;2(1):498–510. Available at https://doi.org/10.3390/encyclopedia2010032. Accessed 2 January, 2023.

  35. Heinrich B, Hristova D, Klier M, et al. Requirements for data quality metrics. Journal of Data and Information Quality 2017;9(2):1–32. Available at https://doi.org/10.1145/3148238.Accessed 22 January, 2023.

  36. Ehrlinger L, Werth B, Wöß W. Automated Continuous Data Quality Measurement with QuaIIe. International Journal of Advanced Software Engineering (IJASE) 2018;11:400–417. Available at http://www.iariajournals.org/software/soft_v11_n34_2018_paged.pdf. Accessed 22 September, 2022.

  37. Cappiello C, Comuzzi M. A utility-based model to define the optimal data quality level in IT service offerings. Paper presented at the 17th European Conference on Information Systems (ECIS), Verona, Italy. June, 2009.

  38. Bose R. Advanced analytics: opportunities and challenges. Industrial Management & Data Systems 2009;109(2):155–172. Available at https://doi.org/10.1108/02635570910930073. Accessed 22 January, 2023.

  39. Warth J, Kaiser G, Kügler M. The impact of data quality and analytical capabilities on planning performance: insights from the automotive industry. Paper presented at the Proceedings of the 10th International Conference on Wirtschaftsinformatik, Zurich, Switzerland, February 2011.

  40. Redman TC. Data Quality: Management and Technology. New York, NY: Bantam Books, 1992. Available at https://dl.acm.org/doi/https://doi.org/10.5555/133848. Accessed 22 September, 2022.

  41. Even A, Shankaranarayanan G. Utility-driven assessment of data quality. The Data Base for Advances in Information Systems 2007;38(2):75–93. Available at https://doi.org/10.1145/1240616.1240623. Accessed 30 January, 2023.

  42. Jones-Farmer LA, Ezell JD, Hazen BT. Applying control chart methods to enhance data quality. Technometrics 2014;56(1):29–41. Available at https://doi.org/10.1080/00401706.2013.804437. Accessed 30 January, 2023.

  43. Rousidis D, Sicilia MÁ, Garoufallou E, et al. Data quality issues and content analysis for research data repositories: the case of dryad. Paper presented at the 18th International Conference on Electronic Publishing, Thessaloniki, Greece. June, 2014

  44. Huser V, Kahn MG, Brown JS,et al. Methods for examining data quality in healthcare integrated data repositories. Biocomputing 2018 2018; 628–633. Available at https://doi.org/10.1142/9789813235533_0059. Accessed 30 January, 2023.

  45. Rajan NS, Gouripeddi R, Mo P, et al. Towards a content agnostic computable knowledge repository for data quality assessment. Computer Methods and Programs in Biomedicine 2019;177:193–201. Available at https://doi.org/10.1016/j.cmpb.2019.05.017. Accessed 30 January, 2023.

  46. Liaw ST, Guo JGN, Ansari S, et al. Quality assessment of real-world data repositories across the data life cycle: a literature review. Journal of the America Medical Informatics Association 2021;28(7):1591–1599. Available at https://doi.org/10.1093/jamia/ocaa340. Accessed 30 January, 2023.

  47. Timocin T. Data Quality in the Interface of Industrial Manufacturing and Machine Learning. Uppsala, Sweden: Uppsala University, 2020. Available at http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-419983. Accessed 1 January, 2023.

  48. Gupta N, Mujumdar S, Patel H, et al. Data quality for machine learning tasks. Paper presented at the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. Virtual Event, Singapore. August, 2021.

  49. Gupta N, Patel H, Afzal S, et al. Data quality toolkit: automatic assessment of data quality and remediation for machine learning datasets. ArXiv, 2021. Available at https://doi.org/10.48550/arXiv.2108.05935. Accessed March 1, 2023.

  50. Afriliana N, Król D, Gaol FL. Computational intelligence techniques for assessing data quality: towards knowledge-driven processing. Paper presented at the 21st International Conference on Computational Science. Krakow, Poland. June, 2021.

  51. Symoens SH, Aravindakshan SU, Vermeire FH, et al. QUANTIS: data quality assessment tool by clustering analysis. International Journal of Chemical Kinetics 2019;51(11):872–885. Available at https://doi.org/10.1002/kin.21316. Accessed 1 March, 2023.

  52. Corrales DC, Corrales JC, Ledezma A. How to address the data quality issues in regression models: a guided process for data cleaning. Symmetry. 2018;10(4):99. Available at https://doi.org/10.3390/sym10040099. Accessed 1 March, 2023.

  53. Ali TZ, Abdelaziz TM, Maatuk AM, et al. A framework for improving data quality in data warehouse: a case study. Paper presented at the 21st International Arab Conference on Information Technology (ACIT). Giza, Egypt. November, 2020.

  54. Improving Child and Family Services Through Integrated Data Systems. Baltimore, MD: The Annie E. Casey Foundation, 2019. Available at https://www.aecf.org/blog/improving-child-and-family-services-through-integrated-data-systems. Accessed 1 March, 2023.

  55. Child Welfare Information System (CCWIS) Data Quality Plans. Washington, D.C: Children’s Bureau. Available at https://www.acf.hhs.gov/cb/training-technical-assistance/ccwis-data-quality-plans. Accessed 1 April, 2023.

  56. Font SA, Maguire-Jack K. The scope, nature, and causes of child abuse and neglect. The Annals of the American Academy of Political and Social Science 2020;692(1):26–49. Available at https://doi.org/10.1177/0002716220969642. Accessed 1 April, 2023.

  57. Child maltreatment. Washington, DC: U.S. Department of Health and Human Services. Available at https://www.acf.hhs.gov/cb/report/child-maltreatment-2021. Accessed 1 April, 2023.

  58. National Child Abuse and Neglect Data System. Washington, DC: Children’s Bureau. Available at https://www.acf.hhs.gov/cb/data-research/ncands. Accessed 1 April, 2023.

  59. Brook J, McDonald T. The impact of parental substance abuse on the stability of family reunifications from foster care. Children and Youth Services Review 2009;31(2):193–198. Available at https://doi.org/10.1016/j.childyouth.2008.07.010. Accessed 1 April, 2023.

  60. Berger LM, Slack KS, Waldfogel J, et al. Caseworker-perceived caregiver substance abuse and child protective services outcomes. Child maltreatment 2010;15(3):199–210. Available at https://doi.org/10.1177/1077559510368305. Accessed 15 March, 2023.

  61. Keller S, Korkmaz G, Orr M, et al. The evolution of data quality: understanding the transdisciplinary origins of data quality concepts and approaches. Annual Review of Statistics and Its Application 2017;4(1):85–108. Available at https://doi.org/10.1146/annurev-statistics-060116-054114. Accessed 15 March, 2023.

  62. Redman TC. The impact of poor data quality on the typical enterprise. Communications of the Association for Computing Machinery 1998;41(2):79–82. Available at https://doi.org/10.1145/269012.269025. Accessed 15 March, 2023.

  63. Webster D, Putnam-Hornstein E, Needell B. Using data for child welfare system improvement: Lessons learned from the California Performance Indicators Project. Child Welfare 360: Child Welfare and Technology 2011:6. Available at https://cascw.umn.edu/wp-content/uploads/2013/12/CW360_2011.pdf. Accessed 1 April, 2023.

  64. Webster D, Needell B, Wildfire J. Data are your friends: Child welfare agency self-evaluation in Los Angeles county with the family to family initiative. Children and Youth Services Review 2002;24(6–7):471–484. Available at https://doi.org/10.1016/S0190-7409(02)00197-4. Accessed 1 April, 2023.

  65. Iezzoni LI. Assessing Quality Using Administrative Data. Annals of Internal Medicine 1997;127(8):666. Available at https://doi.org/10.7326/0003-4819-127-8_Part_2-199710151-00048. Accessed 1 April, 2023.

  66. Brownell MD, Jutte DP. Administrative data linkage as a tool for child maltreatment research. Child Abuse & Neglect 2013;37(2):120–124. Available at https://doi.org/10.1016/j.chiabu.2012.09.013. Accessed 1 April, 2023.

  67. Mor Barak ME, Levin A, Nissly JA, Lane CJ. Why do they leave? Modeling child welfare workers’ turnover intentions. Children and Youth Services Review 2006;28(5):548–577. Available at https://doi.org/10.1016/j.childyouth.2005.06.003. Accessed 1 April, 2023.

  68. Peckover S, Hall C, White S. From policy to practice: the implementation and negotiation of technologies in everyday child welfare. Children & Society 2009;23(2):136–148. Available at https://doi.org/10.1111/j.1099-0860.2008.00143.x. Accessed 1 April, 2023.

  69. Lee YW, Pipino LL, Funk JD, et al. Journey to Data Quality. Cambridge, MA: The MIT Press, 2006. Available at https://direct.mit.edu/books/book/2314/Journey-to-Data-Quality. Accessed 1 September, 2022.

  70. Vayghan JA, Garfinkle SM, Walenta C, et al. The internal information transformation of IBM. IBM Systems Journal 2007;46(4):669–683. Available at https://doi.org/10.1147/sj.464.0669. Accessed 15 March, 2023

  71. Wang RY, Strong DM. Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems 1996;12(4):5–33. Available at https://doi.org/10.1080/07421222.1996.11518099. Accessed 1 April, 2023

  72. The Comprehensive Child Welfare Information System Final Rule. Washington, DC: Children’s Bureau. Available at https://www.acf.hhs.gov/cb/training-technical-assistance/comprehensive-child-welfare-information-system-final-rule-overview. Accessed 1 April, 2023.

  73. Schelter S, Lange D, Schmidt P, et al. Automating large-scale data quality verification. Proceedings of the VLDB Endowment 2018;11(12):1781–1794. Available at https://doi.org/10.14778/3229863.3229867. Accessed 1 April, 2023.

  74. Neumaier S, Umbrich J, Polleres A. Automated quality assessment of metadata across open data portals. Journal of Data and Information Quality 2016;8(1):1–29. Available at https://doi.org/10.1145/2964909. Accessed 1 March, 2023.

  75. Ozonze O, Scott PJ, Hopgood AA. Automating electronic health record data quality assessment. Journal of Medical Systems 2023;47(1):23. Available at https://doi.org/10.1007/s10916-022-01892-2. 1 March, 2023.

  76. Maletic JI, Marcus A. Data Cleansing: Beyond Integrity Analysis. Paper presented at the 2000 Conference on Information Quality, Cambridge, Massachusetts, October, 2000.

  77. Child Welfare Services. Oklahoma City, OK: Oklahoma Human Services, 2023. Available at https://oklahoma.gov/okdhs/services/cws.html. Accessed 15 April, 2023

Download references

Funding

This research was supported by the Vice President for Research and Partnerships of the University of Oklahoma and the Data Institute for Societal Challenges (DISC) grant.

Author information

Authors and Affiliations

  1. University of Oklahoma Health Sciences Center, Oklahoma City, OK, USA

    Yutian T. Thompson Ph.D., Yaqi Li Ph.D. & Jane Silovsky Ph.D.

Authors
  1. Yutian T. Thompson Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaqi Li Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jane Silovsky Ph.D.
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yutian T. Thompson Ph.D. or Yaqi Li Ph.D..

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Thompson, Y.T., Li, Y. & Silovsky, J. From Scientific Research to Practical Implementations: Applications to Improve Data Quality in Child Welfare. J Behav Health Serv Res 51, 289–301 (2024). https://doi.org/10.1007/s11414-023-09875-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11414-023-09875-y

Search

Navigation