Skip to main content
SpringerLink
Log in
Book cover

International Conference of Reliable Information and Communication Technology

IRICT 2018: Recent Trends in Data Science and Soft Computing pp 65–73Cite as

A Model for Addressing Quality Issues in Big Data

  • Conference paper
  • First Online:

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 843))

Abstract

Big Data (BD) is everywhere and quite a lot of benefits have been derived from its usage by different organizations. Notwithstanding, there are still numerous technical and research challenges that must be tackled to comprehend and gain its full potential. The major challenges of BD are not just its processing, storage and analytics, there are also challenges associated with it that run across the BD value chain such as the data collection phase, integration and the enforcement of quality. This paper propose a DQ transformation model to evaluate BD quality from the data collection phase through to the visualization phase involving both data-driven and process-driven quality evaluation by assessing the quality of data itself first then assessing the process quality. This is still an ongoing research and hopefully will be experimented using specific Data Quality Dimensions (DQDs) like completeness, consistency, accuracy and timeliness with process quality dimensions such as Throughput, response time, latency with their corresponding metrics.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Tee, J.: The Server Side (2013). http://www.theserverside.com/feature/Handling-the-four-Vof-big-data-volume-velocity-varietyand-Veracity

  2. Levitin, V., Redman, T.C.: Data as a resource: properties, implications, and prescriptions. Sloan Manag. Rev. 40, 89–101 (1998)

    Google Scholar 

  3. Izham Jaya, M., Sidi, F., Ishak, I., Suriani Affendey, L.I.L.L.Y., Jabar, M.A.: A review of data quality research in achieving high data quality within organization. J. Theor. Appl. Inform. Technol. 95(12), 2647–2657 (2017)

    Google Scholar 

  4. Hu, H., Wen, Y., Chua, T.-S., Li, X.: Toward scalable systems for Big Data analytics: a technology tutorial. IEEE Access 2, 652–687 (2014)

    Article  Google Scholar 

  5. Idi, F., Shariat Panahy, P. H., Affendey, L.S., Jabar, M.A.H., Ibrahim, H., Mustapha, A.: Data quality: a survey of data quality dimensions. In: 2012 International Conference on Information Retrieval Knowledge Management (CAMP), pp. 300–304 2(012)

    Google Scholar 

  6. Glowalla, P., Balazy, P., Basten, D., Sunyaev, A.: Process-driven data quality management-an application of the combined conceptual life cycle model. In: 47th Hawaii International Conference on System Sciences (HICSS), pp. 4700–4709 (2014)

    Google Scholar 

  7. Serhani, M.A., El Kassabi, H.T., Taleb, I., Nujum, A.: An hybrid approach to quality, evaluation across Big Data value chain. In: IEEE International Congress on Big Data (BigData Congress), pp. 418–425. IEEE. (2016)

    Google Scholar 

  8. Pääkkönen, P., Pakkala, D.: Reference architecture and classification of technologies, products and services for Big Data systems. Big Data Res. (2015). https://doi.org/10.1016/j.bdr.2015.01.001

    Article  Google Scholar 

  9. Maier, M., Serebrenik, A., Vanderfeesten, I.T.P.: Towards a Big Data Reference Architecture. University of Eindhoven, Eindhoven (2013)

    Google Scholar 

  10. Malik, P.: Governing Big Data: Principles and Practices. IBM J. Res. Dev. 57, 1–13 (2013)

    Article  Google Scholar 

  11. Soares, S.: Big Data Governance: An Emerging Imperative. MC Press, Boise (2012)

    Google Scholar 

  12. Feldman M.: The Big Data challenge: intelligent tiered storage at scale. White Paper (2013)

    Google Scholar 

  13. Strong, Y.W., Lee, Y.E., Wang, R.Y.: Data quality in context. Commun. ACM 40(5), 103–110 (1997)

    Article  Google Scholar 

  14. Wang, R.Y.: A product perspective on total data quality management. Commun. ACM 41(2), 58–65 (1998)

    Article  Google Scholar 

  15. Fürber, C., Hepp, M.: Using SPARQL and SPIN for data quality management on the semantic web. In: International Conference on Business Information Systems pp. 35–46 Springer, Berlin, Heidelberg (2010)

    Google Scholar 

  16. Sidi, F., Shariat Panahy, P.H., Affendey, L.S., Jabar, M.A., Ibrahim, H., Mustapha, A.: Data quality: a survey of data quality dimensions. In: International Conference on Information Retrieval Knowledge Management (CAMP) (2012)

    Google Scholar 

  17. Taleb, I., Dssouli, R., Serhani, M.A.: Big Data pre-processing: a quality framework. In: 2015 IEEE International Congress on Big Data (BigData Congress), pp. 191–198. IEEE (2015)

    Google Scholar 

  18. Hazen, B.T., Boone, C.A., Ezell, J.D., Jones-Farmer, L.A.: Data quality for data science, predictive analytics, and Big Data in supply chain management: an introduction to the problem and suggestions for research and applications. Int. J. Prod. Econ. 154, 72–80 (2014)

    Article  Google Scholar 

  19. Loshin, D.: Big Data Analytics: From Strategic Planning to Enterprise Integration with Tools, Techniques, NoSql, and Graph. Elsevier, Amsterdam (2013)

    Google Scholar 

  20. Rahm, E., Do, H.H.: Data cleaning: problems and current approaches. IEEE Data Eng. Bull. 23(4), 3–13 (2000)

    Google Scholar 

  21. Eckerson, W.W.: Data Quality and the Bottom Line: Achieving Business Success Through a Commitment to High-Quality Data. Data Warehousing Institute, Chatsworth (2002)

    Google Scholar 

  22. Fan, W., Geerts, F.: Foundations of Data Quality Management. Morgan & Claypool, San Rafael (2012)

    MATH  Google Scholar 

  23. Batini, C., Cappiello, C., Francalanci, C., Maurino, A.: Methodologies for data quality assessment and improvement. ACM Comput. Surv. 41(3), 1–52 (2009)

    Article  Google Scholar 

  24. McGilvray, D.: Executing Data Quality Projects: Ten Steps to Quality Data and Trusted Information. Morgan Kaufmann, Burlington (2008)

    Google Scholar 

  25. Jayawardene, V., Sadiq, S., Indulska, M.: An analysis of data quality dimensions, pp. 1–32 (2015)

    Google Scholar 

  26. Loshin, D.: The Practitioner’s Guide to Data Quality Improvement. Elsevier, Amsterdam Morgan Kaufmann OMG Press (2011)

    Google Scholar 

  27. Batini, C., Cappiello, C., Francalanc, C., Maurino, A.: Methodologies for data quality assessment and improvement. ACM Comput. Surv. (CSUR) 41(3), 16 (2009)

    Article  Google Scholar 

  28. Taleb, I., Dssouli, R., Serhani, M.A.: Big Data pre-processing: a quality framework. In: IEE International Congress on Big Data (2015)

    Google Scholar 

  29. Saha, B., Srivastava, D.: Data quality: the other face of Big Data. In: IEEE 30th International Conference on Data Engineering (ICDE), pp. 1294–1297 (2014)

    Google Scholar 

  30. Tang, N.: Big Data cleaning. In: Chen, L., Jia, Y., Sellis, T., Liu, G. (eds.) Web Technologies and Applications, pp. 13–24. Springer, Berlin (2014)

    Google Scholar 

  31. Introducing JSON. http://www.json.org/

  32. Understanding Metadata. NISO Press, Bethesda, MD, USA, (2004)

    Google Scholar 

  33. Oliveira, P., Rodrigues F., Henriques, P.R.: A formal definition of data quality problems. In: IQ (2005)

    Google Scholar 

  34. Glavic, B.: Big Data Provenance: Challenges and Implications for Benchmarking. In: Specifying Big Data Benchmarks, pp. 72–80 Springer, Berlin Heidelberg (2014)

    Google Scholar 

  35. Cheah, Y-W., Canon, R., Plale, B., Ramakrishnan, L.: Milieu: lightweight and configurable Big Data provenance for science. In: 2013 IEEE International Congress on Big Data (BigData Congress) pp. 46-53 (2013)

    Google Scholar 

  36. Ebaid, A., Elmagarmid, A., Ilyas, I.F., Ouzzani, M., Quiane-Ruiz, J.-A., Tang, N., Yin, S.: NADEEF: a generalized data cleaning system. Proc. VLDB Endow. 6(12), 1218–1221 (2013)

    Article  Google Scholar 

  37. Recuero, A.G., Esteves, S., Veiga, L.: Towards quality-of-service driven consistency for Big Data management. Int. J. Big Data Intell. 1(1/2), 74 (2014)

    Article  Google Scholar 

  38. Juddoo, S.: Overview of data quality challenges in the context of Big Data. In: International Conference on Computing, Communication and Security (ICCCS), pp. 1–9 (2015)

    Google Scholar 

  39. Rao, D., Gudivada, V.N., Raghavan, V.V.: Data quality issues in Big Data. In: IEEE International Conference on Big Data (Big Data) (2015)

    Google Scholar 

  40. Pipino, L.L., Lee, Y.W., Wang, R.Y.: Data quality assessment. Commun. ACM 45(4), 211–218 (2002)

    Article  Google Scholar 

  41. Cheah, Y.-W., Canon, R., Plale, B., Ramakrishnan, L.: Milieu: lightweight and configurable Big Data

    Google Scholar 

  42. Monga, M., Sicari, S.: Assessing data quality by a cross-layer approach. In: IEEE International Conference on Ultra Modern Telecommunications & Workshops (ICUMT 2009) (2009)

    Google Scholar 

  43. Ding, X., Wang, H., Zhang, D., Li, J., Gao, H.: A fair data market system with data quality evaluation and repairing recommendation. In: Web Technologies and Applications, pp. 855–858 (2015)

    Google Scholar 

  44. Immonen, A., Pääkkönen, P., Ovaska, E.: Evaluating the quality of social media data in Big Data architecture. In: IEEE Access, vol. 3, pp. 2028–2043 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing, College of Arts and Sciences, Universiti Utara Malaysia, 06010, Sintok, Kedah, Malaysia

    Grace Amina Onyeabor & Azman Ta’a

  2. Federal University of Technology, Minna, Nigeria

    Grace Amina Onyeabor

Authors
  1. Grace Amina Onyeabor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Azman Ta’a
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Grace Amina Onyeabor .

Editor information

Editors and Affiliations

  1. College of Computer Science and Engineering, Taibah University, Medina, Saudi Arabia

    Faisal Saeed

  2. Information Systems Department, Faculty of Computing, Universiti Teknologi Malaysia, Skudai, Malaysia

    Nadhmi Gazem

  3. Information Technology Department, Faculty of Engineering and Information Technology, Taiz University, Taiz, Yemen

    Fathey Mohammed

  4. Information Systems Department, Faculty of Computing, Universiti Teknologi Malaysia, Skudai, Malaysia

    Abdelsalam Busalim

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Onyeabor, G.A., Ta’a, A. (2019). A Model for Addressing Quality Issues in Big Data. In: Saeed, F., Gazem, N., Mohammed, F., Busalim, A. (eds) Recent Trends in Data Science and Soft Computing. IRICT 2018. Advances in Intelligent Systems and Computing, vol 843. Springer, Cham. https://doi.org/10.1007/978-3-319-99007-1_7

Download citation

Publish with us

Policies and ethics

Search

Navigation