A Framework for Describing Big Data Projects

Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 263)


With the ability to collect, store and analyze an ever-growing diversity of data generated with ever-increasing frequency, Big Data is a rapidly growing field. While tremendous strides have been made in the algorithms and technologies that are used to perform the analytics, much less has been done to determine how the team should work together to do a Big Data project. Our research reports on a set of case studies, where researchers were embedded within Big Data teams. Since project methodologies will likely depend on the attributes of a Big Data effort, we focus our analysis on defining a framework to describe a Big Data project. We then use this framework to describe the organizations we studied and some of the socio-technical challenges linked to these newly defined project characteristics.


Big data Data science Project management Process methodology 


  1. 1.
    Das, M., Cui, R., Campbell, D., Agrawal, G., Ramnath, R.: Towards methods for systematic research on big data. In: IEEE International Conference on Big Data (2015)Google Scholar
  2. 2.
    Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4), 1165–1188 (2012)Google Scholar
  3. 3.
    Ahangama, S., Poo, D.: Improving health analytic process through project, communication and knowledge management. In: The International Conference on Information Systems (2015)Google Scholar
  4. 4.
    Saltz, J.: The need for new processes, methodologies and tools to support big data teams and improve big data project effectiveness. In: 3rd IEEE International Conference on Big Data (2015)Google Scholar
  5. 5.
    Kelly, J., Kaskade, J.: CIOs & BIG DATA what your IT team wants you to know (2013).
  6. 6.
    Espinosa, J., Armour, F.: The big data analytics gold rush: a research framework for coordination and governance. In: Proceedings of Hawaii International Conference on System Sciences (2016)Google Scholar
  7. 7.
    Saltz, J., Shamshurin, I.: Exploring the process of doing big data via an ethnographic study of a media advertising company. In: 3rd IEEE International Conference on Big Data (2015)Google Scholar
  8. 8.
    Mariscal, G., Marban, O., Fernandez, C.: A survey of data mining and knowledge discovery process models and methodologies. Knowl. Eng. Rev. (2010)Google Scholar
  9. 9.
    Ransbotham, S., Kiron, D., Prentice, P.K.: Minding the analytics gap. MIT Sloan Manage. Rev. 56(3), 62–68 (2015)Google Scholar
  10. 10.
    Bhardwaj, A., Bhattacherjee, S., Chavan, A., Deshpande, A., Elmore, A., Madden, S., Parameswaran, A.: DataHub: collaborative data science & dataset version management at scale. In: Biennial Conference on Innovative Data Systems Research (2015)Google Scholar
  11. 11.
    Gao, J., Koronios, A., Selle, S.: Towards a process view on critical success factors in big data analytics projects. In: 21st Americas’ Conference on Information Systems (2015)Google Scholar
  12. 12.
    Grady, N., Underwood, M., Roy, A., Chang, W.: Big data: challenges, practices and technologies: NIST big data public working group workshop. In: IEEE International Conference on Big Data (2014)Google Scholar
  13. 13.
    Chandler, N., Oestreich, T.W.: Use analytic business processes to drive business performance (2015).
  14. 14.
    Beyer, M.: Gartner says solving ‘big data’ challenge involves more than just managing volumes of data (2011). Accessed 28 Feb 2016
  15. 15.
    IBM: the four V’s of big data (2013). Accessed 22 Feb 2016
  16. 16.
    Bystrom, K., Jarvelin, K.: Task complexity affects information seeking and use. Inf. Process. Manage. 31(2), 191–213 (1995)CrossRefGoogle Scholar
  17. 17.
    Kaisler, S., Armour, F., Espinosa, J.A., Money, W.: Big data: issues and challenges moving forward. In: System Sciences (HICSS), 2013 44th Hawaii International Conference on System Sciences, pp. 995–1004 (2013)Google Scholar
  18. 18.
    Jagadish, H., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J.M., Ramakrishnan, R., Shahabi, C.: Big data and its technical challenges. Commun. ACM 57(7), 86–94 (2014)CrossRefGoogle Scholar
  19. 19.
    Guo, P.: Big data workflow: overview and challenges. BLOG@CACM (2013).
  20. 20.
    Shearer, C.: The CRISP-DM model: the new blueprint for data mining. J. Data Warehouse. 5(4), 13–22 (2000)Google Scholar
  21. 21.
    Conway, D.: The big data venn diagram (2013).
  22. 22.
    Stanton, J.M.: Introduction to data science (2013). Accessed
  23. 23.
    Eisenhardt, K.: Building theories from case study research. Acad. Manag. Rev. 14(4), 532–550 (1989)Google Scholar
  24. 24.
    Britos, P., Dieste, O., García-Martínez, R.: Requirements elicitation in data mining for business intelligence projects. In: Avison, D., Kasper, G.M., Pernici, B., Ramos, I., Roode, D. (eds.). ITIFIP, vol. 274, pp. 139–150Springer, Heidelberg (2008). doi: 10.1007/978-0-387-09682-7-9_12 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Syracuse UniversitySyracuseUSA

Personalised recommendations