Advertisement

Promise and Perils of Big Data Science for Intelligence Community

  • Karan P. Jani
  • Anmol Soni
Chapter
Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)

Abstract

Collecting, processing, and analyzing digital data in bulk holds critical importance today more than it has at any period in past, or, arguably, in future. Big data science is influencing the global financial, industrial, academic as well as defense sectors. With the exponential rise of open source data from social media and increasing government monitoring, big data science is now closely aligned with national security policies, and the intelligence community. This chapter reviews the role that big data sciences can play in supporting functions of the intelligence community. A major part of the chapter focuses on the inherent limitations of big data, which can affect and even disrupt the chain of operations of intelligence agencies from gathering information to anticipating surprises. The limiting factors range from technical to ethical issues. The chapter concludes that there is a continuing need for experts with domain knowledge from intelligence community to efficiently guide big data analysis to fill any gaps in knowledge. As a case study on limitations of using big data, work in nuclear intelligence using simple analytics is examined to show why big data analysis in certain cases may lead to unnecessary complications.

References

  1. Andrieu, C., De Freitas, N., Doucet, A., & Jordan, M. (2003). An introduction to MCMC for machine learning. Machine Learning, 50(1), 5–43.CrossRefGoogle Scholar
  2. Arbesman, S. (2013). Stop Hyping Big Data and Pay Attention to Long Data, WIRED, 2013 http://www.wired.com/2013/01/forget-big-data-think-long-data/
  3. Baxter, P. (2015). The False Hope of Nuclear Forensics? Assessing the Timeliness of Forensics Intelligence. FAS http://fas.org/pir-pubs/the-false-hope-of-nuclear-forensics-assessing-the-timeliness-of-forensics-intelligence/
  4. Chandrasekaran, N. (2015). Is data the new currency? World Economic Forum Retrieved April 29, 2017 from https://www.weforum.org/agenda/2015/08/is-data-the-new-currency/
  5. Cox, M. & Ellsworth, D. (1997). Application-controlled demand paging for out-of-core visualization. Retrieved April 29, 2017 from https://www.nas.nasa.gov/assets/pdf/techreports/1997/nas-97-010.pdf
  6. Defense Science Board Task Force Report. (2014, January). Assessment of Nuclear Monitoring and Verification Technologies, https://www.acq.osd.mil/DSB/reports/2010s/NuclearMonitoringAndVerificationTechnologies.pdf
  7. Detsch, J. (2015). The mixed consequences of Sino-Indian competition in the Indian Ocean, The Diplomat https://thediplomat.com/2015/01/the-mixed-consequences-of-sino-indian-competition-in-the-indian-ocean/
  8. Diaz, J. (2013). Internet Explorer vs Murder Rate Will Be Your Favorite Chart Today, Gizmodo, http://gizmodo.com/5977989/internet-explorer-vs-murder-rate-will-be-your-favorite-chart-today
  9. Energy.gov (2014, November). Department of Energy Awards $425 Million for Next Generation Supercomputing Technologies. http://energy.gov/articles/department-energy-awards-425-million-next-generation-supercomputing-technologies
  10. Fahey, S. (2012). Big Data Analytics and National Security http://web.stanford.edu/group/mmds/slides2012/s-fahey.pdf
  11. Gandomi, A. Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management. 35 2 2015137–144. ISSN 0268-4012,  https://doi.org/10.1016/j.ijinfomgt.2014.10.007.
  12. Gerbaudo, P. (2012). Tweets and the streets: Social media and contemporary activism. London: Pluto Press.Google Scholar
  13. Harbert, T. (2015). IARPA’s New Director Wants You to Surprise Him, IEE Spectrum, https://spectrum.ieee.org/computing/networks/iarpas-new-director-wants-you-to-surprise-him#TopPageAnchor
  14. Howard, P. N., Duffy, A., Freelon, D., Hussain, M. M., Mari, W., & Maziad, M. (2011). Opening closed regimes: What was the role of social media during the Arab spring?, SSRN,  https://doi.org/10.2139/ssrn.2595096
  15. Jain, S., Rosenblatt, M., Duke, J. (2014). Is big data the new frontier for academic-industry collaboration?. Journal of the American Medical Association. 2014;311(21):2171–2172. doi: https://doi.org/10.1001/jama.2014.1845, https://jamanetwork.com/journals/jama/fullarticle/1854520
  16. Lant, K. (2017). Apple Is Using AI to Unlock Previously Unusable “Dark Data”, Futurism https://futurism.com/apple-is-using-ai-to-unlock-previously-unusable-dark-data/
  17. Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014). The Parable of Google Flu: Traps in Big Data Analysis. Science, 343(6176, March 14), 1203–1205.CrossRefGoogle Scholar
  18. Lee, D. (2017, November 22). Uber concealed huge data breach. BBC News. http://www.bbc.com/news/technology-42075306
  19. Makuch, B. (2014). The Database Tracking the World’s News Could Predict Future Conflicts, Motherboard, http://motherboard.vice.com/read/the-database-tracking-the-worlds-news-could-predict-future-conflicts
  20. Manjoo, F. (2017). Tech’s frightful five: They’ve got us. New York Times. Retrieved September 16, 2017, from https://www.nytimes.com/2017/05/10/technology/techs-frightful-five-theyvegotus.html?mcubz=1
  21. March, G. & Davis, E. (2014). Eight (No, Nine!) Problems With Big Data. New York Times, http://www.nytimes.com/2014/04/07/opinion/eight-no-nine-problems-with-big-data.html?_r=0
  22. McKinsey (2012). McKinsey Global Institute, Big Data: The next frontier for innovation, competition, and productivity Report, June, 2012.Google Scholar
  23. Metropolis, N. (1987). The Beginning of the Monte Carlo Method. Los Alamos Science, No. 15, p. 125, http://library.lanl.gov/cgi-bin/getfile?00326866.pdf
  24. MIT Sloan (2017). What’s Your Data Worth? MIT Sloan Management Review. Retrieved September 16, 2017, from http://sloanreview.mit.edu/article/whats-your-data-worth/
  25. Mitroff, S., Sharpe, B. (2017). Using big data to solve real problems through academic and industry partnerships, Current Opinion in Behavioral Sciences 1891–96, ISSN 2352-1546,  https://doi.org/10.1016/j.cobeha.2017.09.013. http://www.sciencedirect.com/science/article/pii/S2352154617301900
  26. NextGov (2014) Stop Loose Nukes With Big Data And Crowdsourcing, Experts Urge http://www.nextgov.com/big-data/2014/01/stop-loose-nukes-big-data-and-crowdsourcing-experts-urge/77324/
  27. Norrie J. (2017). Big Data Needs Bigger Security, U.S. News, https://www.usnews.com/opinion/economic-intelligence/articles/2017-09-11/equifax-hack-shows-why-big-data-needs-bigger-securityGoogle Scholar
  28. Omand, D., et al. (2012). Introducing social media intelligence (SOCMINT). Intelligence and National Security, 27, 801–823.  https://doi.org/10.1080/02684527.2012.716965.CrossRefGoogle Scholar
  29. Pomerleau, M. (2015). IC embraces open source intel, even if it is double-edged. Defense Systems, https://defensesystems.com/articles/2015/09/25/iarpa-open-source-intelligence.aspx?admgarea=DS
  30. Preston, J. (2011). Syria restores access to Facebook and YouTube, New York Times, http://www.nytimes.com/2011/02/10/world/middleeast/10syria.html
  31. Richards, N. M. & King, J.H. (2013). Three Paradoxes of Big Data. Stanford Law Review. http://www.stanfordlawreview.org/sites/default/files/online/topics/66_StanLRevOnline_41_RichardsKing.pdf
  32. Richelson, J. T. (2002). The Wizards of Langley: Inside The Cia’s Directorate Of Science And Technology (p. 386 pp., apps., bibl., index. Boulder, Colo). Westview Press.Google Scholar
  33. Roberto, V. Z. (2013). Big Data: Challenges and Opportunities. http://www.odbms.org/wp-content/uploads/2013/07/Big-Data.Zicari.pdf
  34. Rosenbach, E. and Peritz, A. (2009). “The role of private corporations in the intelligence community.” Memorandum, “Confrontation or Collaboration? Congress and the intelligence community,” Belfer Center for Science and International Affairs, Harvard Kennedy School, July 2009.Google Scholar
  35. Shane, S. (2017). These are the Ads Russia Bought on Facebook in 2016, New York Times, https://www.nytimes.com/2017/11/01/us/politics/russia-2016-election-facebook.html
  36. SINTEF. (2013). Big Data, for better or worse: 90% of world’s data generated over last two years. ScienceDaily. Retrieved April 29, 2017 from www.sciencedaily.com/releases/2013/05/130522085217.htm
  37. Soubra D. (2012). The 3Vs that define Big Data, Data Science Central, http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data
  38. Symon, P. and Tarapore, A. (2015). Defense intelligence analysis in the age of big data. Forum of Defense Intelligence and Big Data, 2015.Google Scholar
  39. Temple-Retson, D (2012). Predicting The Future: Fantasy Or A Good Algorithm? NPR http://www.npr.org/2012/10/08/162397787/predicting-the-future-fantasy-or-a-good-algorithm
  40. The Wire (2017). FAQ: What the Right to Privacy Judgment Means for Aadhaar and Mass Surveillance, The Wire https://thewire.in/170700/right-to-privacy-aadhaar-supreme-court/
  41. U.S. Joint Chiefs of Staff (2013). Joint Intelligence Report http://www.dtic.mil/doctrine/new_pubs/jp2_0.pdf
  42. Ward, J. S., & Barker, A. (2013). Undefined by data: A survey of big data definitions. arXiv preprint arXiv:1309.5821.Google Scholar
  43. White House (2014). Big Data: Seizing opportunities, preserving values. Executive Office of the President. https://obamawhitehouse.archives.gov/sites/default/files/docs/big_data_privacy_report_5.1.14_final_print.pdf
  44. White House (2016). Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights. Executive Office of the President. https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/2016_0504_data_discrimination.pdf
  45. Wiener J., Facebook (2016). Facebook’s Top Open Data Problems, https://research.fb.com/facebook-s-top-open-data-problems
  46. World Institute for Nuclear Security (2015). Data Analytics for Nuclear Security, https://www.wins.org/files/30.01.2015_data_analytics_for_nuclear_security_rev_1.0_en_final_1.pdf
  47. Young C. (2012). Military intelligence Redefined: Big Data in the Battlefield, Forbes, March 2012 http://www.forbes.com/sites/techonomy/2012/03/12/military-intelligence-redefined-big-data-in-the-battlefield/#367ab43f718f

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Georgia Institute of TechnologyAtlantaUSA

Personalised recommendations