Data-Intensive Computing and the Future of Research

Abstract

Massive data sets are being produced in industry and academia of today. Scientists are probing extreme phenomena in scientific fields with mature theories like astrophysics and particle physics. At the same time we see increasingly exploratory research areas evolve, mining large data sets to find new phenomena and patterns. In industry, but also very much in academia, there are huge efforts in making meaning of human activity on the Internet, and as if these data sets were not enough, sensor networks ‘sensing everything everywhere’ is evolving. Information advantage, be it in business or academia, is crucial in today’s global competition, and that is why there is so much interest in data and the technologies handling the data. What is new in the discussions about data and its underlying value is the increasing rate in the production of information, and how companies and academia are cross-fertilizing the information flows to produce even more information. Internet, Cloud Computing, ‘Big Data’, Internet of Things – it is easy to get lost in the technical discussions forgetting what it is all about: information, how to gather it, how to manage it, and how to make timely and informed decisions based on what we find. During the last decade much of the discussions have been centered on the effects of the cloud computing paradigm shift, but that is only the latest technological achievements in the overall effort of producing and analyzing information. In this chapter we look into the characteristics and evolution of information technology, discussing in more detail the latest paradigm shifts, and the new challenges and opportunities facing the companies and scientists. In the end of the chapter we include a list of suggested research topics in this area.