A framework for discovering interesting business changes from data
Since the world with its markets, innovations and customers is changing faster than ever before, the key to survival for businesses is the ability to detect, assess and respond to changing conditions — rapidly and intelligently. Discovering changes, and reacting to or acting upon them before others do, has therefore become a strategic issue for many companies.
Many businesses collect huge volumes of data. Commonly this data is continuously gathered over long periods and thus reflects changes in the parts of the business from which it has been derived. To control their business operations and to gain a competitive edge, it is crucial for businesses to detect these changes — early and precisely. However, existing data analysis techniques are insufficient for this task. The widely used method for defining key performance indicators is too weak to detect changes early enough, and requires time-consuming in-depth analysis before decisions can be made. State-of-the-art knowledge discovery techniques, on the other hand, provide the required level of detail, but assume that the domain under consideration is stable over time.
This paper presents a framework that detects changes within a data set at virtually any level of granularity. The underlying idea is to derive a rule-based description of the data set at different points in time and to subsequently analyse how these rules change. While rules are themselves a very comprehensible representation of knowledge, further techniques are required to assist the data analyst in interpreting and assessing changes. Therefore the framework also contains methods to discard rules that are non-drivers for change, and to assess the interestingness of the detected changes.
KeywordsAssociation Rule Change Pattern Data Warehouse Business Intelligence Intelligent Data Analysis
- 1.Watterson K: ’A Data Miner’s Tools’ BYTE Magazine (October 1995).Google Scholar
- 2.Fayyad U M, Piatetsky-Shapiro G, Smyth P and Uthurusamy R: ’Advances in Knowledge Discovery and Data Mining’ AAAI Press and MIT Press, Menlo Park and Cambridge, MA, USA (1996).Google Scholar
- 3.Kimball R: ’Data Warehouse Toolkit: Practical Techniques for Building High Dimensional Data Warehouses’ John Wiley & Sons (1996).Google Scholar
- 4.Agrawal R, Imielinski T and Swami A: ’Mining Association Rules Between Sets of Items in Large Databases’ in Proceedings of the ACM SIGMOD International Conference on Management of Data, pp 207–216, Washington DC (1993).Google Scholar
- 7.Chatfield C: ’Time-Series Forecasting’ Chapman and Hall/CRC (2001).Google Scholar
- 10.Böttcher M, Spott M and Nauck D: ’Detecting Temporally Redundant Association Rules’ in Proceedings of 4th International Conference on Machine Learning and Applications, pp 397–403, Los Angeles, USA, IEEE Computer Society (2005).Google Scholar