Mining algorithms for sequential patterns in parallel : Hash based approach
In this paper, we study the problem of mining sequential patterns in a large database of customer transactions. Since finding sequential patterns has to handle a large amount of customer transaction data and requires multiple passes over the database, it is expected that parallel algorithms help to improve the performance significantly. We consider the parallel algorithms for mining sequential patterns on a shared-nothing environment. Three parallel algorithms (Non Partitioned Sequential Pattern Mining(NPSPM), Simply Partitioned Sequential Pattern Mining(SPSPM) and Hash Partitioned Sequential Pattern Mining(HPSPM)) are proposed. In NPSPM, the candidate sequences are just copied among all the nodes, which can lead to memory overflow for large databases. The remaining two algorithms partition the candidate sequences over the nodes, which can efficiently exploit the total system's memory as the number of nodes in increased. If it is partitioned simply, customer transaction data has to be broadcasted to all nodes. HPSPM partitions the candidate sequences among the nodes using hash function, which eliminates the customer transaction data broadcasting and reduces the comparison workload. We describe the implementation of these algorithms on a shared-nothing parallel computer IBM SP2 and its performance evaluation results. Among three algorithms HPSPM attains best performance.
Unable to display preview. Download preview PDF.
- 1.R.Agrawal and R.Srikant. Mining sequential patterns. In Proc. of the 11th Int. Conf. on Data Engineering, pages 3–14, March 1995.Google Scholar
- 2.R.Srikant and R.Agrawal. Mining sequential patterns: Generalizations and performance improvements. In Proc. of the 5th Int. Conf. on Extending Database Technology, March 1996.Google Scholar
- 3.S.-J.Yen and A.L.P.Chen. An efficient approach to discovering knowledge from large databases. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 82-18, December 1996.Google Scholar
- 4.T.Shintani and M.Kitsuregawa. Hash based parallel algorithms for mining association rules. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 19–30, December 1996.Google Scholar
- 5.J.S.Park, M.-S.Chen, and P.S.Yu. Efficient parallel data mining for association rules. In Proc. of the 4th Conf. on Information and Knowledge Management, pages 31–36, November 1995.Google Scholar
- 6.D.W.Cheung, J.Han, V.T.Ng, A.W.Fu, and Y.Fu. A fast distributed algorithms for mining association rules. In Proc. of 4th Int. Conf. on Parallel and Distributed Information Systems, pages 31–42, December 1996.Google Scholar
- 9.E.-H.(Sam)Han, G.Kaxypis, and V.Kumar. Scalable parallel data mining for association rules. In Proc. of 1997 ACM SIGMOD Int. Conf. on Management of Data, pages 277–288, 1997.Google Scholar