Email Surveillance Using Non-negative Matrix Factorization
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
In this study, we apply a non-negative matrix factorization approach for the extraction and detection of concepts or topics from electronic mail messages. For the publicly released Enron electronic mail collection, we encode sparse term-by-message matrices and use a low rank non-negative matrix factorization algorithm to preserve natural data non-negativity and avoid subtractive basis vector and encoding interactions present in techniques such as principal component analysis. Results in topic detection and message clustering are discussed in the context of published Enron business practices and activities, and benchmarks addressing the computational complexity of our approach are provided. The resulting basis vectors and matrix projections of this approach can be used to identify and monitor underlying semantic features (topics) and message clusters in a general or high-level way without the need to read individual electronic mail messages.
- Berry, M., Browne, M. (2005) Understanding Search Engines: Mathematical Modeling and Text Retrieval. SIAM, Philadelphia, PA
- Berry, M., Drmač, Z., Jessup, E. (1999) Matrices, Vector Spaces, and Information Retrieval. SIAM Review 41: pp. 335-362
- Donoho, D. and V. Stodden (2003), “When does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?” Technical report,Department of Statistics, Stanford University. Preprint.
- Giles, J., Wo, L., Berry, M. GTP (General Text Parser) Softwarefor Text Mining. In: Bozdogan, H. eds. (2003) Software for Text Mining, in Statistical Data Mining and Knowledge Discovery. CRC Press, Boca Raton, FL, pp. 455-471
- Grieve, T. (2003, October 14). The Decline and Fall of the Enron Empire.Slate. http://www.salon.com/news/feature/2003/10/14/enron/index_np.html.
- Guillamet, D. and J. Vitria (2002), “Determining a Suitable Metricwhen Using Non-Negative Matrix Factorization,” in Sixteenth International Conference on Pattern Recognition (ICPR'02), Vol. 2, Quebec City, QC, Canada.
- Hoyer, P. (2002), “Non-Negative Sparse Coding,” in Proceedings of the IEEEWorkshop on Neural Networks for Signal Processing, Martigny, Switzerland.
- Hyvärinen, A., Hoyer, P. (2000) Emergence of Phase and Shift Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces. Neural Computation 12: pp. 1705-1720
- Jolliffe, I. (2002) Principle Component Analysis. Springer-Verlag, New York
- Keila, P. and D. Skillicorn (2005, April 23), “Structure in the Enron Email Dataset,” in Proceedings of the Link Analysis, Counterterrorism, and Security Workshop, Fifth SIAM International Conference on Data Mining, Newport Beach, CA, pp. 55–64.
- Lee, D., Seung, H. (1999) Learning the Parts of Objects by Non-Negative Matrix Factorization. Nature 401: pp. 788-791
- Lee, D., Seung, H. (2001) Algorithms for Non-Negative Matrix Factorization. Advances in Neural Information Processing Systems 13: pp. 556-562
- Liu, W. and J. Yi (2003), “Existing and New Algorithms for Non-Negative Matrix Factorization,” Technical report, Departmentof Computer Sciences, University of Texas at Austin. Preprint.
- McCallum, A., A. Corrada-Emmanuel, and X. Wang (2005, April 23). “The Author-Recipient-Topic Model for Topic and Role Discovery in Social Networks with Application to Enron and Academic Email,” in Proceedings ofthe Link Analysis, Counterterrorism, and Security Workshop, Fifth SIAMInternational Conference on Data Mining, Newport Beach, CA, pp. 33–44.
- McLean, B. and P. Elkind (2003), The Smartest Guys in the Room: The Amazing Rise and Scandalous Fall of Enron. Portfolio.
- Mu, Z., R. Plemmons, and P. Santago (2003), “Iterative Ultrasonic Signaland Image Deconvolution for Estimating the Complex Medium Response,”in IEEE Transactions on Ultrasonics and Frequency Control, IEEE, Submitted for publication.
- Prasad, S., Torgersen, T., Pauca, V., Plemmons, R., van der Gracht, J. (2003) Restoring Images with Space Variant Blur via Pupil Phase Engineering. Optics in Info. Systems, Special Issue on Comp. Imaging, SPIEInt. Tech. Group Newsletter 14: pp. 4-5
- Shahnaz, F., Berry, M., Pauca, V., Plemmons, R. (2006) Document Clustering Using Nonnegative Matrix Factorization. Information Processing and Management 42: pp. 373-386 CrossRef
- Xu, W., X. Liu, and Y. Gong (2003), “Document-Clustering based on Non-negative Matrix Factorization,” in Proceedings of SIGIR'03, July 28–August 1, Toronto, CA, pp. 267–273.
- Email Surveillance Using Non-negative Matrix Factorization
Computational & Mathematical Organization Theory
Volume 11, Issue 3 , pp 249-264
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers
- Additional Links
- electronic mail
- Enron collection
- non-negative matrix factorization
- topic detection
- constrained least squares