Advanced Maintenance of Data Richness in Business Communication Data—An Evaluation of Dimensionality Reduction Techniques

Abstract

Business negotiations are conducted through communication enabling the declaration of negotiation objectives and active implementation of negotiation strategies to achieve pre-defined goals. The processing of exchanged textual communication enables the automatic transformation of unstructured data into processable structured datasets and subsequently the analysis of textual content without losing the data richness of exchanged communication messages. For this purpose, this paper discusses Text Mining-based pre-processing approaches by comparing TF-IDF and frequency measures. Furthermore, dimensionality reduction techniques from Feature Extraction, Feature Selection from Machine Learning and an additional statistical approach are evaluated to be able to counteract the curse of dimensionality in textual processing. In doing so, the maintenance of data richness in communication data is considered to be the overall goal to determine the dataset with minimal information loss. Therefore, various pre-processed and transformed communication datasets derived from dimensionality reduction are integrated as input data into selected classification models to measure the prediction performance with ROC analysis. The overall results of ROC show that quantified business communication data reduced by PCA delivers the most valuable data based on Porter’s stemming algorithm followed by the quantified data combinations of Optimize Selection using a SVM classifier.

Co-Author Prof. Mareike Schoop, PhD

The content of this chapter was extended based on the published article: Muhammed-Fatih Kaya, Mareike Schoop, 2020, Maintenance of Data Richness in Business Communication Data. In: Proceedings of the 28th European Conference on Information Systems (ECIS 2020), https://aisel.aisnet.org/ecis2020_rp/189 .