Efficient Pre-processing and Feature Selection for Clustering of Cancer Tweets

  • P. G. LavanyaEmail author
  • K. Kouser
  • Mallappa Suresha
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 910)


The impact of social media in our daily life cannot be overlooked. Harnessing this rich and varied data for information is a challenging job for the data analysts. As each type of data from social media is unstructured, these data have to be processed, represented and then analysed in different ways suitable to our requirements. Though retail industry and political people are using social media to a great extent to gather feedback and market their new ideas, its significance in other fields related to public like health care and security is not dealt with effectively. Though the information coming from social media may be informal, it contains genuine opinions and experiences which are very much necessary to improve the healthcare service. This work explores analysing the Twitter data related to the most dreaded disease ‘cancer’. We have collected over one million tweets related to various types of cancer and summarized the same to a bunch of representative tweets which may give key inputs to healthcare professionals regarding symptoms, diagnosis, treatment and recovery related to cancer. This, when correlated with clinical research and inputs, may provide rich information to provide a holistic treatment to the patients. We have proposed additional pre-processing to the raw data. We have also explored a combination of feature selection methods, two feature extraction methods and a soft clustering algorithm to study the feasibility of the same for our data. The results have proved our intuition right about underlying information and also show that there is a tremendous scope for further research in the area.


Cancer Centroid Clustering Feature selection Feature extraction Fuzzy C-means K-means Laplacian score PCA Summarization SVD Tweets Twitter Variance Visualization 


  1. 1.
  2. 2.
  3. 3.
    Lavanya, P.G., Mallappa, S.: Automatic summarization and visualisation of healthcare tweets. In: Proceedings of the International Conference on 2017Advances in Computing, Communications and Informatics (ICACCI), pp. 1557–1563 (2017).
  4. 4.
    Crockett, K., Mclean, D., Latham, A., Alnajran, N.: Cluster Analysis of twitter data: a review of algorithms. In: Proceedings of the 9th International Conference on Agents and Artificial Intelligence, pp. 239–249 (2017).
  5. 5.
    Cunha, J., Silva, C., Antunes, M.: Health twitter big d ata management with hadoop framework. Procedia Comput. Sci. 64, 425–431 (2015). Scholar
  6. 6.
    Carchiolo, V., Longheu, A., Malgeri, M.: Using twitter data and sentiment analysis to study diseases dynamics. In: Proceedings of the International Conference on Information Technology in Bio-and Medical Informatics, pp. 16–24 (2015). Scholar
  7. 7.
    Tripathy, R.M., Sharma, S., Joshi, S., Mehta, S., Bagchi, A.: Theme based clustering of tweets. In: Proceedings of the 1st IKDD Conference on Data Sciences, pp. 1–5 (2014).
  8. 8.
    Sechelea, A., Do Huu, T., Zimos, E., Deligiannis, N.: Twitter data clustering and visualization. ICT, pp. 1–5 (2016).
  9. 9.
    Dutta, S., Ghatak, S., Roy, M., Ghosh, S., Das, A.K.: A graph based clustering technique for tweet summarization. In: Proceedings of the 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO)(Trends and Future Directions), pp. 1–6 (2015).
  10. 10.
    Jiwanggi, M.A., Adriani, M.: Topic summarization of microblog document in Bahasa Indonesia using the phrase reinforcement algorithm. Procedia Comput. Sci. 81, 229–236 (2016). Scholar
  11. 11.
    Zhuang, H., Rahman, R., Hu, X., Guo, T., Hui, P., Aberer, K.: Data summarization with social contexts. In: Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 397–406 (2016).
  12. 12.
    Sindhuja, P., Suneetha, J.: An Advanced approach for summarization and timeline generation of evolutionary tweet streamsGoogle Scholar
  13. 13.
    Ventola, C.L.: Social media and health care professionals: benefits, risks, and best practices. Pharm. Ther. 39, 491 (2014)Google Scholar
  14. 14.
    Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31, 651–666 (2010). Scholar
  15. 15.
    Panda, S., Sanat, S., Jena, P., Chattopadhyay, S.: Comparing fuzzy-C means and K-means clustering techniques: a comprehensive study. In: Advances in Computer Science, Engineering & Applications, pp. 451–460. Springer, Berlin, Heidelberg (2012). Scholar
  16. 16.
    Dash, M., Liu, H.: Feature selection for clustering. In: Pacific-Asia Conference on knowledge discovery and data mining, pp. 110–121 (2000)CrossRefGoogle Scholar
  17. 17.
    Vasan, K.K., Surendiran, B.: Dimensionality reduction using principal component analysis for network intrusion detection. Perspect. Sci. 8, 510–512 (2016). Scholar
  18. 18.
    Wang, Y., Zhu, L.: Research and implementation of SVD in machine learning. In: Proceedings of the 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS), pp. 471–475 (2017).
  19. 19.
    Steinberger, J., Jevzek, K.: Evaluation measures for text summarization. Comput. Inform. 28, 251–275 (2012)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2020

Authors and Affiliations

  1. 1.DoS in Computer ScienceUniversity of MysoreMysuruIndia
  2. 2.Government First Grade CollegeGundlupetIndia

Personalised recommendations