Skip to main content
SpringerLink
Log in

A distributed learning based sentiment analysis methods with Web applications

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

The main challenge of using deep learning (DL) for sentiment analysis tasks is that insufficient data leads to a decrement in classification accuracy. In addition, privacy issues are always concerned for sentiment data analysis. To tackle the above two mentioned problems, We propose a model based on the federated learning framework (Fed_BERT_MSCNN), which contains a Bidirectional Encoder Represent-ations from Transformers (BERT) module and a multi-scale convolution layer. It uses the BERT_MSCNN model for training on the data sets of multiple companies, and employs the federated learning framework to collect the model parameters of different distributed nodes. Finally, these model parameters are transmitted to the central node. The central node performs a weighted average of all model parameters, sending a set of common model parameters to the distributed nodes. According to the experimental results, the proposed model performs better than the state-of-the-art models in terms of accuracy, F1-score, and computational efficiency. In addition, we optimize the model parameters in order to practice in distributed computing models for web applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11

Similar content being viewed by others

References

  1. Armbrust, M., Fox, A., Griffith, R., et al.: A view of cloud computing[J]. Communications of the ACM. 53(4), 50–58 (2010)

    Article  Google Scholar 

  2. Azrour, M., Mabrouki, J., Guezzaz, A., et al.: New enhanced authentication protocol for internet of things[J]. Big Data Mining and Analytics. 4(1), 1–9 (2021)

    Article  Google Scholar 

  3. Bengio, Y., Ducharme, R., Vincent, P., et al.: A neural probabilistic language model[J]. The journal of Machine Learning Research. 3, 1137–1155 (2003)

    MATH  Google Scholar 

  4. Bi, R., Liu, Q., Ren, J., et al.: Utility aware offloading for mobile-edge computing[J]. Tsinghua Science and Technology. 26(2), 239–250 (2020)

    Article  Google Scholar 

  5. Blitzer, J., Dredze, M., Pereira, F., et al.: Boom-Boxes and Blenders: domain adaptation for sentiment classification[C]//Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’07), Pereira. 2007, 447.

  6. Calero, C., Mancebo, J., García, F., et al.: 5Ws of green and sustainable software[J]. Tsinghua Science and Technology. 25(3), 401–414 (2019)

    Article  Google Scholar 

  7. Chen, Y., Sun, X., Jin, Y.: Communication-efficient federated deep learning with layerwise asynchronous model update and temporally weighted aggregation[J]. IEEE Transactions on Neural Networks and Learning Systems. 31(10), 4229–4238 (2019)

    Article  Google Scholar 

  8. Chen, J., Cai, T., He, W., et al.: A blockchain-driven supply chain finance application for auto retail industry[J]. Entropy. 22(1), 95 (2020)

    Article  Google Scholar 

  9. Dai, W., Kumar, A., Wei, J., et al.: High-performance distributed ML at scale through parameter server consistency models[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 29(1) (2015)

  10. Dastjerdi, A.V., Gupta, H., Calheiros, R.N., et al.: Fog computing: Principles, architectures, and applications[M]//Internet of things. Morgan Kaufmann. 61–75 (2016)

  11. Devlin, J., Chang, M. W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805 (2018)

  12. Gai, K., Qiu, M., Zhao, H., et al.: Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing[J]. Journal of Network and Computer Applications. 59, 46–54 (2016)

    Article  Google Scholar 

  13. Guo, H.Y.: Text classification based on word vector and topic vector[D]. Wuhan: Huazhong University of Science and Technology. 4–24 (2016)

  14. He, Q., Yan, J., Yang, Y., et al.: Chord4s: A p2p-based decentralised service discovery approach[C]//2008 IEEE International Conference on Services Computing. IEEE, 1: 221-228 (2008)

  15. He, Q., Zhou, R., Zhang, X., et al.: Keyword search for building service-based systems[J]. IEEE Transactions on Software Engineering. 43(7), 658–674 (2016)

    Article  Google Scholar 

  16. Ho, Q., Cipar, J., Cui, H., et al.: More effective distributed ml via a stale synchronous parallel parameter server[C]//Advances in neural information processing systems. 1223-1231 (2013)

  17. Hu, M., Ji, Z., Yan, K., et al.: Detecting anomalies in time series data via a meta-feature based approach[J]. IEEE Access. 6, 27760–27776 (2018)

    Article  Google Scholar 

  18. Hu, M., Feng, X., Ji, Z., et al.: A novel computational approach for discord search with local recurrence rates in multivariate time series[J]. Information Sciences. 477, 220–233 (2019)

    Article  Google Scholar 

  19. Ji, Z., Wang, B., Deng, S.P., et al.: Predicting dynamic deformation of retaining structure by LSSVR-based time series method[J]. Neurocomputing. 137, 165–172 (2014)

    Article  Google Scholar 

  20. Jin, N., Wu, J., Ma, X., et al.: Multi-task learning model based on multi-scale CNN and LSTM for sentiment classification[J]. IEEE Access. 8, 77060–77072 (2020)

    Article  Google Scholar 

  21. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv:1404.2188 (2014)

  22. Karpathy, A., Toderici, G., Shetty, S., et al.: Large-scale video classification with convolutional neural networks[C]//Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 1725-1732 (2014)

  23. Kelt, J.: Search before the purchase: Understanding buyer search activity as it builds to online purchase[J]. DoubleClick, February (2005)

    Google Scholar 

  24. Kim, Y.: Convolutional neural networks for sentence classification [C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 1746-1751. (2014) https://doi.org/10.3115/v1/D14-181

  25. Konečný, J., McMahan, H. B., Yu, F.X., et al.: Federated learning: Strategies for improving communication efficiency[J]. arXiv preprint arXiv:1610.05492, (2016)

  26. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems. 25, 1097–1105 (2012)

    Google Scholar 

  27. Li, M., Andersen, D. G., Park, J. W., et al.: Scaling distributed machine learning with the parameter server[C]//11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14). 583-598 (2014)

  28. Li, T., Sanjabi, M., Beirami, A., et al.: Fair resource allocation in federated learning[J]. arXiv preprint arXiv:1905.10497 (2019)

  29. Liang, W., Hu, Y., Zhou, X., et al.: Variational Few-Shot Learning for Microservice-Oriented Intrusion Detection in Distributed Industrial IoT[J]. IEEE Transactions on Industrial Informatics. (2021)

  30. Lin, Y., Han, S., Mao, H., et al.: Deep gradient compression: Reducing the communication bandwidth for distributed training[J]. arXiv preprint arXiv:1712.01887 (2017)

  31. Liu, Y., Pei, A., Wang, F., et al.: An attention-based category-aware GRU model for the next POI recommendation[J]. International Journal of Intelligent Systems. (2021)

  32. Mabrouki, J., Azrour, M., Fattah, G., et al.: Intelligent monitoring system for biogas detection based on the internet of things: Mohammedia, morocco city landfill case[J]. Big Data Mining and Analytics. 4(1), 10–17 (2021)

    Article  Google Scholar 

  33. Mahmud, M.S., Huang, J.Z., Salloum, S., et al.: A survey of data partitioning and sampling methods to support big data analysis[J]. Big Data Mining and Analytics. 3(2), 85–101 (2020)

    Article  Google Scholar 

  34. Malek, Y.N., Najib, M., Bakhouya, M., et al.: Multivariate deep learning approach for electric vehicle speed forecasting[J]. Big Data Mining and Analytics. 4(1), 56–64 (2021)

    Article  Google Scholar 

  35. Mao, Y., You, C., Zhang, J., et al.: A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials. 19(4), 2322–2358 (2017)

    Article  Google Scholar 

  36. McMahan, H. B., Moore, E., Ramage, D., et al.: Federated learning of deep networks using model averaging[J]. arXiv preprint arXiv:1602.05629, (2016)

  37. McMahan, B., Moore, E., Ramage, D., et al.: Communication-efficient learning of deep networks from decentralized data[C]//Artificial intelligence and statistics. PMLR. 1273–1282 (2017)

  38. Niu, F., Recht, B., Ré, C., et al.: Hogwild!: A lock-free approach to parallelizing stochastic gradient descent[J]. arXiv preprint arXiv:1106.5730 (2011)

  39. O'Connor, B., Balasubramanyan, R., Routledge, B. R., et al.: From tweets to polls: Linking text sentiment to public opinion time series[C]//Fourth international AAAI conference on weblogs and social media. (2010)

  40. Pang, B., Lee, L.: Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales[J]. arXiv preprint cs/0506075 (2005)

  41. Peters, M., Neumannm, M., Iyyer, M., et al.: Deep Contextualized Word Representations[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). (2018)

  42. Qi, L., Wang, X., Xu, X., et al.: Privacy-aware cross-platform service recommendation based on enhanced locality-sensitive hashing[J]. IEEE Transactions on Network Science and Engineering. (2020)

  43. Qi, L., Hu, C., Zhang, X., et al.: Privacy-aware data fusion and prediction with spatial-temporal context for smart city industrial environment[J]. IEEE Transactions on Industrial Informatics. 17(6), 4159–4167 (2020)

    Article  Google Scholar 

  44. Qiu, M., Dai, H.N., Sangaiah, A.K., et al.: Guest editorial: Special section on emerging privacy and security issues brought by artificial intelligence in industrial informatics[J]. IEEE Transactions on Industrial Informatics. 16(3), 2029–2030 (2019)

    Article  Google Scholar 

  45. Radford, A., Narasimhan, K., Salimans, T., et al.: Improving language understanding by generative pre-training[J]. (2018)

  46. Rakhlin, A.: Convolutional neural networks for sentence classification[J]. GitHub, (2016)

  47. Rehak, D., Dodds, P., Lannom, L.: A model and infrastructure for federated learning content repositories[C]//Interoperability of web-based educational systems workshop. 143 (2005)

  48. Satyanarayanan, M.: The emergence of edge computing[J]. Computer. 50(1), 30–39 (2017)

    Article  Google Scholar 

  49. Shakarami, A., Ghobaei-Arani, M., Shahidinejad, A.: A survey on the computation offloading approaches in mobile edge computing: A machine learning-based perspective[J]. Computer Networks. 107496, (2020)

  50. Shi, W., Cao, J., Zhang, Q., et al.: Edge computing: Vision and challenges[J]. IEEE internet of things journal. 3(5), 637–646 (2016)

    Article  Google Scholar 

  51. Tan, X., Zhang, J., Zhang, Y., et al.: A PUF-based and cloud-assisted lightweight authentication for multi-hop body area network[J]. Tsinghua Science and Technology. 26(1), 36–47 (2020)

    Article  Google Scholar 

  52. Wang, F., Zhu, H., Srivastava, G., et al.: Robust collaborative filtering recommendation with user-item-trust records[J]. IEEE Transactions on Computational Social Systems. (2021)

  53. Wei, X., Lin, H., Yu, Y., et al.: Low-resource cross-domain product review sentiment classification based on a CNN with an auxiliary large-scale corpus[J]. Algorithms. 10(3), 81 (2017)

    Article  Google Scholar 

  54. Xu, Z.W.: Cloud-Sea Computing Systems: Towards Thousand-Fold Improvement in Performance per Watt for the Coming Zettabyte Era[J]. Journal of Computer Science and Technology. 029(002), 177–181 (2014)

    Article  Google Scholar 

  55. Xu, Y., Zhang, C., Zeng, Q., et al.: Blockchain-enabled accountability mechanism against information leakage in vertical industry services[J]. IEEE Transactions on Network Science and Engineering. (2020)

  56. Yan, K., Shen, W., Jin, Q., et al.: Emerging privacy issues and solutions in cyber-enabled sharing services: From multiple perspectives[J]. IEEE Access. 7, 26031–26059 (2019)

    Article  Google Scholar 

  57. Yuan, Y., Huang, J., Yan, K.: Virtual Reality Therapy and Machine Learning Techniques in Drug Addiction Treatment[C]//2019 10th International Conference on Information Technology in Medicine and Education (ITME). IEEE. 241–245 (2019)

  58. Yuan, Y., Huang, J., Ma, X., et al.: Children's Drawing Psychological Analysis using Shallow Convolutional Neural Network[C]//2020 International Conferences on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics (Cybermatics). IEEE. 692–698 (2020)

  59. Zhang, X., Dou, W., He, Q., et al.: LSHiForest: A generic framework for fast tree isolation based ensemble anomaly analysis[C]//2017 IEEE 33rd International Conference on Data Engineering (ICDE). IEEE, 983-994 (2017)

  60. Zhang, W., Chen, X., Jiang, J.: A multi-objective optimization method of initial virtual machine fault-tolerant placement for star topological data centers of cloud systems[J]. Tsinghua Science and Technology. 26(1), 95–111 (2020)

    Article  Google Scholar 

  61. Zhang, J., Yan, K., Mo, Y.: Multi-Task Learning for Sentiment Analysis with Hard-Sharing and Task Recognition Mechanisms[J]. Information. 12(5), 207 (2021)

    Article  Google Scholar 

  62. Zhang, C., Xu, Y., Hu, Y., et al.: A blockchain-based multi-cloud storage data auditing scheme to locate faults[J]. IEEE Transactions on Cloud Computing. (2021)

  63. Zhou, X., Liang, W., Kevin, I., et al.: Academic influence aware and multidimensional network analysis for research collaboration navigation based on scholarly big data[J]. IEEE Transactions on Emerging Topics in Computing. (2018)

  64. Zhou, X., Xu, X., Liang, W., et al.: Deep Learning Enhanced Multi-Target Detection for End-Edge-Cloud Surveillance in Smart IoT[J]. IEEE Internet of Things Journal. (2021)

  65. Zhou, X., Xu, X., Liang, W., et al.: Intelligent small object detection based on digital twinning for smart manufacturing in industrial CPS[J]. IEEE Transactions on Industrial Informatics. (2021)

  66. Zhou, X., Yang, X., Ma, J., et al.: Energy Efficient Smart Routing Based on Link Correlation Mining for Wireless Edge Computing in IoT[J]. IEEE Internet of Things Journal. (2021)

  67. Zonglin, L.I.U., Meishan, Z., Ranran, Z., et al.: Multi-task learning model for legal judgment predictions with charge keywords[J]. Journal of Tsinghua University (Science and Technology). 59(7), 497–504 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Information Engineering, China Jiliang University, Hangzhou, China

    Guanghao Xiong & Ke Yan

  2. National University of Singapore, Singapore, 4 Architecture Drive, Singapore, 117566, Singapore

    Ke Yan

  3. Faculty of Data Science, Shiga University, Hikone, 5228522, Japan

    Xiaokang Zhou

  4. RIKEN Center for Advanced Intelligence Project, RIKEN, Tokyo, 1030027, Japan

    Xiaokang Zhou

Authors
  1. Guanghao Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ke Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaokang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ke Yan or Xiaokang Zhou.

Ethics declarations

Conflicts of interests

The authors declare that they have no competing interests.

Additional information

This article belongs to Topical Collection: Special Issue on Resource Management at the Edge for Future Web, Mobile, and IoT Applications

Guest Editors: Qiang He, Fang Dong, Chenshu Wu, and Yun Yang

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiong, G., Yan, K. & Zhou, X. A distributed learning based sentiment analysis methods with Web applications. World Wide Web 25, 1905–1922 (2022). https://doi.org/10.1007/s11280-021-00994-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-021-00994-0

Keywords

Search

Navigation