Skip to main content
SpringerLink
Log in

Multi-Agent Systems for Collaborative Inference Based on Deep Policy Q-Inference Network

  • Research
  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

This study tackles the problem of increasing efficiency and scalability in deep neural network (DNN) systems by employing collaborative inference, an approach that is gaining popularity because to its ability to maximize computational resources. It involves splitting a pre-trained DNN model into two parts and running them separately on user equipment (UE) and edge servers. This approach is advantageous because it results in faster and more energy-efficient inference, as computation can be offloaded to edge servers rather than relying solely on UEs. However, a significant challenge of collaborative belief is the dynamic coupling of DNN layers, which makes it difficult to separate and run the layers independently. To address this challenge, we proposed a novel approach to optimize collaborative inference in a multi-agent scenario where a single-edge server coordinates the assumption of multiple UEs. Our proposed method suggests using an autoencoder-based technique to reduce the size of intermediary features and constructing tasks using the deep policy inference Q-inference network’s overhead (DPIQN). To optimize the collaborative inference, employ the Deep Recurrent Policy Inference Q-Network (DRPIQN) technique, which allows for a hybrid action space. The results of the tests demonstrate that this approach can significantly reduce inference latency by up to 56% and energy usage by up to 72% on various networks. Overall, this proposed approach provides an efficient and effective method for implementing collaborative inference in multi-agent scenarios, which could have significant implications for developing DNN systems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Hao, Z., Xu, G., Luo, Y., Hu, H., An, J. Mao, S.: Multi-agent collaborative inference via dnn decoupling: Intermediate feature compression and edge learning. IEEE Trans. Mob. Comput. (2022)

  2. He, K., Zhang, X., Ren, S. Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition 770–778 (2016)

  3. Zou, W., Sun, Y., Zhou, Y., Lu, Q., Nie, Y., Sun, T., Peng, L.: Limited Sensing and Deep Data Mining: A New Exploration of Developing City-Wide Parking Guidance Systems. IEEE Intell. Transp. Syst. Mag. 14(1), 198–215 (2022)

    Article  Google Scholar 

  4. Li, T., Xia, T., Wang, H., Tu, Z., Tarkoma, S., Han, Z., Hui, P.: Smartphone App Usage Analysis: Datasets, Methods, and Applications. IEEE Communications Surveys & Tutorials 24(2), 937–966 (2022)

    Article  Google Scholar 

  5. Shan, N., Ye, Z., Cui, X.: Collaborative intelligence: Accelerating deep neural network inference via device-edge synergy. Security and Communication Networks 2020, 1–10 (2020)

    Article  Google Scholar 

  6. Shlezinger, N., Farhan, E., Morgenstern, H. Eldar, Y.C.: June. Collaborative inference via ensembles on the edge. In ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 8478–8482 IEEE (2021)

  7. Li, H., Huang, Q., Huang, J., Susilo, W.: Public-Key Authenticated Encryption With Keyword Search Supporting Constant Trapdoor Generation and Fast Search. IEEE Trans. Inf. Forensics Secur. 18, 396–410 (2023)

    Article  Google Scholar 

  8. Sun, G., Li, Y., Liao, D., Chang, V.: Service Function Chain Orchestration Across Multiple Domains: A Full Mesh Aggregation Approach. IEEE Trans. Netw. Serv. Manage. 15(3), 1175–1191 (2018)

    Article  Google Scholar 

  9. Rodriguez-Conde, I., Campos, C., Fdez-Riverola, F.: Horizontally Distributed Inference of Deep Neural Networks for AI-Enabled IoT. Sensors 23(4), 1911 (2023)

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  10. Dai, M., Sun, G., Yu, H., Niyato, D.: Maximize the Long-Term Average Revenue of Network Slice Provider via Admission Control Among Heterogeneous Slices. IEEE/ACM Trans. Netw. (2023)

  11. Li, T., Li, Y., Hoque, M.A., Xia, T., Tarkoma, S., Hui, P.: To What Extent We Repeat Ourselves? Discovering Daily Activity Patterns Across Mobile App Usage. IEEE Trans. Mob. Comput. 21(4), 1492–1507 (2022)

    Article  Google Scholar 

  12. Liu, X., Zhou, G., Kong, M., Yin, Z., Li, X., Yin, L., Zheng, W.: Developing Multi-Labelled Corpus of Twitter Short Texts: A Semi-Automatic Method. Systems 11(8), 390 (2023)

    Article  Google Scholar 

  13. Li, X., Sun, Y.: Stock intelligent investment strategy based on support vector machine parameter optimization algorithm. Neural Comput. Appl. 32(6), 1765–1775 (2020)

    Article  Google Scholar 

  14. Li, X., Sun, Y.: Application of RBF neural network optimal segmentation algorithm in credit rating. Neural Comput. Appl. 33(14), 8227–8235 (2021)

    Article  Google Scholar 

  15. Hong, Z.W., Su, S.Y., Shann, T.Y., Chang, Y.H., Lee, C.Y.: A deep policy inference q-network for multi-agent systems. arXiv preprint (2017) arXiv:1712.07893

  16. Liu, X., Wang, S., Lu, S., Yin, Z., Li, X., Yin, L., Zheng, W.: Adapting Feature Selection Algorithms for the Classification of Chinese Texts. Systems 11(9), 483 (2023)

    Article  CAS  Google Scholar 

  17. Dai, W., Zhou, X., Li, D., Zhu, S., Wang, X.: Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics. IEEE Trans. Industr. Inf. 18(4), 2331–2341 (2022)

    Article  Google Scholar 

  18. Chen, Z., Wang, X.: Decentralized computation offloading for multi-user mobile edge computing: A deep reinforcement learning approach. EURASIP J. Wirel. Commun. Netw. 2020(1), 1–21 (2020)

    Article  CAS  Google Scholar 

  19. Wang, Q., Dai, W., Zhang, C., Zhu, J., Ma, X.: A Compact Constraint Incremental Method for Random Weight Networks and Its Application. IEEE Trans. Neural.  Netw. Learn Syst. (2023)

  20. He, W., Guo, S., Guo, S., Qiu, X., Qi, F.: Joint DNN partition deployment and resource allocation for delay-sensitive deep learning inference in IoT. IEEE Internet Things J. 7(10), 9241–9254 (2020)

    Article  Google Scholar 

  21. Li, L., Yao, L.: Fault Tolerant Control of Fuzzy Stochastic Distribution Systems With Packet Dropout and Time Delay. IEEE Trans. Autom. Sci. Eng. (2023)

  22. Guo, Y., Zhang, C., Wang, C.: Jia, X, Towards Public Verifiable and Forward-Privacy Encrypted Search by Using Blockchain. IEEE Trans. Dependable Secure Comput. 20(3), 2111–2126 (2023)

    Article  Google Scholar 

  23. Levchuk, G., Pattipati, K., Serfaty, D., Fouse, A., McCormack, R.: Active inference in multiagent systems: context-driven collaboration and decentralized purpose-driven team adaptation. In Artificial Intelligence for the Internet of Everything,  pp. 67–85. Academic Press (2019)

  24. Li, C., Dong, M., Xin, X., Li, J., Chen, X., Ota, K.: Efficient Privacy-preserving in IoMT with Blockchain and Lightweight Secret Sharing. IEEE Internet of Things Journal (2023)

  25. Cheng, B., Zhu, D., Zhao, S.: Chen, J, Situation-Aware IoT Service Coordination Using the Event-Driven SOA Paradigm. IEEE Trans. Netw. Serv. Manage. 13(2), 349–361 (2016)

    Article  Google Scholar 

  26. Dai, X., Xiao, Z., Jiang, H., Alazab, M., Lui, J.C.S., Dustdar, S.: Liu, J, Task Co-Offloading for D2D-Assisted Mobile Edge Computing in Industrial Internet of Things. IEEE Trans. Industr. Inf. 19(1), 480–490 (2023)

    Article  Google Scholar 

  27. Chen, Y., Zhang, N., Zhang, Y., Chen, X., Wu, W., Shen, X.: Energy efficient dynamic offloading in mobile edge computing for internet of things. IEEE Transactions on Cloud Computing 9(3), 1050–1060 (2019)

    Article  Google Scholar 

  28. Jiang, H., Dai, X., Xiao, Z., Iyengar, A. K.: Joint Task Offloading and Resource Allocation for Energy-Constrained Mobile Edge Computing. IEEE Trans. Mob. Comput. (2022)

  29. Dai, X., Xiao, Z., Jiang, H., Lui, J. C. S.: UAV-Assisted Task Offloading in Vehicular Edge Computing Networks. IEEE Trans. Mob. Comput. (2023)

  30. Wang, Q., Hu, J., Wu, Y., Zhao, Y.: Output synchronization of wide-area heterogeneous multi-agent systems over intermittent clustered networks. Inf. Sci. 619, 263–275 (2023)

    Article  Google Scholar 

  31. Peivandizadeh, A., Molavi, B.: Compatible Authentication and Key Agreement Protocol for Low Power and Lossy Network in Iot Environment. Available at SSRN 4454407 (2023)

  32. Aghakhani, S., Larijani, A., Sadeghi, F., Martín, D., Shahrakht, A.A.: A Novel Hybrid Artificial Bee Colony-Based Deep Convolutional Neural Network to Improve the Detection Performance of Backscatter Communication Systems. Electronics 12(10), 2263 (2023)

    Article  Google Scholar 

  33. Wang, Y., Wang, X., Ariffin, M.M., Abolfathi, M., Alqhatani, A., Almutairi, L.: Attack detection analysis in software-defined networks using various machine learning method. Comput. Electr. Eng. 108, 108655 (2023)

    Article  Google Scholar 

  34. Darabi, N., Hashem, M.B., Bandyopadhyay, S., Trivedi, A.R.: April. Exploiting Programmable Dipole Interaction in Straintronic Nanomagnet Chains for Ising Problems. In 2023 24th International Symposium on Quality Electronic Design (ISQED) 1–1 IEEE (2023)

  35. Tayebati, S., Cho, K.T.: A hybrid machine learning framework for clad characteristics prediction in metal additive manufacturing. (2023). arXiv preprint arXiv:2307.01872

  36. Mokari, H., Firouzmand, E., Sharifi, I., Doustmohammadi, A.: May. Deception attack detection and resilient control in platoon of smart vehicles. In 2022 30th International Conference on Electrical Engineering (ICEE) 29–35 IEEE (2022)

  37. Ghadermazi, P., Chan, S.H.J.: Microbial Interactions from a New Perspective: Reinforcement Learning Reveals New Insights into Microbiome Evolution. bioRxiv 2023–05 (2023)

  38. Mou, J., Gao, K., Duan, P., Li, J., Garg, A., Sharma, R, A.: Machine Learning Approach for Energy-Efficient Intelligent Transportation Scheduling Problem in a Real-World Dynamic Circumstances. IEEE Trans. Intell. Transp. Syst. (2022)

  39. Zhang, L., Sun, C., Cai, G., Koh, L. H.: Charging and discharging optimization strategy for electric vehicles considering elasticity demand response. eTransportation, 18, 100262 (2023)

  40. Liu, Z., Feng, J., Uden, L.: Technology opportunity analysis using hierarchical semantic networks and dual link prediction. Technovation 128, 102872 (2023)

    Article  Google Scholar 

  41. Liu,  J., Fan, C., Peng, Y., Du, J., Wang, Z., Chu, C.: Emergent Leader-follower Relationship in Networked Multiagent Systems. SCIENCE CHINA Information Sciences (2023)

  42. Lu, J., Osorio, C.: A Probabilistic Traffic-Theoretic Network Loading Model Suitable for Large-Scale Network Analysis. Transportation Science 52(6), 1509–1530 (2018)

    Article  Google Scholar 

  43. Gao, J., Wu, D., Yin, F., Kong, Q., Xu, L., Cui, S.: MetaLoc: Learning to Learn Wireless Localization. IEEE J. Sel. Areas Commun. (2023)

  44. Wu, Q., Fang, J., Zeng, J., Wen, J.: Luo, F, Monte Carlo Simulation-Based Robust Workflow Scheduling for Spot Instances in Cloud Environments. Tsinghua Science and Technology 29(1), 112–126 (2024)

    Article  Google Scholar 

  45. Luo, R., Peng, Z., Hu, J., Ghosh, B.: K, Adaptive optimal control of affine nonlinear systems via identifier–critic neural network approximation with relaxed PE conditions. Neural Netw. 167, 588–600 (2023)

    Article  PubMed  Google Scholar 

  46. Chen, J., Xu, M., Xu, W., Li, D., Peng, W., Xu, H.: A Flow Feedback Traffic Prediction Based on Visual Quantified Features. IEEE Trans. Intell. Transp. Syst. 24(9), 10067–10075 (2023)

    Article  ADS  Google Scholar 

  47. Chen, J., Wang, Q., Peng, W., Xu, H., Li, X., Xu, W.: Disparity-Based Multiscale Fusion Network for Transportation Detection. IEEE Trans. Intell. Transp. Syst. 23(10), 18855–18863 (2022)

    Article  Google Scholar 

  48. Li, K., Ji, L., Yang, S., Li, H., Liao, X.: Couple-Group Consensus of Cooperative-Competitive Heterogeneous Multiagent Systems: A Fully Distributed Event-Triggered and Pinning Control Method. IEEE Transactions on Cybernetics 52(6), 4907–4915 (2022)

    Article  PubMed  Google Scholar 

Download references

Funding

This work was supported by the Science and Technology Research Program of Chongqing Municipal Education Commission of China. (Grant No. KJQN202004002). Also, this work was sponsored in part by Anhui Provincial Quality Engineering Project for Higher Education Institutions (2020-2023 "Logistics Facilities, Equipment, and Information Technology", the original high-quality offline open course (2020kfkc535).

Author information

Authors and Affiliations

  1. Academy of Engineering, Fundamental Informatics and Information Technology, People’s Friendship University of Russia, Moscow, 117198, Russia

    Shangshang Wang

  2. College of Electronical and Information Engineering, Chongqing Open University, Chongqing, 400052, China

    Yuqin Jing

  3. Department of Business Administration, Lu’an Vocational and Technical College, Lu’an, Anhui, 2370000, China

    Kezhu Wang

  4. School of Education Science, Nanjing Normal University, Nanjing, Jiangsu, 210097, China

    Xue Wang

  5. Smart Campus Management Office, Nanjing Audit University, Nanjing, Jiangsu, 211815, China

    Xue Wang

Authors
  1. Shangshang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuqin Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kezhu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ShangshangWang: Conceptualization, Methodology, Formal analysis, Supervision, Writing—original draft, Writing—review & editing.

Yuqin Jing: Writing—original draft, Writing—review & editing, Investigation, Data Curation, Validation, Resources, Writing—review & editing.

Kezhu Wang: Writing—original draft, Writing—review & editing, Investigation, Data Curation, Validation, Resources, Writing—review & editing.

Xue Wang: Investigation, Data Curation, Validation, Resources, Writing—review & editing.

Corresponding author

Correspondence to Xue Wang.

Ethics declarations

Ethics Approval and Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Jing, Y., Wang, K. et al. Multi-Agent Systems for Collaborative Inference Based on Deep Policy Q-Inference Network. J Grid Computing 22, 38 (2024). https://doi.org/10.1007/s10723-024-09750-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10723-024-09750-w

Keywords

Search

Navigation