Skip to main content
SpringerLink
Log in

Adaptive task recommendation based on reinforcement learning in mobile crowd sensing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Adaptive task recommendation in Mobile crowd sensing (MCS) is a challenging problem, mainly because perceptual tasks are spatio-temporal in nature and worker preferences are dynamically changing. Although there have been some approaches to address the dynamics of task recommendation, these approaches suffer from several problems. First, they only learn the worker’s past preferences and cannot cope with the situation where the worker’s preferences may change in the next moment, and they only consider the current optimal recommendation instead of global optimization. Second, existing methods do not scale efficiently to the arrival of new workers or tasks, requiring the entire model to be retrained. To address these issues, we propose an adaptive task recommendation method (ATRec) based on reinforcement learning. Specifically, we formalize the adaptive task recommendation problem for each target worker as an interactive Markov decision process (MDP). Then, we use an improved matrix decomposition technique to construct worker-personalized latent factor states based on information such as task content and spatio-temporal context, enabling us to use a unified MDP to learn optimal strategies for different workers. After that, we design an adaptive update algorithm (AUA) based on Deep Q Network (DQN) to more accurately learn the dynamic changes of workers’ preferences to adaptively update the task recommendation list of workers. In addition, we propose a personalized dimension reduction method (PDR) to reduce the size of the task set. Through comprehensive experimental results and analysis, we demonstrate the effectiveness of the ATRec approach. Compared with existing methods, ATRec can better solve the problem of adaptive task recommendation, and can more accurately predict workers’ preferences and make recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Availability of data

The data that support the findings of this study are available from https://www.lucami.org/en/research/ldos-comoda-dataset/ and https://grouplens.org/datasets/movielens/

Notes

  1. https://www.lucami.org/en/research/ldos-comoda-dataset/

  2. https://grouplens.org/datasets/movielens/

References

  1. Ganti RK, Ye F, Lei H (2011) Mobile crowdsensing: current state and future challenges. IEEE Commun Mag 49(11):32–39

    Article  Google Scholar 

  2. Montori F, Bedogni L, Bononi L (2017) A collaborative internet of things architecture for smart cities and environmental monitoring. IEEE Int Things J 5(2):592–605

    Article  Google Scholar 

  3. Jovanović S, Jovanović M, Škorić T, Jokić S, Milovanović B, Katzis K, Bajić D (2019) A mobile crowd sensing application for hypertensive patients. Sensors 19(2):400

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  4. Huang J, Kong L, Dai H-N, Ding W, Cheng L, Chen G, Jin X, Zeng P (2020) Blockchain-based mobile crowd sensing in industrial systems. IEEE Trans Industrial Inf 16(10):6553–6563

    Article  Google Scholar 

  5. Kim K, Zabihi H, Kim H, Lee U (2017) Trailsense: a crowdsensing system for detecting risky mountain trail segments with walking pattern analysis. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 1(3):1–31

    Google Scholar 

  6. Wan J, Liu J, Shao Z, Vasilakos AV, Imran M, Zhou K (2016) Mobile crowd sensing for traffic prediction in internet of vehicles. Sensors 16(1):88

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  7. Wang J, Wang L, Wang Y, Zhang D, Kong L (2018) Task allocation in mobile crowd sensing: state-of-the-art and future opportunities. IEEE Int Things J 5(5):3747–3757

    Article  Google Scholar 

  8. Yang G, Li Y, Song Y, Li J, He X, Kong L, Liu M (2020) A real-time recommendation algorithm for task allocation in mobile crowd sensing. In: Wireless algorithms, systems, and applications: 15th international conference, WASA 2020, Qingdao, China, September 13–15, 2020, Proceedings, Part I 15, Springer, pp 640–652

  9. Wang J, Liu J, Zhao Z, Zhao G (2021) A task recommendation framework for heterogeneous mobile crowdsensing. J Supercomput 77:12121–12142

    Article  Google Scholar 

  10. Wang J, Zhang Z, Zhao G (2023) Task recommendation method for fusion of multi-view social relationship learning and reasoning in the mobile crowd sensing system. Comput Commun 206:60–72

    Article  Google Scholar 

  11. Wang J, Yan Y, Zhao G (2023) Task recommendation method combining multimodal cognition and collaboration in mobile crowdsensing systems. Comput Netw 229:109796

    Article  Google Scholar 

  12. Tang W, Zhang K, Ren J, Zhang Y, Shen XS (2020) Privacy-preserving task recommendation with win-win incentives for mobile crowdsourcing. Inf Sci 527:477–492

    Article  Google Scholar 

  13. Wang J, Hao S, Zhao G (2023) Cssa-based collaborative optimization recommendation of users in mobile crowdsensing. Peer-to-Peer Network Appl 1–15

  14. Zhao G, Wang X, Wang J, Liu J (2023) Task recommendation for mobile crowd sensing system based on multi-view user dynamic behavior prediction. Peer-to-Peer Netw Appl 1–15

  15. Zhang J, Zhang X (2021) Multi-task allocation in mobile crowd sensing with mobility prediction. IEEE Trans Mobile Comput 1081–1094

  16. Huang Y, Chen H, Ma G, Lin K, Ni Z, Yan N, Wang Z (2021) Opat: Optimized allocation of time-dependent tasks for mobile crowdsensing. IEEE Trans Ind Inf 18(4):2476–2485

    Article  Google Scholar 

  17. Yin B, Li J, Wei X (2021) Rational task assignment and path planning based on location and task characteristics in mobile crowdsensing. IEEE Trans Comput Social Syst 9(3):781–793

    Article  Google Scholar 

  18. Shen X, Xu D, Song L, Zhang Y (2023) Heterogeneous multi-project multi-task allocation in mobile crowdsensing using an ensemble fireworks algorithm. Appl Soft Comput 110571

  19. Shen X, Chen Q, Pan H, Song L, Guo Y (2022) Variable speed multi-task allocation for mobile crowdsensing based on a multi-objective shuffled frog leaping algorithm. Appl Soft Comput 127:109330

    Article  Google Scholar 

  20. Dai W, Wang Y, Ma J, Jin Q (2020) Btr: a feature-based bayesian task recommendation scheme for crowdsourcing system. IEEE Trans Comput Social Syst 7(3):780–789

    Article  Google Scholar 

  21. Song S, Liu Z, Li Z, Xing T, Fang D (2020) Coverage-oriented task assignment for mobile crowdsensing. IEEE Int Things J 7(8):7407–7418

    Article  Google Scholar 

  22. Yucel F, Yuksel M, Bulut E (2020) Qos-based budget constrained stable task assignment in mobile crowdsensing. IEEE Trans Mobile Comput 20(11):3194–3210

    Article  Google Scholar 

  23. Yuen M-C, King I, Leung K-S (2011) Task matching in crowdsourcing. In: 2011 International conference on internet of things and 4th international conference on cyber, physical and social computing, IEEE, pp 409–412

  24. Karaliopoulos M, Koutsopoulos I, Titsias M (2016) First learn then earn: optimizing mobile crowdsensing campaigns through data-driven user profiling. In: Proceedings of the 17th ACM international symposium on mobile Ad Hoc networking and computing, pp 271–280

  25. Schnitzer S, Neitzel S, Rensing C (2017) From task classification towards similarity measures for recommendation in crowdsourcing systems. arXiv:1707.06562

  26. Wang Z, Zhao J, Hu J, Zhu T, Wang Q, Ren J, Li C (2020) Towards personalized task-oriented worker recruitment in mobile crowdsensing. IEEE Trans Mobile Comput 20(5):2080–2093

    Article  Google Scholar 

  27. Zhang C, Zhu L, Xu C, Ni J, Huang C, Shen X (2021) Location privacy-preserving task recommendation with geometric range query in mobile crowdsensing. IEEE Trans Mobile Comput 21(12):4410–4425

    Article  Google Scholar 

  28. Zhao Z, Wang Y, Wang H (2021) Sdn-based cross-domain cooperative method for trusted nodes recommendation in mobile crowd sensing. Peer-to-Peer Netw Appl 14:3793–3805

    Article  Google Scholar 

  29. Anand S, Ram A, Mishra MK (2021) Recommendation of tasks with multiple incentives in mobile crowdsensing. In: 2021 6th International conference on communication and electronics systems (ICCES), IEEE, pp 1–4

  30. Yang G, Li Y, He X, Song Y, Wang J, Liu M (2021) Profile-free and real-time task recommendation in mobile crowdsensing. IEEE Trans Comput Social Syst 8(6):1311–1322

    Article  Google Scholar 

  31. Wang Y, Cai Z, Zhan Z-H, Zhao B, Tong X, Qi L (2020) Walrasian equilibrium-based multiobjective optimization for task allocation in mobile crowdsourcing. IEEE Trans Comput Soc Syst 7(4):1033–1046

    Article  Google Scholar 

  32. Wu F, Yang S, Zheng Z, Tang S, Chen G (2020) Fine-grained user profiling for personalized task matching in mobile crowdsensing. IEEE Trans Mobile Comput 20(10):2961–2976

    Article  Google Scholar 

  33. Chen S, Zhao X, Liu J, Gao G, Du Y (2022) Social-network-assisted task recommendation algorithm in mobile crowd sensing. In: Proceedings of the 7th international conference on information and education innovations, pp 136–142

  34. Zhao Y, Deng L, Zheng K (2023) Adataskrec: an adaptive task recommendation framework in spatial crowdsourcing. ACM Trans Inf Syst

  35. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press

  36. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

    Article  ADS  CAS  PubMed  Google Scholar 

  37. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

    Article  ADS  CAS  PubMed  Google Scholar 

  38. Shani G, Heckerman D, Brafman RI, Boutilier C (2005) An MDP-based recommender system. J Mach Learn Res 6(9)

  39. Zhao X, Xia L, Zhang L, Ding Z, Yin D, Tang J (2018) Deep reinforcement learning for page-wise recommendations. In: Proceedings of the 12th ACM conference on recommender systems, pp 95–103

  40. Zhao X, Zhang L, Ding Z, Xia L, Tang J, Yin D (2018) Recommendations with negative feedback via pairwise deep reinforcement learning. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1040–1048

  41. Zheng G, Zhang F, Zheng Z, Xiang Y, Yuan NJ, Xie X, Li Z (2018) Drn: a deep reinforcement learning framework for news recommendation. In: Proceedings of the 2018 world wide web conference, pp 167–176

  42. Hu B, Shi C, Liu J (2017) Playlist recommendation based on reinforcement learning. In: Intelligence science I: Second IFIP TC 12 international conference, ICIS 2017, Shanghai, China, October 25-28, 2017, Proceedings 2, Springer, pp 172–182

  43. Lu T, Pál D, Pál M (2010) Contextual multi-armed bandits. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceedings, pp 485–492

  44. Slivkins A (2011) Contextual bandits with similarity information. In: Proceedings of the 24th annual conference on learning theory, JMLR Workshop and Conference Proceedings, pp 679–702

  45. Tekin C, Schaar M (2015) Contextual online learning for multimedia content aggregation. IEEE Trans Multimed 17(4):549–561

    Article  Google Scholar 

  46. Li L, Chu W, Langford J, Schapire RE (2010) A contextual-bandit approach to personalized news article recommendation. In: Proceedings of the 19th international conference on world wide web, pp 661–670

  47. Wang L, Wang C, Wang K, He X (2017) Biucb: a contextual bandit algorithm for cold-start and diversified recommendation. In: 2017 IEEE International conference on big knowledge (ICBK), IEEE, pp 248–253

  48. Zou L, Xia L, Ding Z, Song J, Liu W, Yin D (2019) Reinforcement learning to optimize long-term user engagement in recommender systems. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2810–2818

  49. Chen H (2021) A DQN-based recommender system for item-list recommendation. In: 2021 IEEE International conference on big data (Big Data), IEEE, pp 5699–5702

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61802257 and 61602305; and by the Natural Science Foundation of Shanghai under Grant 18ZR1426000 and 19ZR1477600; and by the Social Livelihood Planning Project of Nan-tong Science and Technology Bureau MS12021060; and by the Opening Foundation of Agile and Intelligent Computing Key Laboratory of Si-chuan Province.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Shanghai for Science and Technology, 516 Jungong Rd., Shanghai, 200093, People’s Republic of China

    Guisong Yang, Guochen Xie & Jingru Wang

  2. College of Communication and Art Design, University of Shanghai for Science and Technology, 516 Jungong Rd., Shanghai, 200093, People’s Republic of China

    Xingyu He

  3. Library & Department of Computer Science and Engineering, University of Shanghai for Science and Technology, 516 Jungong Rd., Shanghai, 200093, People’s Republic of China

    Li Gao

  4. Department of Computer Science and Engineering, Peking University, 5 Yiheyuan Rd., Beijing, 100871, People’s Republic of China

    Yunhuai Liu

Authors
  1. Guisong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guochen Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingru Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xingyu He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Li Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yunhuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, provision of main ideas and analysis were performed by Guisong Yang, Xingyu He, Li Gao and Yunhuai Liu. The first draft of the manuscript was written by Guochen Xie and Jingru Wang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Li Gao.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Ethics approval

All authors contributed to the conception and design of the study. All authors read and approved the final manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, G., Xie, G., Wang, J. et al. Adaptive task recommendation based on reinforcement learning in mobile crowd sensing. Appl Intell 54, 2277–2299 (2024). https://doi.org/10.1007/s10489-023-05247-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05247-3

Keywords

Search

Navigation