Goldilocks: Learning Pattern-Based Task Assignment in Mobile Crowdsensing

  • Jinghan Jiang
  • Yiqin Dai
  • Kui WuEmail author
  • Rong Zheng
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 300)


Mobile crowdsensing (MCS) depends on mobile users to collect sensing data, whose quality highly depends on the expertise/experience of the users. It is critical for MCS to identify right persons for a given sensing task. A commonly-used strategy is to “teach-before-use”, i.e., training users with a set of questions and selecting a subset of users who have answered the questions correctly the most of times. This method has large room for improvement if we consider users’ learning curve during the training process. As such, we propose an interactive learning pattern recognition framework, Goldilocks, that can filter users based on their learning patterns. Goldilocks uses an adaptive teaching method tailored for each user to maximize her learning performance. At the same time, the teaching process is also the selecting process. A user can thus be safely excluded as early as possible from the MCS tasks later on if her performance still does not match the desired learning pattern after the training period. Experiments on real-world datasets show that compared to the baseline methods, Goldilocks can identify suitable users to obtain more accurate and more stable results for multi-categories classification problems.


Mobile crowdsensing Learning pattern recognition Task assignment 



This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) under grant No. SPG 494083-16.


  1. 1.
    Ahlgren, B., Hidell, M., Ngai, E.C.H.: Internet of Things for smart cities: interoperability and open data. IEEE Internet Comput. 20(6), 52–56 (2016)CrossRefGoogle Scholar
  2. 2.
    Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., Allahbakhsh, M.: Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions. ACM Comput. Surv. (CSUR) 51(1), 7 (2018)CrossRefGoogle Scholar
  3. 3.
    Du, J., Ling, C.X.: Active teaching for inductive learners. In: Proceedings of the 2011 SIAM International Conference on Data Mining, pp. 851–861. SIAM (2011)Google Scholar
  4. 4.
    Ebbinghaus, H.: Memory: a contribution to experimental psychology. Ann. Neurosci. 20(4), 155 (2013)CrossRefGoogle Scholar
  5. 5.
    Fiandrino, C., Kantarci, B., Anjomshoa, F., Kliazovich, D., Bouvry, P., Matthews, J.: Sociability-driven user recruitment in mobile crowdsensing Internet of Things platforms. In: 2016 IEEE Global Communications Conference (GLOBECOM), pp. 1–6. IEEE (2016)Google Scholar
  6. 6.
    Ganti, R.K., Ye, F., Lei, H.: Mobile crowdsensing: current state and future challenges. IEEE Commun. Mag. 49(11), 32–39 (2011)CrossRefGoogle Scholar
  7. 7.
    Grantcharov, T.P., Bardram, L., Funch-Jensen, P., Rosenberg, J.: Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am. J. Surg. 185(2), 146–149 (2003)CrossRefGoogle Scholar
  8. 8.
    Guo, B., Liu, Y., Wu, W., Yu, Z., Han, Q.: Activecrowd: a framework for optimized multitask allocation in mobile crowdsensing systems. IEEE Trans. Hum.-Mach. Syst. 47(3), 392–403 (2017)CrossRefGoogle Scholar
  9. 9.
    Johns, E., Mac Aodha, O., Brostow, G.J.: Becoming the expert-interactive multi-class machine teaching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2616–2624 (2015)Google Scholar
  10. 10.
    Jones, V., Jo, J.H.: Ubiquitous learning environment: an adaptive teaching system using ubiquitous technology. In: Beyond the Comfort Zone: Proceedings of the 21st ASCILITE Conference, vol. 468, p. 474. Perth, Western Australia (2004)Google Scholar
  11. 11.
    Karaliopoulos, M., Telelis, O., Koutsopoulos, I.: User recruitment for mobile crowdsensing over opportunistic networks. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 2254–2262. IEEE (2015)Google Scholar
  12. 12.
    Li, H., Li, T., Wang, Y.: Dynamic participant recruitment of mobile crowd sensing for heterogeneous sensing tasks. In: 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems, pp. 136–144. IEEE (2015)Google Scholar
  13. 13.
    Liu, C.H., Zhang, B., Su, X., Ma, J., Wang, W., Leung, K.K.: Energy-aware participant selection for smartphone-enabled mobile crowd sensing. IEEE Syst. J. 11(3), 1435–1446 (2017)CrossRefGoogle Scholar
  14. 14.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  15. 15.
    Patil, K.R., Zhu, J., Kopeć, Ł., Love, B.C.: Optimal teaching for limited-capacity human learners. In: Advances in Neural Information Processing Systems, pp. 2465–2473 (2014)Google Scholar
  16. 16.
    Shah, N., Zhou, D., Peres, Y.: Approval voting and incentives in crowdsourcing. In: International Conference on Machine Learning, pp. 10–19 (2015)Google Scholar
  17. 17.
    Singla, A., Bogunovic, I., Bartók, G., Karbasi, A., Krause, A.: On actively teaching the crowd to classify. In: NIPS Workshop on Data Driven Education (2013)Google Scholar
  18. 18.
    Singla, A., Bogunovic, I., Bartók, G., Karbasi, A., Krause, A.: Near-optimally teaching the crowd to classify. In: ICML, no. 2 in 1, p. 3 (2014)Google Scholar
  19. 19.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD birds-200-2011 dataset. California Institute of Technology (2011)Google Scholar
  20. 20.
    Wang, E., Yang, Y., Wu, J., Liu, W., Wang, X.: An efficient prediction-based user recruitment for mobile crowdsensing. IEEE Trans. Mob. Comput. 17(1), 16–28 (2018)CrossRefGoogle Scholar
  21. 21.
    Xu, Q., Zheng, R.: When data acquisition meets data analytics: a distributed active learning framework for optimal budgeted mobile crowdsensing. In: IEEE INFOCOM 2017-IEEE Conference on Computer Communications, pp. 1–9. IEEE (2017)Google Scholar
  22. 22.
    Yang, S., Han, K., Zheng, Z., Tang, S., Wu, F.: Towards personalized task matching in mobile crowdsensing via fine-grained user profiling. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 2411–2419. IEEE (2018)Google Scholar
  23. 23.
    Yin, X., Goudriaan, J., Lantinga, E.A., Vos, J., Spiertz, H.J.: A flexible sigmoid function of determinate growth. Ann. Bot. 91(3), 361–371 (2003)CrossRefGoogle Scholar
  24. 24.
    Zhang, X.Y., Wang, S., Yun, X.: Bidirectional active learning: a two-way exploration into unlabeled and labeled data set. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3034–3044 (2015)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Zhao, L., Sukthankar, G., Sukthankar, R.: Incremental relabeling for active learning with noisy crowdsourced annotations. In: 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, pp. 728–733. IEEE (2011)Google Scholar
  26. 26.
    Zhu, J.: Machine teaching for Bayesian learners in the exponential family. In: Advances in Neural Information Processing Systems, pp. 1905–1913 (2013)Google Scholar
  27. 27.
    Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International conference on Machine learning (ICML 2003), pp. 912–919 (2003)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of VictoriaVictoriaCanada
  2. 2.Department of Computer ScienceNational University of Defense TechnologyChangshaChina
  3. 3.Department of Computing and SoftwareMcMaster UniversityHamiltonCanada

Personalised recommendations