Skip to main content

Goldilocks: Learning Pattern-Based Task Assignment in Mobile Crowdsensing

  • Conference paper
  • First Online:
Quality, Reliability, Security and Robustness in Heterogeneous Systems (QShine 2019)


Mobile crowdsensing (MCS) depends on mobile users to collect sensing data, whose quality highly depends on the expertise/experience of the users. It is critical for MCS to identify right persons for a given sensing task. A commonly-used strategy is to “teach-before-use”, i.e., training users with a set of questions and selecting a subset of users who have answered the questions correctly the most of times. This method has large room for improvement if we consider users’ learning curve during the training process. As such, we propose an interactive learning pattern recognition framework, Goldilocks, that can filter users based on their learning patterns. Goldilocks uses an adaptive teaching method tailored for each user to maximize her learning performance. At the same time, the teaching process is also the selecting process. A user can thus be safely excluded as early as possible from the MCS tasks later on if her performance still does not match the desired learning pattern after the training period. Experiments on real-world datasets show that compared to the baseline methods, Goldilocks can identify suitable users to obtain more accurate and more stable results for multi-categories classification problems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions


  1. 1.

    The term is meant to emphasize that our ultimate goal is identifying the right people for right tasks rather than evaluating people’s general learning capability.

  2. 2.

    A round means that the user answers a question and then is told whether her answer is correct or not as well as what the ground truth is.

  3. 3.

    In our later experiments, to make Eqs. (4) and (3) have the same range values, we \(\nu = 0.011\) and \(c = -0.018\).

  4. 4.

    This function is just one possible candidate, and people can adopt other desired shape here.


  1. Ahlgren, B., Hidell, M., Ngai, E.C.H.: Internet of Things for smart cities: interoperability and open data. IEEE Internet Comput. 20(6), 52–56 (2016)

    Article  Google Scholar 

  2. Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., Allahbakhsh, M.: Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions. ACM Comput. Surv. (CSUR) 51(1), 7 (2018)

    Article  Google Scholar 

  3. Du, J., Ling, C.X.: Active teaching for inductive learners. In: Proceedings of the 2011 SIAM International Conference on Data Mining, pp. 851–861. SIAM (2011)

    Google Scholar 

  4. Ebbinghaus, H.: Memory: a contribution to experimental psychology. Ann. Neurosci. 20(4), 155 (2013)

    Article  Google Scholar 

  5. Fiandrino, C., Kantarci, B., Anjomshoa, F., Kliazovich, D., Bouvry, P., Matthews, J.: Sociability-driven user recruitment in mobile crowdsensing Internet of Things platforms. In: 2016 IEEE Global Communications Conference (GLOBECOM), pp. 1–6. IEEE (2016)

    Google Scholar 

  6. Ganti, R.K., Ye, F., Lei, H.: Mobile crowdsensing: current state and future challenges. IEEE Commun. Mag. 49(11), 32–39 (2011)

    Article  Google Scholar 

  7. Grantcharov, T.P., Bardram, L., Funch-Jensen, P., Rosenberg, J.: Learning curves and impact of previous operative experience on performance on a virtual reality simulator to test laparoscopic surgical skills. Am. J. Surg. 185(2), 146–149 (2003)

    Article  Google Scholar 

  8. Guo, B., Liu, Y., Wu, W., Yu, Z., Han, Q.: Activecrowd: a framework for optimized multitask allocation in mobile crowdsensing systems. IEEE Trans. Hum.-Mach. Syst. 47(3), 392–403 (2017)

    Article  Google Scholar 

  9. Johns, E., Mac Aodha, O., Brostow, G.J.: Becoming the expert-interactive multi-class machine teaching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2616–2624 (2015)

    Google Scholar 

  10. Jones, V., Jo, J.H.: Ubiquitous learning environment: an adaptive teaching system using ubiquitous technology. In: Beyond the Comfort Zone: Proceedings of the 21st ASCILITE Conference, vol. 468, p. 474. Perth, Western Australia (2004)

    Google Scholar 

  11. Karaliopoulos, M., Telelis, O., Koutsopoulos, I.: User recruitment for mobile crowdsensing over opportunistic networks. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 2254–2262. IEEE (2015)

    Google Scholar 

  12. Li, H., Li, T., Wang, Y.: Dynamic participant recruitment of mobile crowd sensing for heterogeneous sensing tasks. In: 2015 IEEE 12th International Conference on Mobile Ad Hoc and Sensor Systems, pp. 136–144. IEEE (2015)

    Google Scholar 

  13. Liu, C.H., Zhang, B., Su, X., Ma, J., Wang, W., Leung, K.K.: Energy-aware participant selection for smartphone-enabled mobile crowd sensing. IEEE Syst. J. 11(3), 1435–1446 (2017)

    Article  Google Scholar 

  14. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  15. Patil, K.R., Zhu, J., Kopeć, Ł., Love, B.C.: Optimal teaching for limited-capacity human learners. In: Advances in Neural Information Processing Systems, pp. 2465–2473 (2014)

    Google Scholar 

  16. Shah, N., Zhou, D., Peres, Y.: Approval voting and incentives in crowdsourcing. In: International Conference on Machine Learning, pp. 10–19 (2015)

    Google Scholar 

  17. Singla, A., Bogunovic, I., Bartók, G., Karbasi, A., Krause, A.: On actively teaching the crowd to classify. In: NIPS Workshop on Data Driven Education (2013)

    Google Scholar 

  18. Singla, A., Bogunovic, I., Bartók, G., Karbasi, A., Krause, A.: Near-optimally teaching the crowd to classify. In: ICML, no. 2 in 1, p. 3 (2014)

    Google Scholar 

  19. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD birds-200-2011 dataset. California Institute of Technology (2011)

    Google Scholar 

  20. Wang, E., Yang, Y., Wu, J., Liu, W., Wang, X.: An efficient prediction-based user recruitment for mobile crowdsensing. IEEE Trans. Mob. Comput. 17(1), 16–28 (2018)

    Article  Google Scholar 

  21. Xu, Q., Zheng, R.: When data acquisition meets data analytics: a distributed active learning framework for optimal budgeted mobile crowdsensing. In: IEEE INFOCOM 2017-IEEE Conference on Computer Communications, pp. 1–9. IEEE (2017)

    Google Scholar 

  22. Yang, S., Han, K., Zheng, Z., Tang, S., Wu, F.: Towards personalized task matching in mobile crowdsensing via fine-grained user profiling. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 2411–2419. IEEE (2018)

    Google Scholar 

  23. Yin, X., Goudriaan, J., Lantinga, E.A., Vos, J., Spiertz, H.J.: A flexible sigmoid function of determinate growth. Ann. Bot. 91(3), 361–371 (2003)

    Article  Google Scholar 

  24. Zhang, X.Y., Wang, S., Yun, X.: Bidirectional active learning: a two-way exploration into unlabeled and labeled data set. IEEE Trans. Neural Netw. Learn. Syst. 26(12), 3034–3044 (2015)

    Article  MathSciNet  Google Scholar 

  25. Zhao, L., Sukthankar, G., Sukthankar, R.: Incremental relabeling for active learning with noisy crowdsourced annotations. In: 2011 IEEE Third International Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on Social Computing, pp. 728–733. IEEE (2011)

    Google Scholar 

  26. Zhu, J.: Machine teaching for Bayesian learners in the exponential family. In: Advances in Neural Information Processing Systems, pp. 1905–1913 (2013)

    Google Scholar 

  27. Zhu, X., Ghahramani, Z., Lafferty, J.D.: Semi-supervised learning using gaussian fields and harmonic functions. In: Proceedings of the 20th International conference on Machine learning (ICML 2003), pp. 912–919 (2003)

    Google Scholar 

Download references


This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) under grant No. SPG 494083-16.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Kui Wu .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jiang, J., Dai, Y., Wu, K., Zheng, R. (2020). Goldilocks: Learning Pattern-Based Task Assignment in Mobile Crowdsensing. In: Chu, X., Jiang, H., Li, B., Wang, D., Wang, W. (eds) Quality, Reliability, Security and Robustness in Heterogeneous Systems. QShine 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 300. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-38818-8

  • Online ISBN: 978-3-030-38819-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics