An Approach to Building High-Quality Tag Hierarchies from Crowdsourced Taxonomic Tag Pairs

  • Fahad Almoqhim
  • David E. Millard
  • Nigel Shadbolt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8238)


Building taxonomies for web content is costly. An alternative is to allow users to create folksonomies, collective social classifications. However, folksonomies lack structure and their use for searching and browsing is limited. Current approaches for acquiring latent hierarchical structures from folksonomies have had limited success. We explore whether asking users for tag pairs, rather than individual tags, can increase the quality of derived tag hierarchies. We measure the usability cost, and in particular cognitive effort required to create tag pairs rather than individual tags. Our results show that when applied to tag pairs a hierarchy creation algorithm (Heymann-Benz) has superior performance than when applied to individual tags, and with little impact on usability. However, the resulting hierarchies lack richness, and could be seen as less expressive than those derived from individual tags. This indicates that expressivity, not usability, is the limiting factor for collective tagging approaches aimed at crowdsourcing taxonomies.


Folksonomies Taxonomies Collective Intelligence Social Information Processing Social Metadata Tag similarities 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    O’Reilly, T.: What is web 2.0: design patterns and business models for the next generation of software (2005), (June 20, 2013)
  2. 2.
    Vander Wal, T.: Folksonomy Coinage and Definition (2007),
  3. 3.
    Strohmaier, M., Helic, D., Benz, D., Körner, C., Kern, R.: Evaluation of Folksonomy Induction Algorithms. ACM Transactions on Intelligent Systems and Technology 3(4), Article 74 (2012)Google Scholar
  4. 4.
    Golder, S., Huberman, B.: Usage patterns of collaborative tagging systems. Journal of Information Science 32(2), 198–208 (2006)CrossRefGoogle Scholar
  5. 5.
    Guy, M., Tonkin, E.: Tidying up tags. D-Lib Magazine 12(1) (January 2006) ISSN 1082-9873 Google Scholar
  6. 6.
    Heymann, P., Garcia-Molinay, H.: Collaborative Creation of Communal Hierarchical Taxonomies in Social Tagging Systems. InfoLab Technical Report, Stanford (2006)Google Scholar
  7. 7.
    Plangprasopchok, A., Lerman, K., Getoor, L.: From saplings to a tree: Integrating structured metadata via relational affinity propagation. In: Proceedings of the AAAI Workshop on Statistical Relational AI, Menlo Park, CA, USA (2010)Google Scholar
  8. 8.
    Benz, D., Hotho, A., Stutzer, S.: Semantics made by you and me: Self-emerging ontologies cancapture the diversity of shared knowledge. In: 2nd Web Science Conference (WebSci 2010), Raleigh, NC, USA (2010)Google Scholar
  9. 9.
    Laniado, D., Eynard, D., Colombetti, M.: Using WordNet to turn a folksonomy into a hierarchy of concepts. In: 4th Italian Semantic Web Workshop: Semantic Web Application and Perspectives, Bari, Italy, pp. 192–201 (2007)Google Scholar
  10. 10.
    Mika, P.: Ontologies are us: A unified model of social networks and semantics. Web Semantics: Science, Services and Agents on the World Wide Web 5(1), 5–15 (2007)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Zheng, H., Wu, X., Yu, Y.: Enriching WordNet with Folksonomies. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 1075–1080. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Lin, H., Davis, J.: Computational and crowdsourcing methods for extracting ontological structure from folksonomy. In: Aroyo, L., Antoniou, G., Hyvönen, E., ten Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC 2010, Part II. LNCS, vol. 6089, pp. 472–477. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  13. 13.
    Solskinnsbakk, G., Gulla, J.: Mining tag similarity in folksonomies. In: 3rd International Workshop on Search and Mining User-generated Contents (SMUC 2011), Glasgow, Scotland, pp. 53–60 (2011)Google Scholar
  14. 14.
    Schmitz, C., Hotho, A., Jäschke, R., Stumme, G.: Mining association rules in folksonomies. In : 10th IFCS Conference: Studies in Classification, Data Analysis and Knowledge Organization, Ljubljana, Slovenia, pp.261-270 (2006) Google Scholar
  15. 15.
    Schmitz, P.: Inducing ontology from flickr tags. In: Collaborative Web Tagging Workshop at WWW2006, Edinburgh, Scotland (2006)Google Scholar
  16. 16.
    Sanderson, M., Croft, B.: Deriving concept hierarchies from text. In: 22nd ACM Conference of the Special Interest Group in Information Retrieval, Berkeley, California, USA, pp. 206–213 (1999)Google Scholar
  17. 17.
    Frey, B., Dueck, D.: Clustering by passing messages between data points. Science 315(5814), 972–976 (2007)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Plangprasopchok, A., Lerman, K.: Constructing Folksonomies from User-Specified Relations on Flickr. In: 18th International World Wide Web Conference, Madrid, Spain, pp. 781–790 (2009)Google Scholar
  19. 19.
    Andrews, P., Pane, J.: Sense induction in folksonomies: a review. Artificial Intelligence Review, 1-28 (2012)Google Scholar
  20. 20.
    Brooke, J.: SUS-A quick and dirty usability scale. Usability Evaluation in Industry, 189–194 (1996)Google Scholar
  21. 21.
    Tullis, T., Stetson, J.: A comparison of questionnaires for assessing website usability. In: Usability Professional Association Conference, Minneapolis, USA, pp. 1–12 (2004)Google Scholar
  22. 22.
    Bangor, A., Kortum, P., Miller, J.: An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction 24(6), 574–594 (2008)CrossRefGoogle Scholar
  23. 23.
    Dellschaft, K., Staab, S.: On how to perform a gold standard based evaluation of ontology learning. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L.M. (eds.) ISWC 2006. LNCS, vol. 4273, pp. 228–241. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  24. 24.
    McLellan, S., Muddimer, A., Peres, S.: The Effect of Experience on System Usability Scale Ratings. Journal of Usability Studies 7(2), 56–67 (2012)Google Scholar
  25. 25.
    Halpin, H., Robu, V., Shepherd, H.: The Complex Dynamics of Collaborative Tagging. In: 6th International Conference on the World Wide Web, Banff, Canada, pp. 211–220 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Fahad Almoqhim
    • 1
  • David E. Millard
    • 1
  • Nigel Shadbolt
    • 1
  1. 1.Electronics and Computer ScienceUniversity of SouthamptonSouthamptonUnited Kingdom

Personalised recommendations