Helping Users Sort Faster with Adaptive Machine Learning Recommendations

  • Steven M. Drucker
  • Danyel Fisher
  • Sumit Basu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6948)


Sorting and clustering large numbers of documents can be an overwhelming task: manual solutions tend to be slow, while machine learning systems often present results that don’t align well with users’ intents. We created and evaluated a system for helping users sort large numbers of documents into clusters. iCluster has the capability to recommend new items for existing clusters and appropriate clusters for items. The recommendations are based on a learning model that adapts over time – as the user adds more items to a cluster, the system’s model improves and the recommendations become more relevant. Thirty-two subjects used iCluster to sort hundreds of data items both with and without recommendations; we found that recommendations allow users to sort items more rapidly. A pool of 161 raters then assessed the quality of the resulting clusters, finding that clusters generated with recommendations were of statistically indistinguishable quality. Both the manual and assisted methods were substantially better than a fully automatic method.


Mixed initiative interactions adaptive user interfaces information interfaces interactive clustering machine learning 

Supplementary material

Electronic Supplementary Material (17,914 KB)


  1. 1.
    Agarawala, A., Balakrishnan, R.: Keepin’ it real: pushing the desktop metaphor with physics, piles and the pen. In: Proc. CHI 2006 (2006)Google Scholar
  2. 2.
    Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)CrossRefGoogle Scholar
  3. 3.
    Basu, S., Fisher, D., Drucker, S., Lu, H.: Assisting Users with Clustering Tasks by Combining Metric Learning and Classification. In: AAAI 2010 (2010)Google Scholar
  4. 4.
    Bauer, D., Fastrez, P., Hollan, J.: Spatial Tools for Managing Personal Information Collections. In: Proc. HICSS 2005 (2005)Google Scholar
  5. 5.
    Blei, D.M., Ng, A., Jordan, M.I., Lafferty, J.: Latent Dirichlet allocation. Journal of Machine Learning Research 3, 993–1022 (2003)zbMATHGoogle Scholar
  6. 6.
    Chang, J., Boyd-Graber, J., Gerrish, S., Wang, C., Blei, D.: Reading Tea Leaves: How Humans Interpret Topic Models. In: Proceedings of NIPS (2009)Google Scholar
  7. 7.
    Cohn, D., Caruana, R.: Semi-Supervised Clustering: Incorporating User Feedback to Improve Cluster Utility. In: Proceedings of the Conf. on Artificial Intelligence. AAAI Press, Menlo Park (2000)Google Scholar
  8. 8.
    Czerwinski, M., Dumais, S., Robertson, G., Dziadosz, S., Tiernan, S., van Dantzich, M.: Visualizing implicit queries for information management and retrieval. In: Proc CHI 1999 (1999)Google Scholar
  9. 9.
    DesJardins, M., MacGlashan, J., Ferraioli, J.: Interactive Visual Clustering. In: Proc. of the Int’l Conf. on Intelligent User Interfaces. ACM Press, New York (2007)Google Scholar
  10. 10.
    Dourish, P., Lamping, J., Rodden, T.: Building Bridges: Customisation and Mutual Intelligibility in Shared Category Management. In: Proc. GROUP 1999 (1999)Google Scholar
  11. 11.
    Jones, W., Dumais, S.: The spatial metaphor for user interfaces: experimental tests of reference by location versus name. ACM Trans. Inf. Syst. 4(1) (January 1986)Google Scholar
  12. 12.
    Malone, T.: How do people organize their desks?: Implications for the design of office information systems. ACM Trans. Office Info. Sys. 1(1), 99–112 (1983)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Mander, R., Salomon, G., Wong, Y.Y.: A “pile” metaphor for supporting casual organization of information. In: Proc. CHI 1992 (1992)Google Scholar
  14. 14.
    Pirolli, P., Schank, P., Hearst, M., Diehl, C.: Scatter/Gather Browsing Communicates the Topic Structure of a Very Large Text Collection. In: Proc. CHI (1996)Google Scholar
  15. 15.
    Robertson, G., Czerwinski, M., Larson, K., Robbins, D.C., Thiel, D., van Dantzich, M.: Data mountain: using spatial memory for document management. In: Proc. UIST 1998 (1998)Google Scholar
  16. 16.
    Watanabe, N., Washida, M., Igarashi, T.: Bubble clusters: an interface for manipulating spatial aggregation of graphical objects. In: Proc. UIST 2007 (2007)Google Scholar
  17. 17.
    Whittaker, S., Hirschberg, J.: The character, value, and management of personal paper archives. ACM Trans. on Comp.-Human Int. 8(2), 150–170 (2001)CrossRefGoogle Scholar
  18. 18.
    Zeng, H.-J., He, Q.-C., Chen, Z., Ma, W.-Y., Ma, J.: Leaning to Cluster Web Search Results. In: Proceedings of SIGIR (2004)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Steven M. Drucker
    • 1
  • Danyel Fisher
    • 1
  • Sumit Basu
    • 1
  1. 1.Microsoft ResearchRedmondUSA

Personalised recommendations