Skip to main content

Exploiting Classical Bibliometrics of CSCW: Classification, Evaluation, Limitations, and the Odds of Semantic Analytics

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNPSE,volume 7946)

Abstract

The purpose of this study is to conduct a bibliometric and taxonomy-based analysis of the field of CSCW to map its recent evolution at a quantitative and qualitative level. A model for semantic analytics and social evaluation is also discussed with emphasis on the hypothesis of putting crowds into the loop of bibliography classification process to improve the current labor-intensive, time-consuming, unrepeatable and sometimes subjective task of scientometricians. A total of 1,480 publications were carefully reviewed and subjected to scientometric data analysis methods and techniques. Analyzed parameters included document orientation, deviation from trend in the total number of citations, and publication activity by author’s affiliation country. A semantic classification of 541 publications allows identifying growing trends and lacking research indicators. At a human-centered perspective, limitations are unfilled in the limitative analytical spectrum, laborious and time-consuming processes of data seeking, gathering, cataloguing and analysis, subjective results at a taxonomic level, lack of more bibliographic data analytics perspectives, and absence of human-centered results concerning cognitive aspects in meta-knowledge research practices. Hypotheses are suggested towards a crowd-enabled model for bibliography evaluation in order to understand the ways as humans and machines can work cooperatively and massively on scientific data to solve complex problems.

Keywords

  • Bibliographic data analytics
  • crowd-enabled model
  • CSCW
  • HCI
  • human computation
  • scientometrics
  • semantics
  • taxonomy

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-642-39062-3_9
  • Chapter length: 20 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-39062-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Quinn, A.J., Bederson, B.B.: Human Computation: a survey and taxonomy of a growing field. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2011), pp. 1403–1412 (2011)

    Google Scholar 

  2. Schmidt, K.: The concept of ‘work’ in CSCW. Computer Supported Cooperative Work 20(4-5), 341–401 (2011)

    CrossRef  Google Scholar 

  3. Grudin, J., Poltrock, S.E.: Taxonomy and theory in Computer Supported Cooperative Work. In: Kozlowski, S.W.J. (ed.) The Oxford Handbook of Industrial and Organizational Psychology. Oxford University Press, New York (2012)

    Google Scholar 

  4. Chi, E.H., Pirolli, P., Suh, B., Kittur, A., Pendleton, B., Mytkowicz, T.: Augmented social cognition: using Social Web technology to enhance the ability of groups to remember, think, and reason. In: Proceedings of the 35th SIGMOD International Conference on Management of Data Providence (2009)

    Google Scholar 

  5. Nichols, D.M., Twidale, M.B., Cunningham, S.J.: Metadatapedia: a proposal for aggregating metadata on data archiving. In: Proceedings of the ACM iConference 2012, pp. 370–376 (2012)

    Google Scholar 

  6. Jacovi, M., Soroka, V., Gilboa-Freedman, G., Ur, S., Shahar, E., Marmasse, N.: The chasms of CSCW: a citation graph analysis of the CSCW conference. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work, CSCW 2006 (2006)

    Google Scholar 

  7. Sajid, N.A., Ali, T., Afzal, M.T., Ahmad, M., Qadir, M.A.: Exploiting reference section to classify paper’s topics. In: Proceedings of the ACM Conference on Management of Emergent Digital Ecosystems (MEDES 2011), pp. 220–225 (2011)

    Google Scholar 

  8. Bartneck, C., Hu, J.: Scientometric analysis of the CHI proceedings. In: Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 699–708 (2009)

    Google Scholar 

  9. Nicolaisen, J.: Bibliometrics and citation analysis: from the Science Citation Index to cybermetrics. Journal of the American Society for Information Science and Technology 61(1), 205–207 (2010)

    CrossRef  Google Scholar 

  10. Mikki, S.: Google Scholar compared to Web of Science: a literature review. Nordic Journal of Information Literacy in Higher Education 1(1), 41–51 (2009)

    Google Scholar 

  11. Mittleman, D.D., Briggs, R.O., Murphy, J., Davis, A.: Toward a taxonomy of groupware technologies. In: Briggs, R.O., Antunes, P., de Vreede, G.-J., Read, A.S. (eds.) CRIWG 2008. LNCS, vol. 5411, pp. 305–317. Springer, Heidelberg (2008)

    CrossRef  Google Scholar 

  12. Holsapple, C.W., Luo, W.: A citation analysis of influences on collaborative computing research. Computer Supported Cooperative Work 12(3), 351–366 (2003)

    CrossRef  Google Scholar 

  13. Serenko, A., Bontis, N.: Meta-review of knowledge management and intellectual capital literature: citation impact and research productivity rankings. Knowledge and Process Management 11(3), 185–198 (2004)

    CrossRef  Google Scholar 

  14. Horn, D.B., Finholt, T.A., Birnholtz, J.P., Motwani, D., Jayaraman, S.: Six degrees of Jonathan Grudin: a social network analysis of the evolution and impact of CSCW research. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW 2004), pp. 582–591 (2004)

    Google Scholar 

  15. Convertino, G., George, T., Councill, I.: Mapping the intellectual landscape of CSCW. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (2006)

    Google Scholar 

  16. Wania, C.E., Atwood, M.E., McCain, K.W.: How do design and evaluation interrelate in HCI research? In: Proceedings of the 6th Conference on Designing Interactive Systems, pp. 90–98 (2006)

    Google Scholar 

  17. Oulasvirta, A.: A bibliometric exercise for SIGCHI Conference on Human Factors in Computing Systems (2006), http://www.hiit.fi/node/290 (retrieved in January 2013)

  18. Wainer, J., Barsottini, C.: Empirical research in CSCW – a review of the ACM/CSCW conferences from 1998 to 2004. Journal of the Brazilian Computer Society 13, 27–36 (2007)

    CrossRef  Google Scholar 

  19. Henry, N., Goodell, H., Elmqvist, N., Fekete, J.-D.: 20 years of four HCI conferences: a visual exploration. International Journal of Human Computer Interaction 23(3), 239–285 (2007)

    CrossRef  Google Scholar 

  20. Kaye, J.: Some statistical analyses of CHI. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2009), pp. 2585–2594 (2009)

    Google Scholar 

  21. Wainer, J., Barsottini, C.G.N., Lacerda, D., de Marco, L.: Empirical evaluation in computer science research published by ACM. Information and Software Technology 51(6), 1081–1085 (2009)

    CrossRef  Google Scholar 

  22. Bartneck, C., Hu, J.: The fruits of collaboration in a multidisciplinary field. Scientometrics 85(1), 41–52 (2010)

    CrossRef  Google Scholar 

  23. Antunes, P., Pino, J.: A review of CRIWG research. In: Kolfschoten, G., Herrmann, T., Lukosch, S. (eds.) CRIWG 2010. LNCS, vol. 6257, pp. 1–15. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  24. Fitzpatrick, G., Ellingsen, G.: A review of 25 years of CSCW research in healthcare: contributions, challenges and future agendas. In: Computer Supported Cooperative Work, pp. 1–57 (2012)

    Google Scholar 

  25. Garfield, E.: Citation indexes to science: a new dimension in documentation through association of ideas. Science 122, 108–111 (1955)

    CrossRef  Google Scholar 

  26. Kitchenham, B., Brereton, O.P., Budgen, D., Turner, M., Bailey, J., Linkman, S.: Systematic literature reviews in software engineering – a systematic literature review. Information and Software Technology 51, 7–15 (2009)

    CrossRef  Google Scholar 

  27. Cruz, A., Correia, A., Paredes, H., Fonseca, B., Morgado, L., Martins, P.: Towards an overarching classification model of CSCW and groupware: a socio-technical perspective. In: Herskovic, V., Hoppe, H.U., Jansen, M., Ziegler, J. (eds.) CRIWG 2012. LNCS, vol. 7493, pp. 41–56. Springer, Heidelberg (2012)

    CrossRef  Google Scholar 

  28. Haglund, L., Olsson, P.: The impact on university libraries of changes in information behavior among academic researchers: a multiple case study. The Journal of Academic Librarianship 34(1), 52–59 (2008)

    CrossRef  Google Scholar 

  29. Jacsó, P.: Google Scholar revisited. Online Information Review 32(1), 102–114 (2008)

    CrossRef  Google Scholar 

  30. Barkhuus, L., Rode, J.: From mice to men – 24 years of evaluation in CHI. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2007) (2007)

    Google Scholar 

  31. Priem, J., Hemminger, B.: Scientometrics 2.0: toward new metrics of scholarly impact on the Social Web. First Monday 15(7) (2010)

    Google Scholar 

  32. Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D., Jacobs, D.: Dynamic changes in motivation in collaborative citizen-science projects. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (2012)

    Google Scholar 

  33. Santos, J., Paredes, H., Fonseca, B., Correia, A.: Preliminary study of an observatory for CSCW scientific publications. In: Proceedings of the 18th International Conference of European University Information Systems organisation (EUNIS 2012): E-science and E-repositories. Virtual Libraries, Virtual Laboratories (2012)

    Google Scholar 

  34. Kittur, A., Chi, E., Suh, B.: Crowdsourcing for usability: using micro-task markets for rapid, remote, and low-cost user measurements. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2008) (2008)

    Google Scholar 

  35. Schneider, D., Moraes, K., de Souza, J.M., Esteves, M.G.P.: CSCWD: five characters in search of crowds. In: Proceedings of the 16th International Conference on Computer Supported Cooperative in Design (CSCWD 2012), pp. 634–641 (2012)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Correia, A., Fonseca, B., Paredes, H. (2013). Exploiting Classical Bibliometrics of CSCW: Classification, Evaluation, Limitations, and the Odds of Semantic Analytics. In: Holzinger, A., Ziefle, M., Hitz, M., Debevc, M. (eds) Human Factors in Computing and Informatics. SouthCHI 2013. Lecture Notes in Computer Science, vol 7946. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39062-3_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39062-3_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39061-6

  • Online ISBN: 978-3-642-39062-3

  • eBook Packages: Computer ScienceComputer Science (R0)