Electronic Markets

, Volume 27, Issue 2, pp 111–124 | Cite as

Let the crowd be my peers? How researchers assess the prospects of social peer review

  • Christian MattEmail author
  • Christian Hoerndlein
  • Thomas Hess
Research Paper


While Internet technologies have provided social networks for researchers as more open means to make their work available to other scholars, the traditionally closed, peer review-based publishing process has remained nearly untouched. We ask researchers about their intention to go one step further and use social peer review (SPR), which enables them to directly publish their work within a web-based social network, where, instead of the traditional pre-publication peer review, it can be evaluated and critiqued by the entire academic community. Based on a sample of 1429 international scholars from various fields and by drawing upon adoption and institutional theory, this study seeks to identify scientists’ motivational drivers for engaging in this new forms of scholarly communication. We find that the adoption of SPR is driven more by extrinsic factors than by researchers’ intrinsic motivation or normative influences to make science more open. Further challenges for SPR are low scores on the most relevant performance criteria, as well as low acceptance by established scientists. However, rather than a substitute, SPR is well perceived as a possible supplement to the traditional peer-based review system.


Social peer review Scholarly communication Social computing 

JEL classification

3.010: Adoption 3.540: Virtual Communities 3.550: Web 2.0 2.20.9: Surveys 



We like to thank two anonymous reviewers as well as the special issue editors for their valuable feedback. We also thank Springer Science+Business Media for their support in the sample acquisition phase.


  1. Abrahamson E., & Bartner L. R. (1990). When do bandwagon diffusions roll? How far do they go? And when do they roll backwards?: A computer simulation. Academy of Management Proceedings.Google Scholar
  2. Ajjan, H., & Hartshorne, R. (2008). Investigating faculty decisions to adopt web 2.0 technologies: theory and empirical tests. The Internet and Higher Education, 11(2), 71–80.CrossRefGoogle Scholar
  3. Alberts, B., Hanson, B., & Kelner, K. L. (2008). Reviewing peer review. Science, 321(5885), 15.CrossRefGoogle Scholar
  4. Berger, B., Matt, C., Steininger, D. M., & Hess, T. (2015). It is not just about competition with “free”: differences between content formats in consumer preferences and willingness to pay. Journal of Management Information Systems, 32(3), 105–128.CrossRefGoogle Scholar
  5. Buhl, H. U., Müller, G., Fridgen, G., & Röglinger, M. (2012). Business and information systems engineering: a complementary approach to information systems-what we can learn from the past and may conclude from present reflection on the future. Journal of the Association for Information Systems, 13(4), 236–253.CrossRefGoogle Scholar
  6. Cenfetelli, R. T., & Bassellier, G. (2009). Interpretation of formative measurement in information systems research. MIS Quarterly, 33(4), 689–707.CrossRefGoogle Scholar
  7. Chesbrough, H. W. (2006). Open innovation: The new imperative for creating and profiting from technology. Boston: Harvard Business Press.Google Scholar
  8. Chin, W. W. (1998). The partial least squares approach to structural equation modeling. In G. A. Marcoulides (Ed.), Modern methods for business research (pp. 295–336). Mahwah: Erlbaum.Google Scholar
  9. Chong A. Y. L., & Ngai E. W. (2013). What influences travellers' adoption of a location-based social media service for their travel planning?.In Proceedings of the 2013 Pacific Asia Conference on Information Systems (PACIS 2013). Jeju Island.Google Scholar
  10. Christensen, C. (2013). The innovator's dilemma: When new technologies cause great firms to fail. Boston: Harvard Business Review Press.Google Scholar
  11. Currie, W. (2009). Contextualising the IT artefact: towards a wider research agenda for IS using institutional theory. Information Technology & People, 22(1), 63–77.CrossRefGoogle Scholar
  12. Delamothe, T., & Smith, R. (2004). Open access publishing takes off: the dream is now achievable. British Medical Journal, 328(7430), 1–3.CrossRefGoogle Scholar
  13. Dimaggio, P., & Powell, W. (1983). The iron cage revisited: institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147–160.CrossRefGoogle Scholar
  14. Fisher, R. J. (1993). Social desirability bias and the validity of indirect questioning. Journal of Consumer Research, 20(2), 303–315.CrossRefGoogle Scholar
  15. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50.CrossRefGoogle Scholar
  16. Franzoni, C., & Sauermann, H. (2014). Crowd science: the organization of scientific research in open collaborative projects. Research Policy, 43(1), 1–20.CrossRefGoogle Scholar
  17. Friedland, R., & Alford, R. R. (1991). Bringing society back in: Symbols, practices and institutional contradictions. In W. W. Powell & P. J. DiMaggio (Eds.), The new institutionalism in organizational analysis (pp. 232–263). Chicago: University of Chicago Press.Google Scholar
  18. Friesike, S., Widenmayer, B., Gassmann, O., & Schildhauer, T. (2015). Opening science: towards an agenda of open science in academia and industry. The Journal of Technology Transfer, 40(4), 581–601.CrossRefGoogle Scholar
  19. Haeussler, C. (2011). Information-sharing in academia and the industry: a comparative study. Research Policy, 40(1), 105–122.CrossRefGoogle Scholar
  20. Hair, J. F., Ringle, C. M., & Sarstedt, M. (2011). PLS-SEM: indeed a silver bullet. Journal of Marketing Theory and Practice, 19(2), 139–152.CrossRefGoogle Scholar
  21. Hess, T. (2014). What is a media company? A reconceptualization for the online world. International Journal on Media Management, 16(1), 3–8.CrossRefGoogle Scholar
  22. Hess, T., & Hoerndlein, C. (2015). Incentives and more: four aspects that every innovation in scholarly communication needs to consider—answer to “Kingsley/Kennan: open access: the whipping boy for problems in scholarly publishing”. Communications of the Association of Information Systems, 37(18), 373–377.Google Scholar
  23. Hess, T., & Matt, C. (2013). The internet and the value chains of the media industry. In S. Diehl & M. Karmasin (Eds.), Media and convergence management (pp. 37–55). Berlin/Heidelberg: Springer.CrossRefGoogle Scholar
  24. Hess, T., Matt, C., Benlian, A., & Wiesböck, F. (2016). Options for formulating a digital transformation strategy. MIS Quarterly Executive, 15(2), 123–139. Google Scholar
  25. Hoerndlein C., Benlian A., Hess T. (2012). Institutional influences in individual-level innovation adoption outside organizational contexts: A scale development study. In Proceedings of the 33th International Conference on Information Systems (ICIS2012). Orlando.Google Scholar
  26. Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.CrossRefGoogle Scholar
  27. Katz, M. L., & Shapiro, C. (1985). Network externalities, competition, and compatibility. The American Economic Review, 75(3), 424–440.Google Scholar
  28. Kingsley, D. A., & Kennan, M. A. (2015). Open access: the whipping boy for problems in scholarly publishing. Communications of the Association for Information Systems, 37(18), 329–350.Google Scholar
  29. Li, L., Wang, Y., Liu, G., Wang, M., & Wu, X. (2015). Context-aware reviewer assignment for trust enhanced peer review. PloS One, 10(6), e0130493.CrossRefGoogle Scholar
  30. Lincoln, B. (1995). Book review “the new institutionalism in organizational research” (edited by Walter W. Powell and Paul J. Dimaggio). Social Forces, 73(3), 1147–1148.Google Scholar
  31. Mann, F., von Walter, B., Hess, T., & Wigand, R. T. (2009). Open access publishing in science. Communications of the ACM, 52(3), 135–139.CrossRefGoogle Scholar
  32. Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago/London: University of Chicago Press.Google Scholar
  33. Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: formal structure as myth and ceremony. American Journal of Sociology, 83(2), 340–363.CrossRefGoogle Scholar
  34. Mignerat, M., & Rivard, S. (2009). Positioning the institutional perspective in information systems research. Journal of Information Technology, 24(4), 369–391.CrossRefGoogle Scholar
  35. Nistor, N., Baltes, B., Dascălu, M., Mihăilă, D., Smeaton, G., & Trăuşan-Matu, Ş. (2014). Participation in virtual academic communities of practice under the influence of technology acceptance and community factors. A learning analytics application. Computers in Human Behavior, 34, 339–344.CrossRefGoogle Scholar
  36. Orlikowski, W. J., & Barley, S. R. (2001). Technology and institutions: what can research on information technology and research on organizations learn from each other? MIS Quarterly, 25(2), 145–165.CrossRefGoogle Scholar
  37. Parameswaran, M., & Whinston, A. B. (2007). Research issues in social computing. Journal of the Association for Information Systems, 8(6), 336–350.CrossRefGoogle Scholar
  38. Partha, D., & David, P. A. (1994). Toward a new economics of science. Research Policy, 23(5), 487–521.CrossRefGoogle Scholar
  39. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903.CrossRefGoogle Scholar
  40. Polanyi, M. (1962). The republic of science: its political and economic theory. Minerva, 1(1), 54–73.CrossRefGoogle Scholar
  41. Ponte, D., & Simon, J. (2011). Scholarly communication 2.0: exploring researchers' opinions on web 2.0 for scientific knowledge creation, evaluation and dissemination. Serials Review, 37(3), 149–156.CrossRefGoogle Scholar
  42. Priem J., & Hemminger B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social web. First Monday, 15(7).
  43. Ringle, C. M., Sarstedt, M., & Straub, D. (2012). A critical look at the use of PLS-SEM in MIS Quarterly. MIS Quarterly, 36(1), iii–xiv.Google Scholar
  44. Sauermann, H., & Roach, M. (2014). Not all scientists pay to be scientists: Phds’ preferences for publishing in industrial employment. Research Policy, 43(1), 32–47.CrossRefGoogle Scholar
  45. Scott, W. R. (2008). Institutions and organizations: Ideas, interests, and identities (3rd ed.). Los Angeles: Sage Publications.Google Scholar
  46. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178–182.CrossRefGoogle Scholar
  47. Squazzoni, F., Bravo, G., & Takács, K. (2013). Does incentive provision increase the quality of peer review? An experimental study. Research Policy, 42(1), 287–294.CrossRefGoogle Scholar
  48. Stephan, P. E. (1996). The economics of science. Journal of Economic Literature, 34(3), 1199–1235.Google Scholar
  49. Straub, D. W. (2008). Editor's comments: why do top journals reject good papers? MIS Quarterly, 32(3), iii–vii.CrossRefGoogle Scholar
  50. Suls, J., & Martin, R. (2009). The air we breathe: a critical look at practices and alternatives in the peer-review process. Perspectives on Psychological Science, 4(1), 40–50.CrossRefGoogle Scholar
  51. Sun, M. (1989). Peer review comes under peer review. Science, 244(4907), 910–912.CrossRefGoogle Scholar
  52. TechCrunch (2015). Academia pushes a new kind of peer review for research with ‘sessions’. Accessed 22 Dec 2015.
  53. Teo, H. H., Wei, K. K., & Benbasat, I. (2003). Predicting intention to adopt interorganizational linkages: an institutional perspective. MIS Quarterly, 27(1), 19–49.CrossRefGoogle Scholar
  54. Thelwall, M., & Kousha, K. (2015). Researchgate: disseminating, communicating, and measuring scholarship? Journal of the Association for Information Science and Technology, 66(5), 876–889.CrossRefGoogle Scholar
  55. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly, 27(3), 425–478.CrossRefGoogle Scholar
  56. Venkatesh, V., Thong, J. Y., & Xu, X. (2012). Consumer acceptance and use of information technology: extending the unified theory of acceptance and use of technology. MIS Quarterly, 36(1), 157–178.Google Scholar
  57. Wang, F.-Y., Carley, K. M., Zeng, D., & Mao, W. (2007). Social computing: from social informatics to social intelligence. IEEE Intelligent Systems, 22(2), 79–83.CrossRefGoogle Scholar
  58. Ware, M. (2011). Peer review: recent experience and future directions. New Review of Information Networking, 16(1), 23–53.CrossRefGoogle Scholar
  59. Yan, Y., Davison, R. M., & Mo, C. (2013). Employee creativity formation: the roles of knowledge seeking, knowledge contributing and flow experience in web 2.0 virtual communities. Computers in Human Behavior, 29(5), 1923–1932.CrossRefGoogle Scholar
  60. Zucker, L. G. (1977). The role of institutionalization in cultural persistence. American Sociological Review, 42(5), 726–743.CrossRefGoogle Scholar

Copyright information

© Institute of Applied Informatics at University of Leipzig 2017

Authors and Affiliations

  • Christian Matt
    • 1
    Email author
  • Christian Hoerndlein
    • 2
  • Thomas Hess
    • 2
  1. 1.Institute of Information SystemsUniversity of BernBernSwitzerland
  2. 2.Institute for Information Systems and New MediaLMU MunichMunichGermany

Personalised recommendations