Advertisement

Agent Simulation of Peer Review: The PR-1 Model

  • Francisco Grimaldo
  • Mario Paolucci
  • Rosaria Conte
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7124)

Abstract

Peer review lies at the core of current scientific research. It is composed of a set of social norms, practices and processes that connect the abstract scientific method with the society of people that apply the method. As a social construct, peer review should be understood by building theory-informed models and comparing them with data collection. Both these activities are evolving in the era of automated computation and communication: new modeling tools and large bodies of data become available to the interested scholar. In this paper, starting from abstract principles, we develop and present a model of the peer review process. We also propose a working implementation of a subset of the general model, developed with Jason, a framework that implements the Belief-Desire-Intention (BDI) model for multi agent systems. After running a set of simulations, varying the initial distribution of reviewer skill, we compare the aggregates that our simplified model produces with recent findings, showing how for some parameter choice the model can generate data in qualitative agreement with measures.

Keywords

Beta Distribution Program Committee Peer Review Good Paper Paper Quality 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bonabeau, E.: Agent-based modeling: methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences of the United States of America 99(suppl. 3), 7280–7287 (2002)CrossRefGoogle Scholar
  2. 2.
    Bordini, R.H., Hübner, J.F.: Jason (March 2007), http://jason.sourceforge.net/
  3. 3.
    Brabazon, T.: The google effect: Googling, blogging, wikis and the flattening of expertise. Libri 56, 157–167 (2006)CrossRefGoogle Scholar
  4. 4.
    Casati, F., Marchese, M., Ragone, A., Turrini, M.: Is peer review any good? a quantitative analysis of peer review. Technical report, Ingegneria e Scienza dell’Informazione, University of Trento (2009)Google Scholar
  5. 5.
    Conte, R., Castelfranchi, C.: Cognitive Social Action. UCL Press, London (1995)Google Scholar
  6. 6.
    Kuhn, T.S.: The Structure of Scientific Revolutions, 3rd edn. University of Chicago Press (December 1996)Google Scholar
  7. 7.
    Neff, B.D., Olden, J.D.: Is peer review a game of chance? BioScience 56(4), 333–340 (2006)CrossRefGoogle Scholar
  8. 8.
    Osman, N., Sabater, J., Sierra, C.: Simulating research behaviour (this volume)Google Scholar
  9. 9.
    Paolucci, M., Balke, T., Conte, R., Eymann, T., Marmo, S.: Review of internet user-oriented reputation applications and application layer networks. Social Science Research Network Working Paper Series (September 2009)Google Scholar
  10. 10.
    Paolucci, M., Picascia, S., Marmo, S.: Electronic reputation systems. In: Handbook of Research on Web 2.0, 3.0, and X.0, ch. 23, pp. 411–429. IGI Global (2010)Google Scholar
  11. 11.
    Rao, A.S.: AgentSpeak(L): BDI Agents Speak Out in a Logical Computable Language. In: Perram, J., Van de Velde, W. (eds.) MAAMAW 1996. LNCS (LNAI), vol. 1038, pp. 42–55. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  12. 12.
    Thurner, S., Hanel, R.: Peer-review in a world with rational scientists: Toward selection of the average (August 2010)Google Scholar
  13. 13.
    Wainberg, J., Kida, T., Smith, J.F.: Stories vs. statistics: The impact of anecdotal data on accounting decision making. Social Science Research Network Working Paper Series (March 2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Francisco Grimaldo
    • 1
  • Mario Paolucci
    • 2
  • Rosaria Conte
    • 2
  1. 1.Departament d’InformàticaUniversitat de ValènciaBurjassotSpain
  2. 2.Institute of Cognitive Sciences and Technologies (ISTC)Italian National Research Council (CNR)RomaItaly

Personalised recommendations