Skip to main content
SpringerLink
Log in

On novel peer review system for academic journals: analysis based on social computing

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

For improving the performance and effectiveness of peer review, a novel review system is proposed, based on analysis of peer review process for academic journals under a parallel model built via Monte Carlo method. The model can simulate the review, application, and acceptance activities of the review systems, in a distributed manner. Simulation experiments are operated on two distinctive review systems. Significant advantages of the novel system are shown by the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data availability

The data in this paper is generated via simulations.

References

  1. Wager, E., Jefferson, T.: Shortcomings of peer review in biomedical journals. Learn. Publ. 14, 257–263 (2001)

    Google Scholar 

  2. Tan, Z., Cai, N., Zhou, J., et al.: On performance of peer review for academic journals: analysis based on distributed parallel system. IEEE Access 7, 19024–19032 (2019)

    Google Scholar 

  3. Resnik, D.B., Smith, E.M.: Bias and groupthink in science’s peer-review system. In: Allen, D.M., Howell, J.W. (eds.) Groupthink in Scence, pp. 99–113. Springer International Publishing (2020)

    Google Scholar 

  4. Dadkhah, M., Kahani, M., Borchardt, G.: A method for improving the integrity of peer review. Sci. Eng. Eth. 24, 1603–1610 (2018)

    Google Scholar 

  5. Prechelt, L., Graziotin, D., Fernandez, D.M.: A community’s perspective on the status and future of peer review in software engineering. Inf. Softw. Tech. 95, 75–85 (2017)

    Google Scholar 

  6. Faggion, C.M.: Improving the peer-review process from the perspective of an author and reviewer. Brit. Dent. J. 220, 167–168 (2016)

    Google Scholar 

  7. Herron, D.M.: Is expert peer review obsolete? A model suggests that post-publication reader review may exceed the accuracy of traditional peer review. Surg. Endosc. 26, 2275–2280 (2012)

    Google Scholar 

  8. Kovanis, M., Trinquart, L., Ravaud, P., et al.: Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication. Scientometrics 113, 651–671 (2017)

    Google Scholar 

  9. Birukou, A., Wakeling, J.R., Bartolini, C., et al.: Alternative to peer review: Novel approaches for research evaluation. Front. Comput. Neurosc. 5, 56 (2011)

    Google Scholar 

  10. Pros and cons of open peer review. Nat. Neurosci. 2, 197–198 (1999)

  11. Mulligan, A., Hall, L.: Peer review in a changing world: an international study measuring the attitudes of researches. J. Am. Soc. Inf. Sci. Tech. 64, 132–161 (2013)

    Google Scholar 

  12. Ortega, J.L.: Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons. Scientometrics 112, 947–962 (2017)

    Google Scholar 

  13. Liu, L., Cai, N., Tan, Z. et al.: Parallel analysis on novel peer review system for academic journals, In: Proc. 33rd Chin. Contr. Dec. Conf., pp. 2514–2519, (2021)

  14. Liu, L., Cai, N., Tan, Z. et al.: On novel peer review system for academic journals: experimental study based on social computing (2021), Preprint, https://arxiv.org/abs/2111.06099

  15. Elsen, M.B., Akhmanova, A., Behrens, T.E., et al.: Implementing a “publish, then review” model of publishing. Elife 9, e64910 (2020)

    Google Scholar 

  16. Look, H., Sparks, S.: The value of UK HEIs contribution to the publishing process: Summary report, JISC Collect., (2010)

  17. Stahel, P.F., Moore, E.E.: Peer review for biomedical publications: we can improve the system. BMC Med. 12, 179 (2014)

    Google Scholar 

  18. Zhang, J., Wang, F., Wang, X., et al.: Cyber-physical-social systems: the state of the art and perspectives. IEEE Trans. Comput. Soc. Syst. 5, 829–840 (2018)

    Google Scholar 

  19. Wang, F.: Back to the future: surrogates, mirror worlds, and parallel universes. IEEE Intelli. Syst. 26, 2–4 (2011)

    Google Scholar 

  20. James, N., Menzies, M.: Collective correlations, dynamics, and behavioural inconsistencies of the cryptocurrency market over time. Nonlin. Dyn. 107, 4001–4017 (2022)

    Google Scholar 

  21. Lai, J.W., Cheong, K.H.: Evaluation of single-prioritization voting systems in controlled collective Parrondo’s games. Nonlin. Dyn. 107, 2965–2974 (2022)

    Google Scholar 

  22. Taher, H., Avitabile, D., Desroches, M.: Bursting in a next generation neural mass model with synaptic dynamics: a slow-fast approach. Nonlin. Dyn. 108, 4261–4285 (2022)

    Google Scholar 

  23. Cai, N., Diao, C., Yan, B.: A social computing-based analysis on monogamous marriage puzzle of human. IEEE Trans. Comput. Soc. Syst. 6, 518–524 (2019)

    Google Scholar 

  24. Yang, X., Liao, L., Yang, Q., et al.: Limited-energy output formation for multiagent systems with intermittent interactions. J. Frankl. Inst. 358, 6462–6489 (2021)

    MathSciNet  MATH  Google Scholar 

  25. Sun, Y., Ji, Z., Liu, Y., et al.: On stabilizability of multi-agent systems. Automatica 142, 110491 (2022)

    MathSciNet  MATH  Google Scholar 

  26. Qu, J., Ji, Z., Shi, Y.: The graphical conditions for controllability of multiagent systems under equitable partition. IEEE Trans. Cybernet. 51, 4661–4672 (2021)

    Google Scholar 

  27. Ji, Z., Lin, H., Cao, S., et al.: The complexity in complete graphic characterizations of multiagent controllability. IEEE Trans. Cybernet. 51, 64–76 (2021)

    Google Scholar 

  28. Golosovsky, M., Solomon, S.: Stochastic dynamical model of a growing citation network based on a self-exciting point process. Phys. Rev. Lett. 109, 098701 (2012)

    Google Scholar 

  29. Candia, C., Jara-Figueroa, C., Rodriguez-Sickert, C., et al.: The universal decay of collective memory and attention. Nat. Hum. Behav. 3, 82–91 (2019)

    Google Scholar 

  30. Pan, R.K., Petersen, A.M., Pammolli, F., et al.: The memory of science: inflation, myopia, and the knowledge network. J. Informetr. 12, 656–678 (2018)

    Google Scholar 

  31. Kovanis, M., Porcher, R., Ravaud, P., et al.: Complex systems approach to scientific publication and peer-review system: development of an agent-based model calibrated with empirical journal data. Scientometrics 106, 695–715 (2016)

    Google Scholar 

  32. Allesina, S.: Modeling peer review: an agent-based approach. Ideas Ecol. Evol. 5, 27–35 (2012)

    Google Scholar 

  33. Thurner, S., Hanel, R.: Peer-review in a world with rational scientists: toward selection of the average. Eur. Phys. J. B 84, 707–711 (2011)

    Google Scholar 

  34. Righi, S., Takács, K.: The miracle of peer review and development in science: an agent-based model. Scientometrics 113, 587–607 (2017)

    Google Scholar 

  35. Grimaldo, F., Paolucci, M.: A simulation of disagreement for control of rational cheating in peer review. Adv. Complex Syst. 16, 1350004 (2013)

    MathSciNet  Google Scholar 

  36. Paolucci, M., Grimaldo, F.: Mechanism change in a simulation of peer review: from junk support to elitism. Scientometrics 99, 663–688 (2014)

    Google Scholar 

  37. Squazzoni, F., Gandelli, C.: Saint Matthew strikes again: an agent-based model of peer review and the scientific community structure. J. Informetr. 6, 265–275 (2012)

    Google Scholar 

  38. Squazzoni, F., Gandelli, C.: Opening the black-box of peer review: an agent-based model of scientist behaviour. J. Artif. Soc. Social Simul. 16, 3 (2013)

    Google Scholar 

  39. Bianchi, F., Grimaldo, F., Bravo, G., et al.: The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures. Scientometrics 116, 1401–1420 (2018)

    Google Scholar 

  40. Etkin, A.: A new method and metric to evaluate the peer review process of scholarly journals. Pub. Res. Q. 30, 23–38 (2014)

    Google Scholar 

  41. Mrowinski, M.J., Fronczak, A., Fronczak, P., et al.: Review time in peer review: quantitative analysis and modelling of editorial workflows. Scientometrics 107, 271–286 (2016)

    Google Scholar 

  42. Hak, D.J., Giannoudis, P., Mauffrey, C.: Increasing challenges for an effective peer-review process. Eur. J. Orthop. Surg. Traumatol. 26, 117–118 (2016)

    Google Scholar 

  43. Sikdar, S., Tehria, P., Marsili, M.: On the effectiveness of the scientific peer-review system: a case study of the journal of high energy physics. Int. J. Digit. Lib. 21, 93–107 (2020)

    Google Scholar 

  44. Lendak, I.: Simulation-based evaluation of a crowdsourced expert peer review system, In: 2019 IEEE Int. Conf. Pervas. Comput. Commun. Worksh, pp. 77–82, (2019)

  45. Emile, S.H.: Interactive platform for peer review: a proposal to improve the current peer review system. World J. Clin. Cases 9, 1247–1250 (2021)

    Google Scholar 

  46. T. Ghosal, R. Verma, S. Saha et al.: Can your paper evade the editors axe? Towards an AI assisted peer review system, Preprint (2018) https://arxiv.org/abs/1802.01403v1

  47. Haagsman, M., Snoek, B., Peeters, A., et al.: Examiners’ use of rubric criteria for grading bachelor theses. Assess. Eval. High. Edu. 46, 1270–1285 (2021)

    Google Scholar 

  48. Bolek, M., Bolek, C., Shopovski, J., et al.: The consistency of peer-reviewers: assessment of separate parts of the manuscripts vs final recommendations. Account. Res. (2022). https://doi.org/10.1080/08989621.2022.2030719

    Article  Google Scholar 

  49. Subramanian, N., Ramanathan, R.: A review of applications of Analytic Hierarchy Process in operations management. Int. J. Prod. Econ. 138, 215–241 (2012)

    Google Scholar 

  50. Barabási, A.L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)

    MathSciNet  MATH  Google Scholar 

  51. Crawley, M.J., Harral, J.E.: Scale dependence in plant biodiversity. Science 291, 864–898 (2001)

    Google Scholar 

  52. Bohorquez, J.C., Gourley, S., Dixon, A.R., et al.: Common ecology quantifies human insurgency. Nature 462, 911–914 (2009)

    Google Scholar 

  53. Redner, S.: How popular is your paper? An empirical study of the citation distribution. Eur. Phys. J. B 4, 131–134 (1998)

    Google Scholar 

  54. Zhou, Y., Cheng, H., Li, Q., et al.: Diversity of temporal influence in popularity prediction of scientific publications. Scientometrics 123, 383–392 (2020)

    Google Scholar 

  55. Eom, Y.H., Fortunato, S.: Characterizing and modeling citation dynamics. PLoS ONE 6, e24926 (2011)

    Google Scholar 

  56. Price, D.J.: Networks of scientific papers. Science 149, 510–515 (1965)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions that have helped to improve this paper considerably.

Funding

This work is supported by National Natural Science Foundation (NNSF) of China (Grant 61867005) and by the 2023 Project for Postgraduate Education Reform of BUPT.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China

    Li Liu & Ning Cai

  2. Library, Beijing University of Posts and Telecommunications, Beijing, China

    Qian Wang

  3. School of Computer Science and Technology, Donghua University, Shanghai, China

    Zong-Yuan Tan

Authors
  1. Li Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zong-Yuan Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ning Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ning Cai.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, L., Wang, Q., Tan, ZY. et al. On novel peer review system for academic journals: analysis based on social computing. Nonlinear Dyn 111, 11613–11627 (2023). https://doi.org/10.1007/s11071-023-08401-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-023-08401-1

Keywords

Search

Navigation