Authors of research papers often want to gain insight on how to improve the chances of their research being published. There have been several “how to” guides created for this purpose. Within its first five years, OOPSLA’s popularity grew to the point that there was a panel focused on the topic of how to get a paper accepted [1]. Mary Shaw’s tutorial/paper on how to write a good software engineering contribution was directed to an ICSE audience, but has many timeless suggestions that are still relevant in many general contexts [2]. This issue’s editorial discusses the SoSyM review process and why a paper might be rejected at the various stages of review.

As with most journals, SoSyM has a defined pipeline that captures the flow of the review for each submission. When a paper is first submitted, the Editors in Chief (EiCs) are notified. The EiCs share in the responsibilities—Bernhard Rumpe concentrates mainly on submissions for Special Issues and Expert Voices, and Jeff Gray supervises the Regular paper submissions. Assistant Editors Geri Georg and Martin Schindler assist in many of the administration needs of the submission and review process. At the initial point of the review process, a decision is made by the EiCs whether a new submission is obviously out of scope with the focus of SoSyM such that the paper should be desk rejected. If the paper is judged to be reviewable, it is then assigned to one of the approximate 50 SoSyM Editors. The assigned Editor has a deeper look at the paper to see if it is within the scope of SoSyM and whether the paper makes a substantial contribution to the modeling topics covered in SoSyM. If so, the Editor invites reviewers to provide the core content of a recommendation. The Editor is notified when the reviewers complete their evaluations, from which the Editor passes on a recommendation to the EiCs. The authors are finally notified by the EiCs regarding the result of the review, which could include rejection, or requests for a major or minor revision, or acceptance. Each reviewed submission proceeds through the pipeline for several possible iterations until a final decision can be made toward rejection or acceptance.

An increasing number of desk rejects have been issued over the past year, with the submission returned to the author without formal assignment to an Editor. There are several reasons for a desk rejection by the EiCs, and we have also observed common reasons why a paper is rejected by a SoSyM Editor after a formal review. We share our observations in the following sections. A suggested related work is an editorial by Jeff Offutt on how to get a paper rejected by the journal of Software Testing, Verification & Reliability [3].

1 Common causes for desk rejects

The purpose of a desk reject is to identify papers that have some serious flaw that would severely hinder, based on the experience of the EiCs and Editors, the potential for acceptance. Rather than place a paper in the review process and engage reviewers in their volunteered time, a desk reject helps the authors to understand immediately the issues with their paper and allows reviewers to focus on papers that have a higher probability of eventual acceptance. A desk reject can be submitted by either an EiC or an Editor for various reasons. The “Quality Guidelines” expected for SoSyM submissions are available on the SoSyM web site (http://www.sosym.org/submission). The most common causes for a desk reject include the following:

  • Out of Scope: Some authors submit a paper to SoSyM that misses the focus of the topics covered in SoSyM. The purpose of SoSyM is to publish significant research contributions in the specific area of software and systems modeling, modeling languages, model transformations and tooling. We sometimes receive submissions that are general software engineering papers that make no contribution toward modeling, per se. Those authors who have questions about whether their contribution is suitable for SoSyM should check out the “Aims and Scope” section on the journal web page (http://www.sosym.org/).

  • Modeling as a Secondary Focus with Core Contribution in Other Area: A slight variation on the previous concern about scope occurs when a submission is received that has some part of the paper adopting a form of modeling, but has the primary contribution in another area (i.e., modeling as a secondary or minor focus). Examples of papers in this category are submissions that introduce some new algorithm and use modeling to describe the algorithm, or a submission that makes a contribution to a general area of software engineering and uses traditionally known modeling concepts without any novel modeling nuance. A SoSyM paper must make a significant research contribution to the specific area of modeling, and not just use or adopt well-known modeling concepts to highlight results in some other area outside of modeling. However, we are very interested in contributions that serve as fundamental empirical studies that investigate common modeling techniques in a specific domain to understand the benefits and shortcomings in a reliably validated form.

  • Incorrect Depth and Rigor Suitable for a Journal Publication: We frequently receive submissions that are under 8 pages in length and lack clear depth in terms of the rigor and analysis of the newly proposed idea. SoSyM does not have a minimum page requirement (e.g., a contribution could in fact be described in just a few pages), but if a paper would not be accepted to MODELS, or even a MODELS workshop, it would not be appropriate to submit to SoSyM, which provides extended page space that allows an author to address a deeper discussion about a novel modeling contribution.

  • Missing Validation Discussion: A SoSyM paper does not necessarily have to contain a deep formal empirical study, as noted by Robert France (one of SoSyM’s founders) when discussing the topic of fairness in reviews with respect to evaluation within a submission [4]. However, a successful submission must provide some form of validation that convinces the reviewers of the claims made in the paper. This can come in many forms and combinations, such as (1) an empirical study on the performance, or (2) a controlled experiment of some tool approach related to the operational function compared to other similar tools, or (3) a human-based empirical evaluation, or (4) deep user survey that assesses a new modeling approach and its benefits toward users, or (5) a very detailed case study on some real system or significant extraction that provides argumentation and demonstration of the benefits of a modeling technique. Regarding the use of case studies, submissions are often desk rejected if a paper merely provides a simple example that fails to convince readers of the benefits through a rigorous argument. For examples of controlled experiments in software engineering, in general, please refer to Sjøberg et al. [5].

  • Comparison to Related Work: Discussions of the modeling contribution should also reveal limitations or current challenges of a modeling contribution and how the new contribution is situated in comparison to related work. Some papers are also desk rejected due to a weak positioning of the new contribution in the context of related work.

2 Frequent reasons for a rejection recommendation by a SoSyM Editor

We have also identified common reasons for rejection as a paper moves through the review cycle, as observed from recent SoSyM Editor recommendations based on reviewer comments. The following reasons for rejection are not unique to SoSyM and represent general advice for authors who desire to improve the success of a new scientific contribution:

  • Lack of Verbal Clarity and Poor Paper Organization: Unfortunately, some submissions have a very strong scientific contribution to share with the modeling community, but the lack of clarity and organization of the submitted manuscript makes it very challenging for a reviewer to understand. This can be due to poor organization within the paper, such that the reviewer struggles to find the core contributions of the paper in relation to the current state-of-the-art. Authors should strive to organize their paper in a coherent structure and make the contribution of the paper very explicit in the paper’s introduction. A submission that has many grammatical errors can be distracting to a reviewer such that they may miss the core contribution as they struggle through the poorly phrased text of the manuscript. We suggest that authors who are not fluent in English seek a native English speaker to proofread their paper before submission.

  • Deficient Understanding of Related Work: Frequently, the authors of a submission may miss a few references to related work, which is easy to address in a revision. However, some authors overlook or are unaware of a key segment of the related work such that it brings into question the true novelty of a newly proposed methodology or modeling approach. A total lack of understanding of a related area of contribution is frequently a cause for rejection.

  • Inadequate Level of Rigor and Shallow Concept Maturity: A desk rejection can often identify the lack of rigor or maturity within a paper at an early stage, but sometimes this decision cannot be made until the paper has been sent to reviewers for comment. Reviewers are charged with reading a paper at a much deeper level than the initial check by the EiC and assigned Editor, and the depth of a review may often reveal that the level of contribution in a new submission is not sufficient and fails to meet the expectation for a journal publication. Papers rejected for this reason can sometimes be resubmitted later after the paper has matured in its level of rigor.

  • Rejection after a Revision: Typically, the same set of reviewers who were assigned to an earlier submission are asked to review a resubmission. Reviewers of resubmissions often look carefully at the comments made in the “revision statement” that the authors submit as an accompaniment to the new version of their paper. Other journal editorials have discussed the task of preparing a revision [6] and the importance of the revision statement [7]. SoSyM authors are encouraged to look over [6, 7] for general suggestions for preparing a journal revision and the associated revision statement. Reviewers can often detect when an author does not spend the proper amount of time on the revision, and it often shows through a casual and uninspired revision statement. Lack of depth in a revision statement can frustrate a reviewer and lead to the impression that the author lacks the passion to see their paper through publication.

  • Systematic Literature Review or Survey that is not Well-structured: SoSyM is very interested in systematic literature reviews that provide a contribution through deep summary and synthesis of a specific modeling area. There is a very well-defined process and structure for writing systematic literature reviews, compared to a survey or review that seems random or unclear in focus and justification. SoSyM authors who desire to submit a paper that surveys a particular area of modeling are encouraged to learn about how to conduct a systematic literature review. A suggested reference that surveys systematic literature reviews in software engineering is the work of Kitchenham et al. [8].

  • Self-Plagiarism: The SoSyM web page provides specific details about what constitutes self-plagiarism and also offers general suggestions on the quality characteristics expected from a new SoSyM submission (http://www.sosym.org/submission/). Unfortunately, SoSyM has experienced cases of self-plagiarism, which usually are not detected until the paper has been sent out to reviewers (e.g., most instances of plagiarism are identified by reviewers who have seen the paper in another similar recent context and recognize the violation after they are deep into the review). A self-plagiarized paper wastes the valuable time of reviewers and results in a sequence of administration activities causing the EiCs to correspond with the representatives from the other venue to which a near-identical paper was submitted. In egregious cases, the supervisors of authors in violation of the self-plagiarism policy may be contacted.

We encourage potential authors of SoSyM papers to understand the potential reasons for rejection as they prepare a submission and suggest that future authors also consider some of the advice in the references listed below.

3 Content of this Issue

This issue contains a Theme Section on Integrated Formal Methods, with Einar Broch Johnsen and Luigia Petre as Guest Editors. Please find the introduction to the theme and the selected papers in the Guest Editorial. This issue also contains one Regular paper:

\(\bullet \) “Supporting Different Process Views through a Shared Process Model” by Jochen Küster, Hagen Völzer, Cédric Favre, Moisés Castelo Branco, and Krzysztof Czarnecki.