Introduction

Scientific progress is fueled by new ideas. While transforming new ideas into research outputs, scholars rely on their peers to identify weaknesses in their work, alternative models, new methodologies and databases that can improve their research quality. Considering the amount of time scholars devote to presenting draft versions of their papers at conferences and seminars, and discussing their new projects face-to-face with other colleagues, it is reasonable to expect that peers’ comments and suggestions will improve the quality of research. However, despite its alleged importance, empirical evidence on the effects of peers’ individual and collective comments on the quality of research is scarce. The goal of this paper is to fill this gap.

I first build a dataset of papers published in academic journals. Based on the acknowledgment sections of the papers, I record the list of scholars that provided comments, and the seminars and conferences at which the paper was presented. To obtain unbiased estimates on how these comments, seminars, and conferences contributed to the quality of a paper, I control for other quality-affecting variables that correlate with peer comments. There are two main confounding variables: the quality of the author and the quality of the research idea. First, it is assumed that high-quality authors produce high-quality papers. Minondo (2020) showed that high-quality scholars were more likely to be invited to present their works at seminars. Papers written by high-quality scholars are also more likely to be accepted at conferences. Furthermore, high-quality scholars may receive more comments on their work because they have more opportunities to interact with other scholars at seminars and conferences, or because their work is more likely to be followed. Second, it is assumed that high-quality papers are built upon high-quality research ideas. Scholars choose their most promising project when deciding which paper they will present at a research seminar and the draft they will send to colleagues. Hence, more promising projects tend to receive more feedback than do less-promising ones.

To control for the quality of the author, I leverage the quality of the affiliated institution. In some specifications, I also use author fixed effects. To control for the quality of the research idea, I use a feature of the job-placement process of economics PhD candidates. During their final academic year, future PhD graduates select a project from their contemporaneous research ideas for their job market paper. This becomes the tool PhD candidates use to show their research skills to potential employers. Because PhD candidates want to maximize job offers, they select their highest-quality project. Thus, the fact that a paper is selected for the job market paper provides a signal of the initial quality. I retrieved information from 2,113 PhD candidates, from top US economics departments, who entered the labor market between 2000 and 2018. I identify their job market papers and additional projects that they could also have selected as their job market papers. I then follow the job market papers and additional projects until they are published. These publications comprise my estimation sample.

Using the job-market status of a paper to control for the quality of the research idea and author’s fixed effect, I find that a one-standard-deviation increase in the number of individual comments is associated with a 16% increase in the ranking of the journal in which a paper is published. Additionally, a one-standard-deviation increase in the number of seminars is associated with a 27% increase in the ranking of the journal. Presenting a paper at a conference has no impact on the quality of the journal in which a paper is published, after I control for the number of individual comments and seminars. I find that comments given by high-quality scholars have a larger positive impact on quality than do comments received from non-top scholars. Likewise, presenting a paper at a top economics department has a larger positive impact on the quality of the journal in which a paper is published than does presenting a paper in a non-top economics department.

I argue that the positive link between peers’ individual and collective comments and the quality of the journal in which a paper is published occurs because informal collaborations improve the quality of a research project. Exchanges with peers can improve the quality of a research project: (i) providing inspiration for novel research ideas, (ii) making scholars aware of new databases, methodologies, and literature; (iii) identifying flaws and limitations in the research; (iv) framing the contribution of the paper in the literature; (v) providing critical advice on various parts of the research; and (vi) suggesting additional analyses and avenues for further research.

This paper is related to studies that analyzed whether peers’ individual and collective comments improve the quality and visibility of papers. Based on a sample of 251 articles published in the Review of Economics and Statistics, Laband and Tollison (2000) found a positive correlation between the number of scholars acknowledged in a paper and the number of citations received by it during the 6 years following publication. The positive impact was larger if the comment was provided by a high-quality scholar. Brown (2005) analyzed whether presentations at research seminars and conferences, and comments received from colleagues increased the likelihood of a paper receiving a revise-and-resubmit decision with an accounting journal. Individual and collective comments were also analyzed to determine whether there were increases in the numbers of citations received by papers published in three leading accounting journals. He found that presenting at research seminars was the only variable that positively correlated with receiving an invitation to revise and resubmit and the number of citations received. Rose and Georg (2021) replicated the previous two studies using a sample of 6,400 papers published in six finance journals. They found that the number of individual comments and research seminars were positively associated with the number of citations received by a paper, whereas presenting at conferences had no impact on quality. I add to this strand of literature by estimating the impact of individual and collective comments in a sample controlling for the initial quality of the research idea. Furthermore, my estimations include author fixed effects, which provide a better control for the quality of the author than those used by previous studies, thus neutralizing a scholar’s tendency to apply strategic acknowledgments. Finally, I analyze whether presenting at top economics departments contributes more to the quality of a paper than does presenting at non-top departments.

My paper is also linked with studies that analyzed how conferences and meetings contribute to the flow of ideas, to increase the probability of publication, and to enhance the visibility of a paper. Iaria et al. (2018) found that, during and after World War I, the ban on Central scientists from participating at international conferences was associated with a drop in citations between Allied and Central scientists. Using data from the Joint Mathematics Meetings between 1990 and 2009, Head et al. (2019) showed that a mathematician was more likely to cite the work of another if they attended the same conference. This probability increased if the two scholars attended the session in which the cited paper was presented. Using data from a major political-science conference that was canceled in 2012, Lopez de Leon and McQuillin (2020) concluded that the probability of a paper being cited increased by five percentage points over a period of 4 years if it was presented at the conference. These papers confirm that knowledge flows from conference presenters to conference attendees. However, as explored in this paper, it is likely that knowledge also flows from conference attendees to conference presenters through comments and suggestions. In line with this argument, Gorodnichenko et al. (2021) found that presenting a paper at a major conference in economics increased the probability of its publication in a high-quality journal. I also obtain this result. However, I also find that this positive association becomes statistically insignificant after I control for the number of individual comments received by a paper and the number of research seminars at which it was presented.

More broadly, this paper is related to the literature that explores how knowledge is produced (Stephan 2010; Fortunato et al. 2018) and, in particular, how peers contribute to that process. Azoulay et al. (2010); Waldinger (2012); Borjas and Doran (2015); Agrawal et al. (2017); Jaravel et al. (2018); Bosquet et al. (2021) analyzed how the premature death, migration, or arrival of scientists affected collaborators’ and other peers’ productivity. My paper contributes to this literature by analyzing another channel by which peers affect the quality of a scholar’s output: individual and collective feedback to ongoing research projects. My finding that peers’ feedback has a large positive effect on the quality of the journal in which a paper was presented agrees with Oettl (2012), who found that a scholar’s output quality decreased after the death of a co-author if the co-author was helpful to other colleagues. Rose and Georg (2021) found that authors that had a central position in the network of informal collaborations were very productive, their papers were published in high-quality journals, and received many citations.

The remainder of the paper is organized as follows. Section Data describes the dataset and presents some summary statistics. Section Regression results discusses the results of the regression analyses, and Section Conclusions concludes.

Data

The sample comprised PhD candidates from the top 40 US economics departments who entered the labor market between 2000 and 2018. To identify the top US economics departments, I used the ranking elaborated by Ideas.Footnote 1

In their last academic year, PhD candidates in economics go to the so-called job market. In the US, the job market runs parallel to the American Economic Association’s Annual Meeting; in Europe, it runs parallel to the European Winter Meetings of the Econometric Society. At these job markets, PhD candidates in economics have interviews with potential employers from academia, research institutions, and private firms. The interviews are organized around the PhD candidate’s job market paper. This paper is usually one of the three chapters that the candidate wrote for her doctoral dissertation. Interviewers will ask the candidate to describe her job market paper and will pose questions about it. If the interview is successful, the PhD candidate will receive an invitation to present her job market paper at the potential employer’s venue.

Since the job market paper plays such a big role in the hiring process of PhD graduates in economics, candidates will select their best piece of research as their job market paper. Specifically, the job market paper is the research project that provides the most novel contribution to literature and better demonstrates the candidate’s skills to perform state-of-the art research among the contemporaneous research projects conducted by the candidate.

Each year, during the fall term, economics departments announce their job market candidates. From the department’s web page, I recorded each PhD candidate’s job market paper and the projects that she could also have selected as job market paper. These were projects whose sole author was the PhD candidate, or ones written with other PhD students. As a general rule, I excluded papers co-authored with scholars already having a PhD. I only included a paper written with a senior scholar if the job market paper was written with the same senior scholar. I followed the job market paper and the papers that could also have been selected as a job market paper until they were published. I define the maturation time of a paper as the difference between the year in which a paper was published and the year at which the PhD candidate entered the labor market.

Based on the acknowledgment sections of the published version of the paper, I retrieved information about the number of research seminars and conferences at which a paper was presented, and the scholars that provided comments. Following Cronin (1995) classification, I sought to capture concept-related acknowledgments, which included scholars who mentored the PhD candidate, and others that also provided inspiration, insights, critical analysis and comments. Therefore, I excluded acknowledgments for research assistance, data sharing, or data-access facilitation.Footnote 2

However, acknowledgments may not always reflect peers’ contribution to a paper. Authors may add high-quality scholars to the acknowledgment section, even when their contribution is very small, because potential reviewers and editors are familiar with their names. Furthermore, authors may use the acknowledgment section to be nice with scholars that can positively influence their career. However, this type of strategic acknowledgment does not seem to be a problem in my sample. First, my sample comprises PhD-candidate papers. Because these candidates graduated from the top US economics departments, there are many high-quality scholars’ names in their papers’ acknowledgment sections. Therefore, the presence of a high-quality scholar in the acknowledgment section does not cause a paper to stand out in the sample. Second, if the only motivation to acknowledge is to gain favor, there will be no positive correlation between individual comments and the paper quality. Contrarily, this correlation is positive and very strong. Furthermore, previous studies found that the positive correlation between acknowledgments and the quality and visibility of a paper was robust to controlling for potential strategic acknowledgments (Laband and Tollison 2000; Rose and Georg 2021) Third, some estimations use author fixed effects, which neutralize scholars’ tendency for narcissism and gratuitous acknowledgments.

Another concern is that scholars may want to present their papers at research seminars and conferences because potential editors or reviewers will be in attendance. Thus, they seek a more positive view following attendance. However, in my sample, the median paper was only presented at one seminar. Hence, it does not seem that scholars could choose their venues to maximize encounters with potential reviewers and editors. Finally, I measure the effect of individual and collective comments on the quality of research conditional on the paper being published. Hence, my results are only applicable to the set of published papers.Footnote 3

Table 7 in the Appendix reports the economics departments and the PhD candidate cohorts included in the sample. It also reports, for each PhD program, the number of graduates from which I could retrieve information and the number of potential job market paper projects that became journal articles. There were differences in the number of PhD candidate cohorts included in the sample across US economics departments. Those differences can be explained by the possibility of accessing the information of old cohorts. Economics departments provide information about the PhD candidates that enter the labor market during a particular year. A few departments also provide links to previous years’ job market candidates. To retrieve information for older cohorts, I used the Internet Archive Library. In some cases, the library contained fairly complete records of the different versions of the website over time. However, in many cases, the information was scant, or there was no copy archived. This explains why I could retrieve information for very old PhD candidates (i.e., 2000) for some economics departments (e.g., UC Berkeley or MIT), whereas I could only retrieve information about the most recent cohort for others (e.g., Ohio State).

I measured the quality of a paper using the Scimago Journal Ranking (SJR) of the journal in which it was published.Footnote 4 I prefer this measure over number-of-citations because of the large heterogeneity in the age of publications in my sample. The mean, median, and standard deviation of the (log) quality of a journal is 1.1. The minimum log quality of a journal is −2.1 and the maximum 3.4. I define author quality as the capacity of a scholar to generate new ideas and transform them into articles that are published in high-ranked journals. Similar to Smeets et al. (2006), I measured the quality of a PhD candidate by the quality of placement after graduation.Footnote 5 If a paper had multiple authors, I added individual author qualities. It is important to point out that this proxy for author(s) quality may have some noise because other factors, such as preferences or family-related issues, can determine the choice of placement. In some specifications, I controlled for the quality of the author(s) using author fixed effects. I computed the individual comments received by counting the scholars listed in the acknowledgment sections. I also computed the number of comments given by top-10 scholars.Footnote 6 I also counted the seminars and conferences at which a paper was presented and the seminars given at the top-10 economics institutions.Footnote 7 Following Gorodnichenko et al. (2021), I used the American Economic Association, the European Economic Association, and the Royal Economic Society as top conferences. I further added the National Bureau of Economic Research (NBER) Summer Institute seminars to this group.

Table 1 Information about the sample
Full size table

Table 1 provides information about the construction and characteristics of the estimation sample. I retrieved information from 2,113 PhD candidates that entered the job market between 2000 and 2018. These job market candidates were working on 5,175 projects that could have been selected as job market papers. Among those projects, 2,116 were selected as job market papers.Footnote 8 By July 2020, 581 of the job market candidates (27%) had published their job market paper or another they could also have selected as a job market paper in a journal included in the SJR in the year of publication.Footnote 9 A total of 858 out of 5,175 potential papers, 17%, was published. 48% of these publications were job market papers. This percentage is larger than the share of job market papers among the potential projects (41%). 18% of publications had more than one author, and 15% were published in a top-5 economics journal.Footnote 10 I retained information about the number of individual comments for all papers in the sample. However, there were some articles that used formulaic acknowledgments, such as “we acknowledge numerous seminar participants”, “several audiences”, or “seminar and conference participants”. Because the number of seminars could not be computed for these publications, the main estimation sample dropped from 858 to 719 articles.

Table 2 Summary statistics of the estimation sample
Full size table

Table 2 reports summary statistics on the individual and collective comments received by publications. It provides statistics for all publications, published job market papers, and other publications. The median publication received nine individual comments. The distribution was not skewed: the average was 10 and the standard deviation was seven. The minimum number of comments received by a publication was zero, whereas the maximum was 49. There were 66 publications out of 719 with no individual comments. The median publication was presented at one seminar only. The maximum number of seminars at which a publication was presented was 27. There were 280 publications out of 719 that were not presented at any seminar. Note that the distribution of seminars per publication was skewed, because the average number of presentations was much larger than the median. Finally, the median publication was not presented at any conference. The average was one and the standard deviation is 2. There is a paper that was presented at 17 different conferences, whereas 405 publications, out of 719, were not presented at any conference. Job market papers received more individual comments and were presented at more research seminars than were non job market papers. Specifically, the median job market paper received five more individual comments and was presented at three more seminars and one more conference than was a non job market paper.

Fig. 1
figure 1

Note: The quality of the journal is measured by the (log) Scimago Journal Ranking

Relationship between peers’ comments and journal quality

Full size image

Panel A of Fig. 1 plots the relationship between the number of individual comments received by a paper and the quality of the journal in which a paper was published. To analyze whether there was a non-linear relationship between these variables, I distributed the papers into individual comment quintiles. The (log) quality of the publication rises almost linearly with quintiles. A similar trend is observed in research seminars (Panel B). Regarding conferences, in panel C, an increase is revealed in the (log) quality of the journal in which the paper was published if the paper was presented in, at least, one conference. The difference in quality between presenting in one or three conferences is not very large, but it rises when the paper was presented at more than three conferences.

These figures suggest that peers’ individual and collective comments improved the quality of the journal in which a paper was published. However, these correlations might also capture the positive associations between scholar quality and the number of individual or collective comments received from peers. They may also capture the quality of the research idea and the comments received from peers. In the next section, I explore the contributions of individual and collective comments to the quality of the journal in which a paper was published after I control for the qualities of the author’s placement and the research idea.

Regression results

To estimate the effect of peers’ individual and collective comments on the quality of the journal in which a paper was published, I applied the following regression equation:

$$\begin{aligned} \begin{aligned} \ln Q_{jt}=\beta _{1} \ln (I_{p}+1)+\beta _{2} \ln (S_{p}+1)+\beta _{3} \ln (C_{p}+1)+\beta _{4} \ln A_{p}+\beta _{5} M_{p}+\gamma _{d}+\lambda _{t}+\epsilon _{jt} \end{aligned} \end{aligned}$$
(1)

where \(Q_{jt}\) is the quality of journal j at the time the paper was published (t). \(I_{p}\) is the number of individual comments received by paper p, and \(S_{p}\) and \(C_{p}\) are the numbers of seminars and conferences at which paper p was presented, respectively. Because the number of individual comments, seminars, and conferences enter as logs in Eq (1), I add 1 to the number of comments, seminars, and conferences variables to keep observations having zero values in the estimation sample.Footnote 11 Note that \(A_{p}\) represents the quality of the author’s placement and \(M_{p}\) is an indicator variable that turns 1 if the paper was a job market paper. \(\epsilon _{jt}\) is the disturbance term.

Because some economics departments may have more social ties with journal editors than do others (Colussi 2018), I controlled for the economics department at which the candidate received a PhD (\(\gamma _{d}\)). Owing to the time elapsing between paper submission and acceptance for publication at a journal, papers from “younger” PhD candidate cohorts were less likely to be included in the estimation sample. This can create a sample selection problem for the dependent variable. To address this problem, I introduced cohort fixed effects (\(\lambda _{t}\)). They also control for other cohort-specific factors that may affect the probability of publication in a high-quality journal, such as the quality of other PhD candidates that entered the job market in the same year, or the number of PhD candidates that decided to pursue an academic career. Following Laband and Tollison (2000) and Rose and Georg (2021), I use ordinary least squares to estimate Eq (1).

Table 3 presents the estimates for the impact of the number of comments given by peers’ individually and collectively at research seminars and conferences.Footnote 12 I clustered standard errors at the author level.Footnote 13 First, I estimated Eq (1) using the number of individual comments only (column 1 of Table 3). This estimation uses the full sample of publications: 858. The \(\ln\) Comment coefficient is positive, indicating that receiving more individual comments is positively correlated with publishing in a high-ranked journal. For example, a one-standard-deviation increase in the number of comments (seven comments), for a paper that received an average number of comments (10 comments), is associated with a 28% increase in the quality of the journal in which a paper was published [(ln(17)-ln(10))*0.535]. This increase elevates a paper published in a journal located in the 2nd quartile of the SJR Economics and Econometrics category (e.g., CESifo Economics Studies; SJR 2019 score: 0.851) to a journal located in the 1st quartile (e.g., Journal of Industrial Economics, SJR 2019 score: 1.059).

Table 3 Contribution of individual comments, seminars, and conferences to the quality of a paper
Full size table

In column 2 of Table 3, the number of seminars was the only independent variable. Note that the number of observations was lower than in column 1, because, as mentioned above, there were some papers that did not provide a valid list of seminars. As expected, presenting a paper at a research seminar is positively correlated with publishing a paper at a high-ranked journal. For example, a one-standard-deviation increase in the number of presentations (five seminars) for a paper that was presented at the average number of research seminars (three seminars) is associated with a 42% increase in the quality of the journal in which a paper is published [(ln(8)-ln(3))*0.426]. There was a positive correlation between the number of conferences in which a paper was presented and the quality of the journal in which a paper was published (column 3). Specifically, a one-standard-deviation increase in the number of conferences (two conferences) for a paper that was presented at the average number of conferences (one conference) is associated with a 43% increase in the quality of the journal [(ln(3)-ln(1))*0.392].

Column 4 presents the results when the specification included all peers’ contribution variables: individual comments, research seminars, and conferences. The \(\ln\) Comment and the \(\ln\) Seminar coefficients remained positive, but had a lower point value than in previous estimations. This result indicates that there was a positive correlation between the number of individual comments received by a paper and the number of seminars and conferences at which it was presented. Interestingly, the conference coefficient was close to zero. This result indicates that the positive association between the number of conferences at which a paper was presented and the quality of the journal in which it was published disappeared after I controlled for the number of individual comments received and seminars at which it was presented. According to the coefficients reported in column 4, a one-standard-deviation increase in the number of individual comments and research seminars for a paper having an average number of comments and seminars is associated with a 48% increase in the quality of the journal in which a paper was published [(ln(17)-ln(10))*0.363 + (ln(8)-ln(3))*0.292].

In column 5, I introduced the quality of the author’s placement as an additional regressor. As expected, it is positively correlated with the quality of the journal in which a paper was published. There is also a reduction in the \(\ln\) Comment and \(\ln\) Seminar coefficients’ point estimates, suggesting that these coefficients captured the positive correlation between the quality of the author’s placement and journal.Footnote 14 Column 6 presents the results when I controlled for the quality of the research idea. The job market paper coefficient was positive and statistically significant. According to the coefficient reported in column 6, the quality of journals in which job market papers were published was, on average, 70% higher than the quality of the journals in which the rest of projects were published (exp .530). The \(\ln\) Comment and \(\ln\) Seminar coefficients remain positive, but with lower point values, specially for \(\ln\) Seminar, than in column 5. This is consistent with the argument that scholars choose to present their most promising projects when they are invited to give a research seminar. Even when I controlled for the quality of the author’s placement and research idea, a one-standard-deviation increase in the number of comments and seminars for a paper with average values of these variables still is associated with a 31% increase in the quality of the journal in which a paper was published [(ln(17)-ln(10))*0.318 + (ln(8)-ln(3))*0.144].Footnote 15

Finally, column 7 reports the estimations from the regression equation including author fixed effects. This estimation controlled for all author-specific variables, including the capacity to transform new ideas into high-quality publications, the “contribution-threshold” to include a peer in the acknowledgment section, the tendency for strategic acknowledgment, and the author’s willingness to present at seminars or conferences.Footnote 16 In this specification, I identify peers’ contribution to the quality of a paper using the variation in the number of individual and collective comments among papers written by the same author during the same period, whose initial quality was identified by the author.

The sample used in column 7 only included scholars that published more than one of the projects that were initiated during doctoral studies. This led to a large reduction in the number of observations.Footnote 17 Despite this drop, and the increase in standard errors, the \(\ln\) Comment and \(\ln\) Seminar coefficients remained positive and statistically significant. A one-standard-deviation increase in the number of comments for a paper that received the average number of comments is associated with a 16% increase in the quality of the journal in which a paper was published [(ln(17)-ln(10))*0.310]. A one-standard-deviation increase in the number of seminars for a paper that was presented at the average number of seminars is associated with a 27% increase in the quality of the journal [(ln(8)-ln(3))*0.271]. For example, the combined effect of these increases (43%) upgraded a paper published in Review of Economics and Statistics (SJR 2019: 8.363) to the quality level of The American Economic Review (SJR 2019: 11.889). Presenting at conferences was not associated with a raise in the quality of the journal in which a paper was published.

Table 4 Peers’ contribution by quality
Full size table

In previous estimations, I assumed that all individual and collective comments contributed equally to the increase in the quality of a paper. However, Laband and Tollison (2000) showed that the positive effect of individual comments on the number of cites a paper receives rises if those comments were made by reputed scholars. It also seems reasonable to expect that comments received at presentations at top economics institutions or leading conferences should contribute more to improve the quality of the journal in which a paper was published.

Table 4 presents the results from individual and collective comments that were distinguished by quality. Column 1 shows that comments given by top-10 scholars had a much larger positive correlation with the quality of the journal in which a paper was published than did comments offered by other scholars. Column 2 reports that presenting a paper at a top-10 economics institution had a strong positive association with publishing a paper at a high-ranked journal. However, presenting a paper at a non-top-10 economics institution had no correlation with publication quality. Presenting a paper at a major economics conference (e.g., American Economic Association, European Economic Association, the Royal Economic Society, and NBER Summer Institute) had a strong positive correlation with the quality of the journal in which a paper was published. The coefficient for rest of conferences was also positive, although its point value was lower than the one for top conferences. The quality of the author’s placement (column 5) and the job market status of a paper (column 6) were positively associated with the quality of the journal in which a paper was published. In these specifications, comments offered by top scholars, giving a seminar at top departments, or presenting a paper at a leading conference had a stronger positive association with the quality of the journal in which a paper is published than comments by non-top scholars or presenting a paper at non-top departments or conferences. When controlling for author fixed effects (column 7), comments given by top-10 scholars had a larger positive association with the quality of the journal in which a paper was published than did comments given by non-top scholars. I found that presenting a paper at a top-10 economics institution was positively associated with the quality of the journal in which a paper was published. However, presenting a paper at a non-top economics institution was not correlated with the quality of the journal in which a paper was published. The coefficient for top conferences was positive, but it was statistically not different from zero. The coefficient for rest of conferences was negative, but it was statistically not different from zero.Footnote 18

Table 5 Contribution of peers’ individual comments to the quality of a paper. Full sample
Full size table

As explained above, there is an important number of publications (139 out of 858) that acknowledged the comments received by participants at research seminars and conferences but did not list the institutions at which these seminars were held, or the name of the conferences. To test the robustness of my results, I re-estimated all specifications with the whole sample (858 observations instead of 719), removing the number of seminars and the number of conferences variables from the regression equation. The estimates for the \(\ln\) Comment coefficient should be taken with caution. Because the number of individual comments was correlated with the number of seminars and conferences, the \(\ln\) Comment coefficient captured the effect that seminars and conferences had on the quality of the journal in which a paper was published. Table 5 confirmed that individual comments had a strong positive association with the quality of the journal in which a paper was published (columns 1–4). Estimates also confirmed that the individual comments given by top scholars had a stronger positive association with the quality of the journal in which a paper was published than did comments provided by non-top scholars (columns 5–8).

Because a paper has more opportunities to be presented at research seminars and more feedback can be received from peers the longer the period that elapses between the first draft and publication, peers’ comments estimates can be capturing the effort that authors put into the paper.Footnote 19 As explained in Sect. 2, I define a paper’s maturity period as the difference between the year a paper was published and the year at which the PhD candidate entered the labor market. Table 8 in the Appendix reports the results. When the specification did not include author fixed effects (column 1), maturation time was positive and statistically significant. However, the \(\ln Comment\) and \(\ln Seminar\) coefficients were very similar to those reported in column 6 of Table 3. This result indicates that these coefficients did not capture the effect that a longer period between the first draft and publication had on the quality of the journal in which a paper is published. Furthermore, the maturation-time coefficient became zero when author fixed effects were used (column 2 of Table 8).

Conclusions

A scholar’s knowledge is limited and, therefore, one cannot be aware of all the elements that may contribute to improve the quality of research. To discover these elements, the author relies on peers, who, at research seminars, conferences, or through conversations, identify limitations in the research project and suggest avenues of improvement. In this paper, I analyzed the extent to which these comments and suggestions improved the quality of research. Because the number of suggestions a paper receives is not independent from the quality of the research idea and author, I used a sample of papers that enabled me to control for these variables: the research projects of job market candidates in economics. I found that a one-standard-deviation increase in the number of individual comments and research seminars is associated with a 43% increase in the quality of the journal in which a paper was published. I found that comments provided by top scholars had a stronger association with the quality of the journal in which a paper is published than did comments given by non-top scholars. I also showed that, although presenting a paper at top economics institutions had a strong positive association with the quality of the journal in which a paper is published, presenting at non-top economics institutions had no association. I further found that presenting at conferences, even at the top ones, was not associated with publishing in a high-ranked journal, after controlling for the number of individual comments and seminars.

My results provide suggestive evidence that peers’ individual and collective comments have a large positive effect on the quality of research projects, especially when they come from top scholars or are received when presenting a paper at a top economics institution. From a policy perspective, these results justify the use of public funding to organize research seminars, provide venues for scholarly interaction, and finance stays at top economics institutions.