The y-axis gives the cumulative proportion of reports written by respondents at or below each percentile. On the x-axis, respondent percentiles are based on the volume of reports they write. Notes: Figure shows the Lorenz curves of reports written by respondents, separated by journal type. Meanwhile, reports should primarily “help the editor reach an informed decision”, with limited emphasis placed on giving comments.įigure 3 Lorenz curves for reports written, separated by journal type As authors, what they most expect from peer review is “a reasonable and well-substantiated decision”, with lesser weight given to timeliness and receiving feedback. We also asked our respondents some general questions about the aims of peer review. The top 10% of the distribution dedicates 25 working days or more, which is quite substantial considering refereeing is usually unpaid. We estimate that the average (median) respondent spends 12 (9) working days per year on refereeing. There is heterogeneity in which journals respondents referee for: almost half of them write no reports for a top five journal, while the top 25% of respondents write approximately 80% of top five reports (Figure 3). The most active referees tend to be more senior and located in the US/Canada, and they often have editorial experience. Note: Figure shows the distribution of the annual number of reports written by our respondents. This distribution is similarly skewed, with around 15% of referees responsible for 40% of all reports written (Figure 2).įigure 1 Distribution of annual number of submissions As referees, our respondents wrote a mean (median) of 10.2 (8) reports per year. A rightward skew is visible in this distribution, with approximately 20% of authors making 50% of submissions (Figure 1). In the two years prior to the survey, our respondents made a mean (median) of 3.5 (3) journal submissions per year. We start by documenting the peer review activity of our respondents. Our findings are presented in greater detail in a recent report (Charness et al. In this column, we use our survey data to evaluate the peer review system in economics and consider a range of proposals potentially responsive to the issues identified. We asked about their recent submission and peer review activity, their perspectives as referees and authors, and their opinions about proposals for reform. Our respondents were fairly representative of the underlying population of academic economists (but deviated most notably on the dimensions of field and location, with more experimental/behavioural economists and researchers based in Europe). To investigate the current state of our system, we surveyed over 1,400 economists between July 2020 and January 2021 about their experiences with and attitudes towards peer review. This finding has particular salience in light of suggestions that economics should re-evaluate its norms around the production of research (Akerlof 2020 Andre and Falk 2021). For example, the use of pre-publication peer review might discourage the pursuit of daring research in favour of safer topics, relative to post-publication peer review (Gross and Bergstrom 2021). More broadly, peer review can systematically influence the research direction of the field. For individual researchers, top five publications are a key input into hiring and tenure decisions (Heckman and Moktan 2020). This omission is not immaterial, given the central role peer review plays in our discipline. Moreover, while recent efforts at understanding the publication process in economics have been fruitful (Galiani and Panizza 2020), there is still much we do not know about peer review in particular. Economics is a dynamic field, but its approach to peer review has remained relatively static.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |