In a working paper entitled "Can We Test for Bias in Scientific Peer Review?", Andrew Oswald proposes a method of detecting whether journal editors (and the peer review process generally, I suppose) discriminate against certain kinds of authors. His approach, in a nutshell, is to look for discrepancies between the editor's comparison of two papers and how those papers were ultimately compared by the scholarly community (based on citations). In tests he runs on two high-ranking American economics journals, he doesn't find a bias by QJE editors against authors from England or Europe (or in favor of Harvard authors), but he does find that JPE editors appear to discriminate against their Chicago colleagues.Eggers goes on to point out that it's not entirely obvious what citation counts, and hence the approach, are measuring. Oswald's approach of course rests on the assumption that citations provide an unbiased measure of the quality of a paper which is probably not true. Also, Oswald uses within-journal paper order as a signal of the editor’s assessment of quality. Again this may not be true. Still, the technique and results are interesting.
(HT: Organizations and Markets.)
No comments:
Post a Comment