That’s why I was so interested to see an article on Slashdot asking if academic journals are still relevant. Do they still ensure quality? Are they nimble enough? Can they compete with the information glut on the Internet? All these are really relevant questions. One reader expresses some of my own thoughts about anonymous peer reviews:
There are certainly parts of the peer review process that are less than ideal–reviewers don’t take the time to understand what they’re reviewing, or they have an emotional reaction to something that seems to undercut their fond hopes for how something will turn out and make stupid, picky attacks on a paper, or they realize that they’re about to get scooped and so ask for every pedantic little thing so they gain more time for their own work. But even with these flaws, the process does a pretty good job at rejecting junk; it just rejects a little too much non-junk, too, or at least makes the process more painful than necessary.
Interestingly, Slashdot itself has some of the properties one might want in a modern-age on-line academic journal: quick to respond to news, filtering data with the help of its readers, offering support for anonymous as well as non-anonymous comments (and favoring non-anonymous ones slightly). In that sense, it is also similar to Wikipedia, or in a different direction, the ArXiv, which both share much of these properties.
Yet all these examples also show the limits of the system. They are not recognized as offering the highest quality information (even if, in reality, they do most of the time), because bad information is shoved through them all the time and there is no guarantee that what you read will not be garbage. And, paradoxically, while they are openly democratic by construction, these outlets give enormous power (censorship, favoring certain view, or even rewriting contents) to a small circle of individuals.
So the question is: how would you fix these flaws? I have ideas, but I’m interested in hearing yours.