Foundations of Science Frequently Flawed at the Core

Researchers who didn't work on a particular paper judge whether or not it meets scientific standards. But some specialists have begun to wonder whether or not peer review fails to detect serious flaws in research. It's almost impossible to find out what happens in the vetting process; peer reviewers are unpaid, anonymous and unaccountable. Their reviews are kept confidential, making it difficult to determine their standards.

One-Third of Studies Have Been Questioned

Now, a study has found that almost one-third of peer-reviewed research articles have been either contradicted or seriously questioned, and some scientists are calling for changes in the system. They suggested that reviewers drop their anonymity and allow comments to be published, and some have additionally proposed that peer reviewers be paid to ensure consistent quality.

Dr. Drummond Rennie, deputy editor of the Journal of the American Medical Association, said of peer review, ''The more we look into it, the harder it is to prove whether it does good or bad." Rennie has called for a study of this issue. He has a personal policy of disclosing his name for his journal article reviews.

Anonymity: Benefit or Hindrance?

Other scientists disagreed with the criticisms, arguing that anonymity allows reviewers to give negative reviews without fearing damage to their future careers.

The peer review system has had many successes; it has saved scientists from publishing seriously flawed work or incorrect medical advice. But it lacks consistent standards; the expertise of the reviewers can vary widely from journal to journal. There is no organization that sets standards or guidelines for peer review.

Dubious Incidents

There have been many incidents that cast doubt on the validity of the current peer review process. In one study, researchers submitted a plagiarized paper to 110 journals, but only two noticed the problem. In another, researchers looked at 18 peer-reviewed papers written by researchers who had later admitted to scientific fraud. Sixteen of the papers were riddled with errors.