“Reliabilities of identifying positive selection by the branch-site and the site-prediction methods” is merely one piece of evidence indicating that TENS rests on much shakier empirical foundations than most scientists realize:
“Our finding means that hundreds of published studies on natural selection may have drawn incorrect conclusions,” said Masatoshi Nei, Penn State Evan Pugh Professor of Biology and the team’s leader…. To demonstrate the faultiness of the statistical methods, Nei’s team compiled data collected by their Emory University colleague, Shozo Yokoyama, on the genes that control the abilities of fish to see light at different water depths and on the genes that control color vision in a variety of animals. The team used these data to compare statistically predicted sites of natural selection with experimentally determined sites. They found that the statistical methods rarely predicted the actual sites of natural selection, which had been identified by Yokoyama through experiments.
“In some cases, statistical method completely failed to identify the true sites where natural selection occurred,” said Nei. “This particular exercise demonstrated the difficulty with which statistical methods are able to detect natural selection.”
To demonstrate how small sample sizes can lead to incorrect results, the team used computer simulations to examine the evolution of genes in three primates: humans, chimpanzees and macaques. The scientists mimicked the procedures used by the authors of a 2007 paper, which applied the branch-site method to 14,000 orthologous genes — genes that are genealogically identical among different species — and which found that the method predicted selection in 32 of the genes. Nei and his team also studied selection using Fisher’s exact test, but this test did not detect any selection.
“The results indicate that the number of nucleotide substitutions that occurred were too small to detect any selection; therefore, all of the 32 cases obtained by the branch-site method must be false positives,” said Nozawa.
“These statistical methods have led many scientists to believe that natural selection acted on many more genes in humans than it did in chimpanzees, and they conclude that this is the reason why humans have developed large brains and other morphological differences,” said Nei. “But I believe that these scientists are wrong. The number of genes that have undergone selection should be nearly the same in humans and chimps. The differences that make us human are more likely due to mutations that were favorable to us in the particular environment into which we moved, and these mutations then accumulated through time.”
Nei said that to obtain a more realistic picture of natural selection, biologists should pair experimental data with their statistical data whenever possible. Scientists usually do not use experimental data because such experiments can be difficult to conduct and because they are very time-consuming.
In other words, scientists usually do not bother to do science because it is difficult and it takes too long. But, non-scientists should still trust their results because on those rare occasions that scientists actually do make use of the scientific method, it works rather well. And, of course, one should trust biologists because physicists get amazingly accurate results.
Needless to say, the quality of their rationalizations tends to indicate that scientists don’t often make use of logic either.