The risks of large-scale testing, and the potential for false discovery, can be seen in the “discovery” of the genetic basis for anxiety and depression. Specifically, serotonin transporter gene 5-HTTLPR. Color Genomics sells a genetic testing product that supposedly can predict which anti-depressant drug works for which patient. This is just one of multiple genetic diagnostic tools whose popularity stems from dissatisfaction with existing drugs. As the Lancet put it in 2016:
“…suboptimal response to these therapies is typical and thought to be in part a result of genetic variation.”
For several years, a search has been underway to uncover links between genetics and mental disorders, links that might guide doctors in choosing the best drugs. The result seems to be a vivid illustration of the risks of being fooled by chance, risks that are multiplied many times over in large-scale genetic testing. There is a rich landscape of ephemeral chance results, waiting for people to find what they are looking for. Dozens of studies that supposedly found a genetic basis for depression were re-examined in large-scale this study last year – it was not able to confirm any of the findings. Derek Lowe, respected commentator on drug discovery and the pharmaceutical industry, wrote in Science News (There is no Depression Gene) that
… talk of a ‘depression gene’ is nonsense. This literature is all noise, all false positives, all junk. As you actually move to larger and larger studies, everything disappears, which is what noise does.
As genetic testing accelerates, a solid understanding of the statistical methodology to control false discovery is needed. In this depression gene case, a huge amount of research work was wasted, and diagnostic tests of minimal validity launched, for want of a solid statistical foundation.