Flaws discovered in oncology microarray studies
Scientists generally classify microarray analysis as a type of discovery-based research. As with hypothesis-based procedures, such research must incorporate sound principles of statistical analysis to yield the most accurate results and conclusions. Although some researchers dispute the reproducibility and validity of clinical studies based on microarrays, few have critically examined the statistical methods employed in them.
Dr. Alain Dupuy of the Université Paris VII Denis Diderot and Richard M. Simon of the National Cancer Institute in Bethesda, Md., have analyzed literature published from 2000 to 2004 in which microarray tests were correlated with clinical outcome and have concluded that studies described in both well- and lesser-known peer-reviewed journals contain fundamental flaws in their statistical methods and reporting.
Of the 90 studies the researchers selected initially, the majority dealt with hematologic, breast and lung cancers. An eligible study had to address one or more of the following outcomes: death, relapse and therapeutic response. For each, the scientists reported the journal’s impact factor (from 1 to 10), among other parameters.
For detailed analysis, the authors chose the 42 articles published in 2004. They divided the statistical analysis types found in them into the categories of outcome-related gene finding, class discovery and supervised prediction. Some studies fell into more than one of these groups.
The scientists found that nine of the 23 studies that reported outcome-related gene finding had unclear, inadequate or unstated methods to deal with false positives. Thirteen of the 28 studies in the class-discovery group claimed without adequate support that expression clusters could help distinguish different outcomes, even though the clustered genes were selected for their correlation with outcome in each case. Finally, 12 of the 28 studies that reported supervised prediction contained a biased estimation of prediction accuracy for binary outcomes.
In summary, one or more of these three major flaws was discovered in 50 percent of the publications — a figure that applied to the subset of studies from highly recognized journals as well.
The researchers concluded their analysis with a list of do’s and don’ts to help others design studies less prone to error and to promote discussion about interpreting and reporting on microarray-based research. Their work appears in the Jan. 17 Journal of the National Cancer Institute.
MORE FROM PHOTONICS MEDIA