The Journal of Molecular Diagnostics Report
By Timothy J. O'Leary, MD, PhD
The Journal of Molecular Diagnostics
Submissions to The Journal of Molecular Diagnostics (JMD) often consist of a comparison of two analytic methods for identifying a single genetic polymorphism or mutation. Ideally both methods are compared with a single reference method (such as sequencing). When submitting manuscripts, authors of these articles often assert superiority of one method over another, but are disappointed to find that, when confidence intervals for sensitivity and specificity have been calculated, the claims of superiority prove unfounded. Many of these disappointments may be avoided by performing a “statistical power analysis” to determine sample size prior to undertaking any experiments at all!
The calculation of sample size typically requires an understanding of four factors.
- The required significance level, α. This is typically set at 0.05.
- The desired statistical power (1-β). Typically the statistical power is set someplace between 0.8 and 0.95. The statistical power should be reported in the manuscript.
- The population prevalence of the mutation/polymorphism.
- The smallest difference between test performance that one wishes to detect at significance (α) and power (1-β).
When one performs a power analysis, one often finds that, with the small differences to be found among molecular diagnostic tests, the number of samples required to detect superiority of one test over another runs into the hundreds. Most studies that are submitted to JMD are inadequately powered to sustain claims of superior sensitivity or specificity for detection of a mutation or polymorphism. Indeed, they generally do not have the statistical power to claim “non-inferiority” to the predicate device.
While it is not currently Journal policy to require that authors conduct a power analysis prior to submitting papers to JMD, I personally urge authors to do just that. While there are calculators available on the internet, only investigators with an unusual degree of experience and sophistication should use them without first consulting with a biostatistician who has demonstrated expertise in comparing clinical laboratory tests. Careful power analysis will enable investigators to determine whether they have the resources to truly demonstrate that the latest and greatest laboratory-developed test or in vitro diagnostic (IVD) is truly superior, from an analytic perspective, to the old standby that has been working for years.