• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

Scientific Publications by FDA Staff

  • Print
  • Share
  • E-mail

Search Publications



Starting Date

Ending Date

Order by

Entry Details

Acad Radiol 2012 Dec;19(12):1508-17

Multi-reader ROC studies with split-plot designs: a comparison of statistical methods.

Obuchowski NA, Gallas BD, Hillis SL


RATIONALE AND OBJECTIVES: Multireader imaging trials often use a factorial design, in which study patients undergo testing with all imaging modalities and readers interpret the results of all tests for all patients. A drawback of this design is the large number of interpretations required of each reader. Split-plot designs have been proposed as an alternative, in which one or a subset of readers interprets all images of a sample of patients, while other readers interpret the images of other samples of patients. In this paper, the authors compare three methods of analysis for the split-plot design. MATERIALS AND METHODS: Three statistical methods are presented: the Obuchowski-Rockette method modified for the split-plot design, a newly proposed marginal-mean analysis-of-variance approach, and an extension of the three-sample U-statistic method. A simulation study using the Roe-Metz model was performed to compare the type I error rate, power, and confidence interval coverage of the three test statistics. RESULTS: The type I error rates for all three methods are close to the nominal level but tend to be slightly conservative. The statistical power is nearly identical for the three methods. The coverage of 95% confidence intervals falls close to the nominal coverage for small and large sample sizes. CONCLUSIONS: The split-plot multireader, multicase study design can be statistically efficient compared to the factorial design, reducing the number of interpretations required per reader. Three methods of analysis, shown to have nominal type I error rates, similar power, and nominal confidence interval coverage, are available for this study design.

Category: Journal Article
PubMed ID: #23122570 DOI: 10.1016/j.acra.2012.09.012
Includes FDA Authors from Scientific Area(s): Medical Devices
Entry Created: 2012-12-31 Entry Last Modified: 2014-11-18