• Decrease font size
  • Return font size to normal
  • Increase font size
U.S. Department of Health and Human Services

Scientific Publications by FDA Staff

  • Print
  • Share
  • E-mail

Search Publications



Starting Date

Ending Date

Order by

Entry Details

Acad Radiol 2013 Jul;20(7):798-806

Statistical Power Considerations for a Utility Endpoint in Observer Performance Studies.

Abbey CK, Samuelson FW, Gallas BD


RATIONALE AND OBJECTIVES: The purpose of this investigation is to compare the statistical power of the most common measure of performance for observer performance studies, area under the ROC curve (AUC), to an expected utility (EU) endpoint. MATERIALS AND METHODS: We have modified a well-known simulation procedure developed by Roe and Metz for statistical power analysis in receiver operating characteristic (ROC) studies. Starting from a set of baseline simulations, we investigate the effects of three parameters that describe properties of the observers (iso-utility slope, unequal variance, and tendency to favor more aggressive or conservative actions) and three parameters that affect experimental design (number of readers, number of cases, and fraction of positive cases). RESULTS: The EU endpoint generally has good statistical power relative to AUC in our simulations. Of 396 total conditions simulated, EU had higher statistical power in 377 cases (95%). In 246 of these cases, EU power was 5 percentage points or more higher than AUC. In simulation runs evaluating the effect of the number of readers and cases on the baseline simulations, EU measure had equivalent power to AUC with fewer readers (9% to 28%) or fewer cases (18% to 41%). CONCLUSION: These simulation studies provide further motivation for considering EU in studies of screening mammography technology and they motivate investigations of utility in other diagnostic tasks.

Category: Journal Article
PubMed ID: #23611439 DOI: 10.1016/j.acra.2013.02.008
Includes FDA Authors from Scientific Area(s): Medical Devices
Entry Created: 2013-04-25 Entry Last Modified: 2014-11-18