Validation of a candidate instrument to assess image quality in digital mammography using ROC analysis

Joana Boita, Ruben E. van Engen, Alistair Mackenzie, Anders Tingberg, Hilde Bosmans, Anetta Bolejko, Sophia Zackrisson, Matthew G Wallis, Debra M. Ikeda, Chantal Van Ongeval, Ruud M. Pijnappel, Mireille Broeders, Ioannis Sechopoulos

Research output: Contribution to journalArticlepeer-review


To validate a candidate instrument, to be used by different professionals to assess image quality in digital mammography (DM), against detection performance results.
A receiver operating characteristics (ROC) study was conducted to assess the detection performance in DM images with four different image quality levels due to different quality issues. Fourteen expert breast radiologists from five countries assessed a set of 80 DM cases, containing 60 lesions (40 cancers, 20 benign findings) and 20 normal cases. A visual grading analysis (VGA) study using a previously-described candidate instrument was conducted to evaluate a subset of 25 of the images used in the ROC study. Eight radiologists that had participated in the ROC study, and seven expert breast-imaging physicists, evaluated this subset. The VGA score (VGAS) and the ROC and visual grading characteristics (VGC) areas under the curve (AUCROC and AUCVGC) were compared.
No large differences in image quality among the four levels were detected by either ROC or VGA studies. However, the ranking of the four levels was consistent: level 1 (partial AUCROC: 0.070, VGAS: 6.77) performed better than levels 2 (0.066, 6.15), 3 (0.061, 5.82), and 4 (0.062, 5.37). Similarity between radiologists’ and physicists’ assessments was found (average VGAS difference of 10 %).
The results from the candidate instrument were found to correlate with those from ROC analysis, when used by either observer group. Therefore, it may be used by different professionals, such as radiologists, radiographers, and physicists, to assess clinically-relevant image quality variations in DM.
Original languageEnglish
JournalEuropean Journal of Radiology
Publication statusPublished - 2021 Mar 30

Subject classification (UKÄ)

  • Medical Image Processing

Cite this